łň 3ŇÇIc@s4ddkZddkZddklZddkZddkTddklZd„Zde fd„ƒYZ d„Z d d d d „Z ei d ƒZd"d d„Zd„Zd„Zei deiƒZei dƒZd„Zdddddddddg d d„Zd „Zed!jo eƒndS(#i˙˙˙˙N(tTree(t*(taccuracycCs_g}g}xC|D];}|i|iƒƒ}|t|ƒ7}|t|ƒ7}qWt||ƒS(s… Score the accuracy of the chunker against the gold standard. Strip the chunk information from the gold standard and rechunk it using the chunker, then compute the accuracy score. @type chunker: C{ChunkParserI} @param chunker: The chunker being evaluated. @type gold: C{tree} @param gold: The chunk structures to score the chunker on. @rtype: C{float} (tparsetflattenttree2conlltagst _accuracy(tchunkertgoldt gold_tagst test_tagst gold_treet test_tree((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyRs t ChunkScorecBs†eZdZd„Zd„Zd„Zd„Zd„Zdd„Zd„Z d „Z d „Z d „Z d „Z d „Zd„ZRS(sč A utility class for scoring chunk parsers. C{ChunkScore} can evaluate a chunk parser's output, based on a number of statistics (precision, recall, f-measure, misssed chunks, incorrect chunks). It can also combine the scores from the parsing of multiple texts; this makes it signifigantly easier to evaluate a chunk parser that operates one sentence at a time. Texts are evaluated with the C{score} method. The results of evaluation can be accessed via a number of accessor methods, such as C{precision} and C{f_measure}. A typical use of the C{ChunkScore} class is:: >>> chunkscore = ChunkScore() >>> for correct in correct_sentences: ... guess = chunkparser.parse(correct.leaves()) ... chunkscore.score(correct, guess) >>> print 'F Measure:', chunkscore.f_measure() F Measure: 0.823 @ivar kwargs: Keyword arguments: - max_tp_examples: The maximum number actual examples of true positives to record. This affects the C{correct} member function: C{correct} will not return more than this number of true positive examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure) - max_fp_examples: The maximum number actual examples of false positives to record. This affects the C{incorrect} member function and the C{guessed} member function: C{incorrect} will not return more than this number of examples, and C{guessed} will not return more than this number of true positive examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure) - max_fn_examples: The maximum number actual examples of false negatives to record. This affects the C{missed} member function and the C{correct} member function: C{missed} will not return more than this number of examples, and C{correct} will not return more than this number of true negative examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure) - chunk_node: A regular expression indicating which chunks should be compared. Defaults to C{'.*'} (i.e., all chunks). @type _tp: C{list} of C{Token} @ivar _tp: List of true positives @type _fp: C{list} of C{Token} @ivar _fp: List of false positives @type _fn: C{list} of C{Token} @ivar _fn: List of false negatives @type _tp_num: C{int} @ivar _tp_num: Number of true positives @type _fp_num: C{int} @ivar _fp_num: Number of false positives @type _fn_num: C{int} @ivar _fn_num: Number of false negatives. cKsÁtƒ|_tƒ|_tƒ|_tƒ|_tƒ|_|iddƒ|_|iddƒ|_|iddƒ|_ |iddƒ|_ d|_ d|_ d|_ d|_t|_dS(Ntmax_tp_examplesidtmax_fp_examplestmax_fn_examplest chunk_nodes.*i(tsett_correctt_guessedt_tpt_fpt_fntgett_max_tpt_max_fpt_max_fnt _chunk_nodet_tp_numt_fp_numt_fn_numt_counttFalset_measuresNeedUpdate(tselftkwargs((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt__init__rs         cCsŠ|io||i|i@|_|i|i|_|i|i|_t|iƒ|_t|iƒ|_t|iƒ|_ t |_ndS(N( R"RRRRRtlenRRRR!(R#((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt_updateMeasuresƒs cCs^|it||i|iƒO_|it||i|iƒO_|id7_t|_dS(s] Given a correctly chunked sentence, score another chunked version of the same sentence. @type correct: chunk structure @param correct: The known-correct ("gold standard") chunked sentence. @type guessed: chunk structure @param guessed: The chunked sentence to be scored. iN(Rt _chunksetsR RRtTrueR"(R#tcorrecttguessed((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pytscores !!cCsD|iƒ|i|i}|djodSnt|iƒ|SdS(s @return: the overall precision for all texts that have been scored by this C{ChunkScore}. @rtype: C{float} iN(R'RRtfloat(R#tdiv((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt precisions cCsD|iƒ|i|i}|djodSnt|iƒ|SdS(s @return: the overall recall for all texts that have been scored by this C{ChunkScore}. @rtype: C{float} iN(R'RRR-(R#R.((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pytrecall¨s gŕ?cCs\|iƒ|iƒ}|iƒ}|djp |djodSnd||d||S(sÍ @return: the overall F measure for all texts that have been scored by this C{ChunkScore}. @rtype: C{float} @param alpha: the relative weighting of precision and recall. Larger alpha biases the score towards the precision value, while smaller alpha biases the score towards the recall value. C{alpha} should have a value in the range [0,1]. @type alpha: C{float} ii(R'R/R0(R#talphatptr((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt f_measurełs   cCs<|iƒt|iƒ}g}|D]}||dq$~S(sÔ @rtype: C{list} of chunks @return: the chunks which were included in the correct chunk structures, but not in the guessed chunk structures, listed in input order. i(R'tlistR(R#tchunkst_[1]tc((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pytmissedĆs cCs<|iƒt|iƒ}g}|D]}||dq$~S(sÔ @rtype: C{list} of chunks @return: the chunks which were included in the guessed chunk structures, but not in the correct chunk structures, listed in input order. i(R'R5R(R#R6R7R8((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt incorrectŃs cCs2t|iƒ}g}|D]}||dq~S(sŸ @rtype: C{list} of chunks @return: the chunks which were included in the correct chunk structures, listed in input order. i(R5R(R#R6R7R8((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyR*ÜscCs2t|iƒ}g}|D]}||dq~S(sŸ @rtype: C{list} of chunks @return: the chunks which were included in the guessed chunk structures, listed in input order. i(R5R(R#R6R7R8((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyR+ĺscCs|iƒ|i|iS(N(R'RR(R#((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt__len__îs cCsdt|ƒ dS(sf @rtype: C{String} @return: a concise representation of this C{ChunkScoring}. s(R&(R#((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt__repr__ňscCs:dd|iƒdd|iƒdd|iƒdS(sJ @rtype: C{String} @return: a verbose representation of this C{ChunkScoring}. This representation includes the precision, recall, and f-measure scores. For other information about the score, use the accessor methods (e.g., C{missed()} and C{incorrect()}). sChunkParse score: s Precision: %5.1f%% ids Recall: %5.1f%% s F-Measure: %5.1f%%(R/R0R4(R#((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt__str__ůs '(t__name__t __module__t__doc__R%R'R,R/R0R4R9R:R*R+R;R<R=(((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyR 4s=     cCsšd}g}x|D]y}t|tƒoYti||iƒo)|i||ft|iƒƒfƒn|t|i ƒƒ7}q|d7}qWt |ƒS(Nii( t isinstanceRtretmatchtnodetappendttupletfreezeR&tleavesR(tttcountRtposR6tchild((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyR( s)tNPtSt/c Cs|tidƒ}t|gƒg}x#|i|ƒD]}|iƒ}|ddjo^t|ƒdjotd|iƒƒ‚nt|gƒ}|di|ƒ|i|ƒq1|ddjo;t|ƒdjotd |iƒƒ‚n|i ƒq1|t jo|di|ƒq1|dit i i i||ƒƒq1Wt|ƒdjotd t|ƒƒ‚n|dS( sš Divide a string of bracketted tagged text into chunks and unchunked tokens, and produce a C{Tree}. Chunks are marked by square brackets (C{[...]}). Words are delimited by whitespace, and each word should have the form C{I{text}/I{tag}}. Words that do not contain a slash are assigned a C{tag} of C{None}. @return: A tree corresponding to the string representation. @rtype: C{tree} @param s: The string to be converted @type s: C{string} @param chunk_node: The label to use for chunk nodes @type chunk_node: C{string} @param top_node: The label to use for the root of the tree @type top_node: C{string} s\[|\]|[^\[\]\s]+it[isUnexpected [ at char %di˙˙˙˙t]isUnexpected ] at char %dsExpected ] at char %d(RBtcompileRtfinditertgroupR&t ValueErrortstartREtpoptNonetnltkttagtutilt str2tuple( tsRttop_nodetseptWORD_OR_BRACKETtstackRCttexttchunk((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt tagstr2trees*  's(\S+)\s+(\S+)\s+([IOB])-?(\S+)?tPPtVPc Csrt|gƒg}xUt|idƒƒD]>\}}|iƒpq(nti|ƒ}|tjotd|‚n|iƒ\}}} } |tj o| |jo d} n| djo| |di j} | djp| o%t |ƒdjo|i ƒq n| djp| o1t| gƒ} |di | ƒ|i | ƒn|di ||fƒq(W|d S( s) Convert a CoNLL IOB string into a tree. Uses the specified chunk types (defaults to NP, PP and VP), and creates a tree rooted at a node labeled S (by default). @param s: The CoNLL string to be converted. @type s: C{string} @param chunk_types: The chunk types to be converted. @type chunk_types: C{tuple} @param top_node: The node label to use for the root. @type top_node: C{string} @return: A chunk structure for a single sentence encoded in the given CONLL 2000 style string. @rtype: L{Tree} s sError on line %dtOtIi˙˙˙˙tBOitBi( Rt enumeratetsplittstript_LINE_RERCRXRUtgroupsRDR&RWRE( R]t chunk_typesR^RatlinenotlineRCtwordRZtstatet chunk_typet mismatch_IRc((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt conllstr2treeEs(      %c Cs¸g}xŤ|D]Ł}yi|i}d}xS|D]K}t|tƒo td‚n|i|d|d||fƒd}q,WWq tj o%|i|d|ddfƒq Xq W|S(sÔ Convert a tree to the CoNLL IOB tag format @param t: The tree to be converted. @type t: C{Tree} @return: A list of 3-tuples containing word, tag and IOB tag. @rtype: C{list} of C{tuple} sB-s7Tree is too deeply nested to be printed in CoNLL formatiisI-Rg(RDRARRUREtAttributeError(RIttagsRLtcategorytprefixtcontents((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyRws   "'cCs=g}t|ƒD]}|ti|ƒq~}di|ƒS(sÝ Convert a tree to the CoNLL IOB string format @param t: The tree to be converted. @type t: C{Tree} @return: A multiline string where each line contains a word, tag and IOB tag. @rtype: C{string} s (Rtstringtjoin(RIR7ttokentlines((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pyt tree2conllstrs 0s\s*(\s*(?P.+?)\s*\s*)?(\s*(?P.+?)\s*\s*)?(\s*(?P.+?)\s*\s*)?\s*(\s*(?P.+?)\s*\s*)?(?P.*?)\s*\s*\s*s#]*?type="(?P\w+)"cCsSt|gƒg}|djogSnxţtid|ƒD]ę}|iƒ}yŚ|idƒocti|ƒ}|djo dG|GHnt|idƒgƒ}|di|ƒ|i|ƒn0|idƒo|i ƒn|di|ƒWq:t t fj ot d|i ƒƒ‚q:Xq:Wt |ƒdjot d ƒ‚n|d S( Ns<[^>]+>|[^\s<]+s(((s%/p/zhu/06/nlp/nltk/nltk/chunk/util.pys s,     Ő .2       /