4Ic@sdZddkZddklZlZlZddklZddkTddk Tde fdYZ dZ d e fd YZd e fd YZd ZdZdZedjo endS(s Visualization tools for CFGs. iN(tContextFreeGrammart Nonterminaltparse_cfg_production(tTree(t*tProductionListcBs'eZeidZdZdZRS(t rightarrowcCsI|iddd|iddddd|id dd dddS(Ntterminalt foregrounds#006000tarrowtfonttsymbolt underlinet0t nonterminaltbluet helveticaitbold(s helveticaisbold(t tag_config(tselft textwidgettoptions((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyt_init_colortagsFscCsg}|id|idf|i|idfx\|iD]N}t|to!|id|idfqF|id|dfqFW|S(Ns%s RR s %ss %rR(tappendtlhstARROWtrhst isinstanceRR (Rtitemtcontentstelt((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyt _item_reprLs !(t__name__t __module__t SymbolWidgettSYMBOLSRRR(((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRCs  sM The CFG Editor can be used to create or modify context free grammars. A context free grammar consists of a start symbol and a list of productions. The start symbol is specified by the text entry field in the upper right hand corner of the editor; and the list of productions are specified in the main text editing box. Every non-blank line specifies a single production. Each production has the form "LHS -> RHS," where LHS is a single nonterminal, and RHS is a list of nonterminals and terminals. Nonterminals must be a single word, such as S or NP or NP_subj. Currently, nonterminals must consists of alphanumeric characters and underscores (_). Nonterminals are colored blue. If you place the mouse over any nonterminal, then all occurances of that nonterminal will be highlighted. Termianals must be surrounded by single quotes (') or double quotes("). For example, "dog" and "New York" are terminals. Currently, the string within the quotes must consist of alphanumeric characters, underscores, and spaces. To enter a new production, go to a blank line, and type a nonterminal, followed by an arrow (->), followed by a sequence of terminals and nonterminals. Note that "->" (dash + greater-than) is automatically converted to an arrow symbol. When you move your cursor to a different line, your production will automatically be colorized. If there are any errors, they will be highlighted in red. Note that the order of the productions is signifigant for some algorithms. To re-order the productions, use cut and paste to move them. Use the buttons at the bottom of the window when you are done editing the CFG: - Ok: apply the new CFG, and exit the editor. - Apply: apply the new CFG, and do not exit the editor. - Reset: revert to the original CFG, and do not exit the editor. - Cancel: revert to the original CFG, and exit the editor. t CFGEditorcBs>eZdZeidZeidedZeidedZ eid#eddZ eid ed Z d$Z e e dZdZdZdZdZdZdZdZdZddZdZdZdZdZdZdZdZd Zd!Z d"Z!RS(%sY A dialog window for creating and editing context free grammars. C{CFGEditor} places the following restrictions on what C{CFG}s can be edited: - All nonterminals must be strings consisting of word characters. - All terminals must be strings consisting of word characters and space characters. Rs(^\s*\w+\s*)(->|(s))s\s*(->|(s))\s*s (^\s*\w+\s*)s(->|(s#((\w+|'[\w ]*'|\"[\w ]*\"|\|)\s*)*$s\w+|->|'[\w ]+'|"[\w ]+"|(t)RiRcCs||_|dj o ||_nttdg|_||_d|_t||_|i |i |i i dddddd|i |ii dddd dd|i|ii dd dddd|iidS( NtSitsidettoptfilltxtexpanditbothtbottom(t_parenttNonet_cfgRRt_set_cfg_callbackt _highlight_matching_nonterminalstToplevelt_topt_init_bindingst_init_startframet _startframetpackt_init_prodframet _prodframet _init_buttonst _buttonframet _textwidgettfocus(Rtparenttcfgtset_cfg_callback((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyt__init__s       cCst|i}|_t||_|iiddt|ddiddt|ddidd|iid|ii i dS(NR'trightttexts Start Symbol:s Productions:tlefti( tFrameR4R7tEntryt_startR8tLabeltinsertR0tstartR (Rtframe((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR6s c Cst|i}|_t|ddd|iddddiddt|dd d|iddddiddt|dd d|iddddiddt|dd d|iddddiddt|dd d|i ddddidd dS(NRDtOktcommandR it takefocusR'REtApplytResettCanceltHelpRC( RFR4R<tButtont_okR8t_applyt_resett_cancelt_help(RRL((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR;scCs^|iid|iid|i|iid|i|iid|i|iid|i|iid|i|iid|i|iid|i|iid |i|iid |i|iid |i|iid |i|iid |i|iid|i|iid|i|iid|idS(Ns CFG Editors ss ssss ss ss ss ss(R4ttitletbindRXRURVRWRY(R((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR5s c Cs?t|i|_t|idddd|_t|idddd|_|iid |ii|iid |ii |ii d d d d|ii ddd dd d|ii ddd|ii ddd|ii dddd|_ |ii d|i|ii d|i|ii d|i|ii d|i|id}|ii d|g}|iiD]"}||i|igfq~}xtt|ddd D]}||d||ddjod(||djoqnd)||ddjoqn||ddGH||dGH||ddi||d||=qqWx|D]\}}|G|GHd!|}xY|D]Q} x>| D]6} t| to|d"| 7}q|d#| 7}qW|d$7}qW|d% d&}|iid'|qW|idS(*Nt backgrounds#e0e0e0texportselectioniROitorienttverticaltyscrollcommandRNR'RCR)tyR+R,RERRs#006000R R R terrortredt>s <>s s cSs|iidS(N(t tk_focusNextR>(teR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pytcyclessis%s ->s %ss %rs |is tend(((RFR4R:tTextR=t Scrollbart _textscrolltconfigtsettyviewR8Rt_linenumR[t_replace_arrowst_analyzet_check_analyzeR0t productionsRRtrangetlentextendRRRJ( RRgt_[1]tpt prod_tuplestiRtrhsstsRR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR9sV   ?!  !   "cCsYd|}d|}x>|iiD]-}|djo|ii|||q$q$WdS(s Remove all tags (except C{arrow} and C{sel}) from the given line of the text widget used for editing the productions. s%d.0s%d.endR tselN(sarrowssel(R=t tag_namest tag_remove(RtlinenumRKRhttag((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyt _clear_tags.s    cGsct|iididd}||ijo*|i||i|i||_ndS(s Check if we've moved to a new line. If we have, then remove all colorization from the line we moved to, and re-colorize the line that we moved from. RJt.iN(tintR=tindextsplitRoRt _analyze_line(RRfR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRr9s % cGsd}xp|iid|d}|djoPn|ii||d|ii||id|ii|dq d}xN|ii|i|dd}|djoPn|iid||dqd S( s Replace any C{'->'} text strings with arrows (char \256, in symbol font). This searches the whole buffer, but is fast enough to be done anytime they press '>'. s1.0s->s end+1charts+2charR s s+1charN(R=tsearchtdeleteRJRttag_add(RRfR ((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRpEs cCs|iddjo d}n^|id|ifjo d}n8d|i}||iijo|i|nd||if}d||if}|ii|||dS( s Given a line number and a regexp match for a token on that line, colorize the token. Note that the regexp match gives us the token's text, start index (on the line), and end index (on the line). is'"Rs->R t nonterminal_s%d.%dN(tgroupRR=R~t_init_nonterminal_tagRKRhR(RtmatchRRRKRh((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyt_analyze_tokenZs!&RcCs|ii|d|dti|ipdSn|i|d}|i|d}|ii|d||ii|d|dS(NRR cSs|i|dddS(NR\s#80ff80(R(RfRR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pytenteruscSs|i|dddS(NR\R(R(RfRR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pytleavewsss(R=RR$t_BOLDR2ttag_bind(RRRRR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRps  cCs|i||ii| d| d}tii|o&||d}tii||n(|idjo|i ||ndS(s( Colorize a given line. s.0s.endcSs|i||dS(NR(R(RRR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyt analyze_tokensRN( RR=tgetR$t_PRODUCTION_RERt _TOKEN_REtsubtstript _mark_error(RRtlineR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR|s cCstii|}|pd|}d|}nXtii|p$d|}d||if}n!d||if}d|}|ii|d|od|}d|}n|ii d||dS(s: Mark the location of an error in a line. s%d.0s%d.ends%d.%ds==RbN( R$t _ARROW_RERt_LHS_RERRKRhR=tcompareR(RRRt arrowmatchRKRh((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRs    cGs^|it|iididd}x(td|dD]}|i|qCWdS(sL Replace C{->} with arrows, and colorize the entire buffer. RhRiiN(RpRR=RRRtR(RRftnumlinesR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRqs  %cCsg}|iidd}ti|id|}tidd|}|id}x>|D]6}|i}|djoq^n|t|7}q^W|S(so Parse the current contents of the textwidget buffer, to create a list of productions. s1.0Rhs->s t s R(R=RtreRRRRR(RRsRDtlinesR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyt_parse_productionss cGs2|idjodSn|iid|_dS(N(R4R/tdestroy(RRf((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyt_destroys cGs|i|idS(N(RVR(RRf((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRUs cGsU|i}t|ii}t||}|idj o|i|ndS(N(RRRHRRR1R/(RRfRsRKR@((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRVs  cGsv|iiddx.|iiD]}|iidd|q#W|i|idj o|i|indS(Ns1.0Rhs%s (R=RR0RsRJRqR1R/(RRft production((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRWs cGs&y|iWnnX|idS(N(RWR(RRf((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRXsc GsVy)t|idtiddddWn&t|idtiddnXdS(NsHelp: Chart Parser DemotwidthiKR tfixed(tShowTextR.t_CFGEditor_HELPR(RRf((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRYs   s(^\s*\w+\s*)(->|((s helveticaisbold("R R!t__doc__R"R#RRtcompileRRRRRR/RBR6R;R5R9RRrRpRRRRRqRRRURVRWRXRY(((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR$s:       T     $     tCFGDemocBseZdZdZdZdZdZdZdZdZ dZ e d Z d Z d Zd ZRS( cCs||_||_t|_|iidt|i|_|iid|i|it |i}|i dddddd|i |i|i |i|i ||i||i|idS( NsContext Free Grammar Demoi R'RER)RaR+i(t_grammart_texttTkR4RZtIntVart_sizeRmR5RFR8t _init_menubarR;t _init_grammart _init_treelett_init_workspace(RtgrammarRDtframe1((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRBs     cCs|id|idS(Ns (R[R(RR(((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR5scCsdS(N((RR?((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR!scCsdS(N((RR?((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR;#scCswt||idd|_|iidddddd|ii|iid |i|iid |idS( NRiR'R(R)R,R+itselecttmove(RRt _prodlistR8R>t add_callbackt_selectprod_cb(RR?((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR%s  cCs;t|dd|_|iiddddd|_dS(NR\twhiteR'R-R)R*(tCanvast_treelet_canvasR8R/t_treelet(RR?((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR,scCsKt|dd|_|iiddddddd|_|idS( NR\RR'RCR)R,R+i(t CanvasFramet _workspaceR8R/t_treetreset_workspace(RR?((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR1s c CsO|ii}t|ii}d|d df}d|d f}|idj o|ii|in|ii i }t ||d|dd}g}xS|i D]H}t |to|i}n|it ||d|ddqWt|||dd |_|ii|ix|D]} | id d q1WdS( NRiRiR t draggableitcolorRiid(RtcanvasRRRRR/t remove_widgetRRKR t TextWidgetRRtTokenttypeRtTreeSegmentWidgett add_widgetR( Rtctfontsizet node_fontt leaf_fontRKtrootnodetleavestwordtleaf((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR;s$  & cCsdS(N((RR((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pytworkspace_markprod\scCs1|djo |i}nxtt|it|iD]}|d|fdjo |inxt|iD]\}}|i||}t|t o0t|t o |i |i i joq~t|ttfo't|to||i joq~Pq~WdG|GHqCWdS(NRRsMATCH AT(R/RRtRutsubtreesRt_markproductiont enumerateRRRR tnodeRDtstrtunicodeR(RtprodttreeRztjRtwidget((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyR_s$)  c Cs{|i}|ii||idj o|iin|i}x:t|D],\}}t|t ot |}qSqSWt |i i |}t |ii}d|d df}d|d f} t||d|d| |_d|id<|ii\} } } } t |d t |d }}|ii|| | d|| | d|i|dS( NRiRiRRiRRtheight(RRt highlightRR/RRRRRRRR RRRttree_to_treesegmenttbboxRR(RRRRRzRRRRRtx1ty1tx2ty2twth((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRxs& !  $   !+cGs|iidS(N(R4R(Rtargs((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRscOs|ii||dS(N(R4tmainloop(RRtkwargs((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRs(R R!RBR5RR;RRRRRR/RRRR(((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRs       !    cCsddkl}l}l}d}g}|iD]}|||q3~\ }}}} } } } } }||||g|||| g|||| g|||| g||| || g||| |g|| | |g|| g|| dd|g||dg||dg||dg|| d g|| d g|| d g|| d g|| d g|| dg|| dg||dgf}|||}di}t||}|idS(Ni(Rt ProductionRsS VP NP PP P N Name V DettuptovertItthetatmantsawtintwithtparktdogtstatuetmysI saw a man in the park(tnltkRRRRRR(RRRt nonterminalsRwR|R&tVPtNPtPPtPtNtNametVtDetRsRRDtd((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pytdemo2s.A  c Csddkl}l}d}g}|iD]}|||q-~\ }}}}} } } } } |d}d}t}t|||}t|ddit|ddd |i i|i dS( Ni(Rt parse_cfgsS VP NP PP P N Name V Dets+ S -> NP VP PP -> P NP NP -> Det N NP -> NP PP VP -> V NP VP -> VP PP Det -> 'a' Det -> 'the' Det -> 'my' NP -> 'I' N -> 'dog' N -> 'man' N -> 'park' N -> 'statue' V -> 'saw' P -> 'in' P -> 'up' P -> 'over' P -> 'with' cSs |GHdS(N((R((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pytcbsRDs Testing CFG Editor tQuitRN( RRR RRR$RIR8RTRR(RR RRwR|R&RRRRRRRRRR R(teditor((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pytdemosA   cCs$ddkl}td\ }}}}}}}}} ||||g||| |g||||g||||g|||||g||||g||||g||g||dd|g||dg|| dg|| dg||d g||d g||d g||d g||d g||dg||dg|| dgf} t} | d} | id| t| | } | idddd| id| i| id| i| i | i | d| i | ddS(Ni(Rs!S, VP, NP, PP, P, N, Name, V, DetRRRRRRRRRRRRRcSs|idS(N(R(Rftt((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pyRstqR+iR)R,RRii( RRRRR[RR8RtmarkonlyR>tmark(RR&RRRRRRRRRsRRRx((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pytdemo3s6'    t__main__(RRt nltk.grammarRRRRRtutilRt ColorizedListRRtobjectR$RR RRR (((s#/p/zhu/06/nlp/nltk/nltk/draw/cfg.pys s *  A| $ # &