³ò ¶ÒÇIc!@sÙdZddkZddkZddkZddkZddkZddkZddkZddkZddk Z ddk Z ddk Z yddk Z Wnddk Z nXyddklZWnddklZnXddklZlZgZegZeiiddƒieiƒD]Zeo eeq#q#[7ZegZeiiddƒieiƒD]Zeo eeqnqn[7Zeiidƒdjoeeiid ƒg7Zneiid ƒoDed d d eiiei dƒeiiei ddƒg7Znedddddg7Zde!fd„ƒYZ"de"e#fd„ƒYZ$de$fd„ƒYZ%de"fd„ƒYZ&ei'ƒZ(d„Z)de+d„Z,hdd <d!d"<d#d$<d%d&<d'd(<d)d*<d+d,<d-d.<d/d0™stdocs; The absolute path identified by this path pointer.cCs6t|idƒ}|dj ot||ƒ}n|S(Ntrb(R RRtSeekableUnicodeStreamReader(R R tstream((s/p/zhu/06/nlp/nltk/nltk/data.pyR œs cCsti|iƒiS(N(RtstatRtst_size(R ((s/p/zhu/06/nlp/nltk/nltk/data.pyR¢scCs+tii|i|idƒŒ}t|ƒS(Nt/(RRRRtsplitR(R RR((s/p/zhu/06/nlp/nltk/nltk/data.pyR¥s!cCs d|iS(NsFileSystemPathPointer(%r)(R(R ((s/p/zhu/06/nlp/nltk/nltk/data.pyt__repr__©scCs|iS(N(R(R ((s/p/zhu/06/nlp/nltk/nltk/data.pyt__str__¬s( RRRRtpropertyRRR RRR'R((((s/p/zhu/06/nlp/nltk/nltk/data.pyR„s      tGzipFileSystemPathPointercBs$eZdZddZed„ZRS(sì A subclass of C{FileSystemPathPointer} that identifies a gzip-compressed file located at a given absolute path. C{GzipFileSystemPathPointer} is appropriate for loading large gzip-compressed pickle objects efficiently. iicstƒ}tiˆidƒ‰x0t‡‡fd†dƒD]}|i|ƒq:Wt|iƒƒ}|ot||ƒ}n|S(NR csˆiˆiƒS((treadt BLOCK_SIZE((R tfile(s/p/zhu/06/nlp/nltk/nltk/data.pyR¿sR(RtgzipR RtitertwritetgetvalueR!(R R R"tline((R R-s/p/zhu/06/nlp/nltk/nltk/data.pyR ¹s i(RRRR,RR (((s/p/zhu/06/nlp/nltk/nltk/data.pyR*°s tZipFilePathPointercBskeZdZdd„Zed„ddƒZed„ddƒZed„Zd „Z d „Z d „Z RS( s~ A path pointer that identifies a file contained within a zipfile, which can be accessed by reading that zipfile. RcCsæt|tƒottii|ƒƒ}ntidd|ƒ}|oˆy|i|ƒWqÐ|i dƒo@g}|i ƒD]!}|i |ƒo ||q€q€~oqÌt d|i |fƒ‚qÐXn||_||_dS(sÞ Create a new path pointer pointing at the specified entry in the given zipfile. @raise IOError: If the given zipfile does not exist, or if it does not contain the specified entry. s(^|/)/+s\1R%sZipfile %r does not contain %rN(t isinstancet basestringtOpenOnDemandZipFileRRRtretsubtgetinfotendswithtnamelistt startswithRtfilenamet_zipfilet_entry(R tzipfiletentryt_[1]tn((s/p/zhu/06/nlp/nltk/nltk/data.pyRÌs< cCs|iS((R>(R ((s/p/zhu/06/nlp/nltk/nltk/data.pyRìsRs€ The C{zipfile.ZipFile} object used to access the zip file containing the entry identified by this path pointer.cCs|iS((R?(R ((s/p/zhu/06/nlp/nltk/nltk/data.pyRïssY The name of the file within C{zipfile} that this path pointer points to.cCsE|ii|iƒ}t|ƒ}|dj ot||ƒ}n|S(N(R>R+R?RRR!(R R tdataR"((s/p/zhu/06/nlp/nltk/nltk/data.pyR ós   cCs|ii|iƒiS(N(R>R9R?R(R ((s/p/zhu/06/nlp/nltk/nltk/data.pyRúscCs#d|i|f}t|i|ƒS(Ns%s/%s(R?R3R>(R RRA((s/p/zhu/06/nlp/nltk/nltk/data.pyRýscCsd|ii|ifS(NsZipFilePathPointer(%r, %r)(R>R=R?(R ((s/p/zhu/06/nlp/nltk/nltk/data.pyR's( RRRRR)R@RARR RRR'(((s/p/zhu/06/nlp/nltk/nltk/data.pyR3Çs       c Csktid|ƒ}|iƒ\}}xCtD];}tii|ƒo>|idƒo.yt||ƒSWqftj o q+qfXq+tii |ƒoÑ|djoatii ||i dƒŒ}tii |ƒo,|idƒot|ƒSqÿt|ƒSqbqftii ||i dƒŒ}tii |ƒo.yt||ƒSWqbtj o q+qbXqfq+q+W|djo~|i dƒ}xltt|ƒƒD]T}di || ||dg||ƒ}yt|ƒSWq™tj oq™Xq™Wntid|fddddd d ƒ} | d d i d „tDƒƒ7} dd} d| | | f} t| ƒ‚dS(s¦ Find the given resource by searching through the directories and zip files in L{nltk.data.path}, and return a corresponding path name. If the given resource is not found, raise a C{LookupError}, whose message gives a pointer to the installation instructions for the NLTK downloader. Zip File Handling: - If C{resource_name} contains a component with a C{.zip} extension, then it is assumed to be a zipfile; and the remaining path components are used to look inside the zipfile. - If any element of C{nltk.data.path} has a C{.zip} extension, then it is assumed to be a zipfile. - If a given resource name that does not contain any zipfile component is not found initially, then C{find()} will make a second attempt to find that resource, by replacing each component I{p} in the path with I{p.zip/p}. For example, this allows C{find()} to map the resource name C{corpora/chat80/cities.pl} to a zip file path pointer to C{corpora/chat80.zip/chat80/cities.pl}. - When using C{find()} to locate a directory contained in a zipfile, the resource name I{must} end with the C{'/'} character. Otherwise, C{find()} will not locate the directory. @type resource_name: C{str} @param resource_name: The name of the resource to search for. Resource names are posix-style relative path names, such as C{'corpora/brown'}. In particular, directory names should always be separated by the C{'/'} character, which will be automatically converted to a platform-appropriate path separator. @rtype: C{str} s(.*\.zip)/?(.*)$|s.zipR%s.gzscResource %r not found. Please use the NLTK Downloader to obtain the resource: >>> nltk.download().tinitial_indents tsubsequent_indenttwidthiBs Searched in:Rcssx|]}d|VqWdS(s - %rN((t.0td((s/p/zhu/06/nlp/nltk/nltk/data.pys _st*iFs %s %s %sN(R7tmatchtgroupsRRtisfileR:R3RtisdirRRR&RR*Rtrangetlentfindt LookupErrorttextwraptfill( t resource_nametmR@tzipentryt path_itemtptpiecestit modified_nametmsgtseptresource_not_found((s/p/zhu/06/nlp/nltk/nltk/data.pyRQ sB(# " * ! cCs |d joC|idƒotii|ƒd}qPtidd|ƒ}ntii|ƒo#tii|ƒ}t d|‚n|od||fGHnt |ƒ}t |dƒ}x4t o,|i d ƒ}|i|ƒ|pPq¾q¾W|iƒ|iƒd S( s§ Copy the given resource to a local file. If no filename is specified, then use the URL's filename. If there is already a file named C{filename}, then raise a C{ValueError}. @type resource_url: C{str} @param resource_url: A URL specifying where the resource should be loaded from. The default protocol is C{"nltk:"}, which searches for the file in the the NLTK data package. sfile:iÿÿÿÿs (^\w+:)?.*/RsFile %r already exists!sRetrieving %r, saving to %rtwbii@Ni(RR<RRR&R7R8RRt ValueErrort_openR tTrueR+R0tclose(t resource_urlR=tverbosetinfiletoutfilets((s/p/zhu/06/nlp/nltk/nltk/data.pytretrieveds"     s;A serialized python object, stored using the pickle module.tpickles9A serialized python object, stored using the yaml module.tyamls3A context free grammar, parsed by nltk.parse_cfg().tcfgs1A probabilistic CFG, parsed by nltk.parse_pcfg().tpcfgs+A feature CFG, parsed by nltk.parse_fcfg().tfcfgsiA list of first order logic expressions, parsed by nltk.sem.parse_fol() using nltk.sem.logic.LogicParser.tfolsyA list of first order logic expressions, parsed by nltk.sem.parse_logic(). Requires an additional logic_parser parametertlogics;A semantic valuation, parsed by nltk.sem.parse_valuation().tvals)The raw (byte string) contents of a file.trawtautoc Cs½|o?ti|ƒ}|tj o|od|fGHn|SqFn|od|fGHn|djoh|idƒ}|d}|djo|d}nti|ƒ}|tjotd|ƒ‚qÒn|d jotit|ƒƒ}n“|d jot it|ƒƒ}nm|d jot i t|ƒi ƒƒ}nA|d jot i t|ƒi ƒƒ}n|d jo+t it|ƒi ƒd|d|ƒ}nÝ|djo.tit|ƒi ƒdtiiƒƒ}n¢|djo%tit|ƒi ƒd|ƒ}np|djotit|ƒi ƒƒ}nD|djot|ƒi ƒ}n!|tjpt‚tdƒ‚|o(y|t|>s<>Rtt.iÿÿÿÿtgziþÿÿÿszCould not determine format for %s based on its file extension; use the "format" argument to specify the format explicitly.RkRlRmRnRot logic_parsertfstruct_parserRpRqRrRssUnknown format type!(t_resource_cachetgetRR&t AUTO_FORMATSRaRktloadRbRlRmt parse_cfgR+t parse_pcfgt parse_fcfgRt parse_logicRqt LogicParsertparse_valuationtFORMATStAssertionErrort TypeError( RetformattcacheRfRwRxt resource_valtresource_url_partstext((s/p/zhu/06/nlp/nltk/nltk/data.pyR|¨s^+                   s##cCsot|dddtƒ}|iƒ}xD|D]<}|i|ƒoq+ntid|ƒoq+n|GHq+WdS(s„ Write out a grammar file, ignoring escaped and empty lines @type resource_url: C{str} @param resource_url: A URL specifying where the resource should be loaded from. The default protocol is C{"nltk:"}, which searches for the file in the the NLTK data package. @type escape: C{str} @param escape: Prepended string that signals lines to be ignored R†RsR‡s^$N(R|tFalset splitlinesR<R7RK(RetescapeRˆtlinestl((s/p/zhu/06/nlp/nltk/nltk/data.pytshow_cfgs  cCstiƒdS(sI Remove all objects from the resource cache. @see: L{load()} N(Rytclear(((s/p/zhu/06/nlp/nltk/nltk/data.pyt clear_cache#scCs‡tid|ƒiƒ\}}|djp|iƒdjot|ƒiƒSn2|iƒdjot|dƒSnti|ƒSdS(sv Helper function that returns an open file object for a resource, given its resource URL. If the given resource URL uses the 'ntlk' protocol, or uses no protocol, then use L{nltk.data.find} to find its path, and open it with the given mode; if the resource URL uses the 'file' protocol, then open the file with the given mode; otherwise, delegate to C{urllib2.urlopen}. @type resource_url: C{str} @param resource_url: A URL specifying where the resource should be loaded from. The default protocol is C{"nltk:"}, which searches for the file in the the NLTK data package. s(?:(\w+):)?(.*)tnltkR-R N( R7RKRLRtlowerRQR turllib2turlopen(RetprotocolR((s/p/zhu/06/nlp/nltk/nltk/data.pyRb*s  t LazyLoadercBs,eZd„Zd„Zd„Zd„ZRS(cCs ||_dS(N(t_LazyLoader__path(R R((s/p/zhu/06/nlp/nltk/nltk/data.pyRHscCs+t|iƒ}|i|_|i|_dS(N(R|R™t__dict__t __class__(R tresource((s/p/zhu/06/nlp/nltk/nltk/data.pyt__loadKs cCs|iƒt||ƒS(N(t_LazyLoader__loadtgetattr(R tattr((s/p/zhu/06/nlp/nltk/nltk/data.pyt __getattr__Ss cCs|iƒd|S(Ns%r(Rž(R ((s/p/zhu/06/nlp/nltk/nltk/data.pyR'Ys (RRRRžR¡R'(((s/p/zhu/06/nlp/nltk/nltk/data.pyR˜Gs   R6cBs;eZdZd„Zd„Zd„Zd„Zd„ZRS(sã A subclass of C{zipfile.ZipFile} that closes its file pointer whenever it is not using it; and re-opens it when it needs to read data from the zipfile. This is useful for reducing the number of open file handles when many zip files are being accessed at once. C{OpenOnDemandZipFile} must be constructed from a filename, not a file-like object (to allow re-opening). C{OpenOnDemandZipFile} is read-only (i.e., C{write} and C{writestr} are disabled. cCsXt|tƒptdƒ‚ntii||ƒ|i|jpt‚|iƒdS(Ns+ReopenableZipFile filename must be a string( R4R5R…R@tZipFileRR=R„Rd(R R=((s/p/zhu/06/nlp/nltk/nltk/data.pyRms cCsO|idjpt‚t|idƒ|_tii||ƒ}|iƒ|S(NR ( tfpRR„R R=R@R¢R+Rd(R tnametvalue((s/p/zhu/06/nlp/nltk/nltk/data.pyR+ts  cOstdƒ‚dS(s<@raise NotImplementedError: OpenOnDemandZipfile is read-onlys OpenOnDemandZipfile is read-onlyN(R (R targstkwargs((s/p/zhu/06/nlp/nltk/nltk/data.pyR0{scOstdƒ‚dS(s<@raise NotImplementedError: OpenOnDemandZipfile is read-onlys OpenOnDemandZipfile is read-onlyN(R (R R¦R§((s/p/zhu/06/nlp/nltk/nltk/data.pytwritestrscCs d|iS(NsOpenOnDemandZipFile(%r)(R=(R ((s/p/zhu/06/nlp/nltk/nltk/data.pyR'ƒs(RRRRR+R0R¨R'(((s/p/zhu/06/nlp/nltk/nltk/data.pyR6cs      R!cBs£eZdZeZdd„Zed„Zed„Zeed„Z d„Z d„Z d„Z e d „d d ƒZe d „d d ƒZe d„d dƒZd„Zdd„Zd„Zed„Zd„Zed„Zd„Zheiefgd<eidfeidfgd<eiefgd<eiefgd<eidfeidfgd <eiefgd!<eiefgd"\}}|i |ƒo"|o ||_nt |ƒSq`q`WndS(Ns[ -]Rii( R7R8R R”t _BOM_TABLERzR"R+RªR<RPR(R tenctbom_infoRÏtbomt new_encoding((s/p/zhu/06/nlp/nltk/nltk/data.pyR³s ( RRRRcRÓRRR+R¿RÂRÄRÅRÆR)RÇR¤RÈRdRªRÌRËR¹RµRÍR¬tBOM_UTF8t BOM_UTF16_LEt BOM_UTF16_BEt BOM_UTF32_LEt BOM_UTF32_BERæR³(((s/p/zhu/06/nlp/nltk/nltk/data.pyR!ŠsB 9  <           ' . %   (8RtsysRtos.pathRStweakrefRlR7R•R@R¬R.tcPickleRkt cStringIORR“RRmRRRBtenvironRzR&tpathsepRIt_[2]t expandusertplatformR<RtprefixtobjectR RRR*R3tWeakValueDictionaryRyRQRRcRjRƒR{R‹R|RR’RbR˜R¢R6R!(((s/p/zhu/06/nlp/nltk/nltk/data.pys sˆ         KK # (,B  W(                  h   '