B_Kc@sddkZddkZddkZddkZddkZddkZddkZddklZl Z ddk l Z ddk l Z ddk lZlZddZdZd dd Zd d Zd d dZde fdYZdddZdZedddZdZdZdZdZeeddZ dZ!dZ"eeddZ#dZ$dZ%d e&fd!YZ'd"e(fd#YZ)d$e)fd%YZ*d&e)fd'YZ+d(e)fd)YZ,d*ee,fd+YZ-d,e,fd-YZ.d.e.fd/YZ/d*ee,fd0YZ-d1ee+fd2YZ0hdd3Z1dS(4iN(tislicetchain(tpprint(t defaultdict(t Deprecatedt slice_boundstselfc Csmddk}t|t|titifp |i}nd|iGHxtt i |i D]\}}|i doqgnt |dtoqgn|i|\}}}}|oU|ddjoD|djpt|t|jo|d}d||f}n|i||||} tid || fd d d d t|dGHqgWdS(Nis%%s supports the following operations:t_t__deprecated__iRis%s.%ss%s%stinitial_indents - tsubsequent_indentt i(tinspecttstrt isinstancettypestTypeTypet ClassTypet __class__t__name__tsortedtpydoct allmethodstitemst startswithtgetattrtFalset getargspectNonetlent formatargspecttextwraptfill( tobjtselfnameR tnametmethodtargstvarargstvarkwtdefaultstargspec((s/p/zhu/06/nlp/nltk/nltk/util.pytusages&     & cCsDddk}ddk}t|i|ijo|iiidjS(s| @rtype: C{boolean} @return: true if this function is run within idle. Tkinter programs that are run in idle should never call C{Tk.mainloop}; so this function should be used to gate all calls to C{Tk.mainloop}. @warning: This function works by checking C{sys.stdin}. If the user has modified C{sys.stdin}, then it may return incorrect results. iNtPyShell(tsysRttypetstdint InstanceTypeRR(R,R((s/p/zhu/06/nlp/nltk/nltk/util.pytin_idle5s icCs ttt|||dS(s Pretty print a sequence of data items @param data: the data stream to print @type data: C{sequence} or C{iterator} @param start: the start position @type start: C{int} @param end: the end position @type end: C{int} N(RtlistR(tdatatstarttend((s/p/zhu/06/nlp/nltk/nltk/util.pytprHs iFcCs!diti|d|GHdS(s Pretty print a string, breaking lines on whitespace @param s: the string to print, consisting of words and spaces @type s: C{string} @param width: the display width @type width: C{int} s twidthN(tjoinRtwrap(tsR6((s/p/zhu/06/nlp/nltk/nltk/util.pyt print_stringUs R cCs%diti|i|d|S(s, Pretty print a list of text tokens, breaking lines on whitespace @param tokens: the tokens to print @type tokens: C{list} @param separator: the string to use to separate tokens @type separator: C{str} @param width: the display width (default=70) @type width: C{int} s R6(R7RR8(ttokenst separatorR6((s/p/zhu/06/nlp/nltk/nltk/util.pyt tokenwrap`s tIndexcBseZdZRS(cCs<ti|tx%|D]\}}||i|qWdS(N(Rt__init__R1tappend(Rtpairstkeytvalue((s/p/zhu/06/nlp/nltk/nltk/util.pyR?ts (Rt __module__R?(((s/p/zhu/06/nlp/nltk/nltk/util.pyR>rst{t}cCs2ti|tii|d||iGHdS(s\ Search C{string} for substrings matching C{regexp} and wrap the matches with braces. This is convenient for learning about regular expressions. @param regexp: The regular expression. @type regexp: C{string} @param string: The string being matched. @type string: C{string} @param left: The left delimiter (printed before the matched substring) @type left: C{string} @param right: The right delimiter (printed after the matched substring) @type right: C{string} @rtype: C{string} @return: A string with markers surrounding the matched substrings. s\g<0>N(tretcompiletMtsubtrstrip(tregexptstringtlefttright((s/p/zhu/06/nlp/nltk/nltk/util.pytre_show~scCsOt|do|iSn.t|tot|iSn tddS(Ntreads2Must be called with a filename or file-like object(thasattrRQRt basestringtopent ValueError(tf((s/p/zhu/06/nlp/nltk/nltk/util.pyt filestrings ccs|djo g}n|i|xY|oQ|id}|V|djo,y|||7}|d8}Wq{q{Xq'q'WdS(sTraverse the nodes of a tree in breadth-first order. (No need to check for cycles.) The first argument should be the tree root; children should be a function taking as argument a tree node and returning an iterator of the node's children. iiN(RR@tpop(ttreetchildrentdepthtqueuetnode((s/p/zhu/06/nlp/nltk/nltk/util.pyt breadth_firsts     cCspd}dg}y|ititiWntj onXy|itidWnttfj onXy|itidWnttfj onX|idxR|D]J}|pqnyt ||}|}Wnt t fj oqXPqW|pIt ddi g}|D]}|o|t |q0q0~n ||fSdS(sr Given a byte string, attempt to decode it. Tries the standard 'UTF8' and 'latin-1' encodings, Plus several gathered from locale information. The calling program *must* first call:: locale.setlocale(locale.LC_ALL, '') If successful it returns C{(decoded_unicode, successful_encoding)}. If unsuccessful it raises a C{UnicodeError}. sutf-8islatin-1s@Unable to decode input data. Tried the following encodings: %s.s, N(RR@tlocalet nl_langinfotCODESETtAttributeErrort getlocalet IndexErrortgetdefaultlocaletunicodet UnicodeErrort LookupErrorR7trepr(R2tsuccessful_encodingt encodingstenctdecodedt_[1]((s/p/zhu/06/nlp/nltk/nltk/util.pytguess_encodings<    CcCs}ddkl}|t}xZ|D]R}t||do*x5||D]}||i|qHWq#||||.*?()ts(?s)[\n]?s (?s)<.*?>R s s (RGRJtstrip(thtmltcleaned((s/p/zhu/06/nlp/nltk/nltk/util.pyt clean_html s cCs,ddkl}||i}t|S(Ni(turlopen(turllibR{RQRz(turlR{Rx((s/p/zhu/06/nlp/nltk/nltk/util.pyt clean_urlscCs|ot|f|d|}n|ot||f|d}nt|}tdt||d}g}t|D]}|t||||!q~S(s A utility that produces a sequence of ngrams from a sequence of items. For example: >>> ngrams([1,2,3,4,5], 3) [(1, 2, 3), (2, 3, 4), (3, 4, 5)] Use ingram for an iterator version of this function. Set pad_left or pad_right to true in order to get additional ngrams: >>> ngrams([1,2,3,4,5], 2, pad_right=True) [(1, 2), (2, 3), (3, 4), (4, 5), (5, None)] @param sequence: the source data to be converted into ngrams @type sequence: C{sequence} or C{iterator} @param n: the degree of the ngrams @type n: C{int} @param pad_left: whether the ngrams should be left-padded @type pad_left: C{boolean} @param pad_right: whether the ngrams should be right-padded @type pad_right: C{boolean} @param pad_symbol: the symbol to use for padding (default is None) @type pad_symbol: C{any} @return: The ngrams @rtype: C{list} of C{tuple}s ii(RR1tmaxRtrangettuple(tsequencetntpad_leftt pad_rightt pad_symboltcountRnti((s/p/zhu/06/nlp/nltk/nltk/util.pytngrams*s cKst|d|S(s A utility that produces a sequence of bigrams from a sequence of items. For example: >>> bigrams([1,2,3,4,5]) [(1, 2), (2, 3), (3, 4), (4, 5)] Use ibigrams for an iterator version of this function. @param sequence: the source data to be converted into bigrams @type sequence: C{sequence} or C{iterator} @return: The bigrams @rtype: C{list} of C{tuple}s i(R(Rtkwargs((s/p/zhu/06/nlp/nltk/nltk/util.pytbigramsOscKst|d|S(s A utility that produces a sequence of trigrams from a sequence of items. For example: >>> trigrams([1,2,3,4,5]) [(1, 2, 3), (2, 3, 4), (3, 4, 5)] Use itrigrams for an iterator version of this function. @param sequence: the source data to be converted into trigrams @type sequence: C{sequence} or C{iterator} @return: The trigrams @rtype: C{list} of C{tuple}s i(R(RR((s/p/zhu/06/nlp/nltk/nltk/util.pyttrigrams`sccst|}|ot|f|d|}n|ot||f|d}ng}x/|djo!|i|i|d8}q_Wx-|D]%}|i|t|V|d=qWdS(s A utility that produces an iterator over ngrams generated from a sequence of items. For example: >>> list(ingrams([1,2,3,4,5], 3)) [(1, 2, 3), (2, 3, 4), (3, 4, 5)] Use ngrams for a list version of this function. Set pad_left or pad_right to true in order to get additional ngrams: >>> list(ingrams([1,2,3,4,5], 2, pad_right=True)) [(1, 2), (2, 3), (3, 4), (4, 5), (5, None)] @param sequence: the source data to be converted into ngrams @type sequence: C{sequence} or C{iterator} @param n: the degree of the ngrams @type n: C{int} @param pad_left: whether the ngrams should be left-padded @type pad_left: C{boolean} @param pad_right: whether the ngrams should be right-padded @type pad_right: C{boolean} @param pad_symbol: the symbol to use for padding (default is None) @type pad_symbol: C{any} @return: The ngrams @rtype: C{iterator} of C{tuple}s iiN(titerRR@tnextR(RRRRRthistorytitem((s/p/zhu/06/nlp/nltk/nltk/util.pytingramsqs   cks&xt|d|D] }|VqWdS(s A utility that produces an iterator over bigrams generated from a sequence of items. For example: >>> list(ibigrams([1,2,3,4,5])) [(1, 2), (2, 3), (3, 4), (4, 5)] Use bigrams for a list version of this function. @param sequence: the source data to be converted into bigrams @type sequence: C{sequence} or C{iterator} @return: The bigrams @rtype: C{iterator} of C{tuple}s iN(R(RRR((s/p/zhu/06/nlp/nltk/nltk/util.pytibigramsscks&xt|d|D] }|VqWdS(s A utility that produces an iterator over trigrams generated from a sequence of items. For example: >>> list(itrigrams([1,2,3,4,5]) [(1, 2, 3), (2, 3, 4), (3, 4, 5)] Use trigrams for a list version of this function. @param sequence: the source data to be converted into trigrams @type sequence: C{sequence} or C{iterator} @return: The trigrams @rtype: C{iterator} of C{tuple}s iN(R(RRR((s/p/zhu/06/nlp/nltk/nltk/util.pyt itrigramsst OrderedDictcBseZedZdZdZdZdZdZdZ dZ dZ eed Z d Z ed Zd Zd ZRS(cKsb|i||id|_|id|_|djoti|nti||dS(Ntkeystdefault_factory(Rtgett_keyst_default_factoryRtdictR?(RR2R((s/p/zhu/06/nlp/nltk/nltk/util.pyR?s  cCs$ti|||ii|dS(N(Rt __delitem__Rtremove(RRB((s/p/zhu/06/nlp/nltk/nltk/util.pyRscCs;yti||SWn tj o|i|SnXdS(N(Rt __getitem__tKeyErrort __missing__(RRB((s/p/zhu/06/nlp/nltk/nltk/util.pyRscCsd|iDS(Ncssx|] }|VqWdS(N((t.0RB((s/p/zhu/06/nlp/nltk/nltk/util.pys s(R(R((s/p/zhu/06/nlp/nltk/nltk/util.pyRpscCs6|i o||ijo tn |iSdS(N(RRR(RRB((s/p/zhu/06/nlp/nltk/nltk/util.pyRs cCs;ti|||||ijo|ii|ndS(N(Rt __setitem__RR@(RRBR((s/p/zhu/06/nlp/nltk/nltk/util.pyRscCsti||iidS(N(RtclearR(R((s/p/zhu/06/nlp/nltk/nltk/util.pyRs cCsti|}|i|_|S(N(RtcopyR(RRr((s/p/zhu/06/nlp/nltk/nltk/util.pyRs cCst|i|iS(N(tzipRtvalues(R((s/p/zhu/06/nlp/nltk/nltk/util.pyRscCs|o|o?t|tptt|t|jpt|Sqt|tp't|tpt|tptt|tpt|to|iSqt|to)g}|D]\}}||q~Sqn d|ijo |iSngSdS(NR( RR1tAssertionErrorRRRRt__dict__R(RR2RRnRBRC((s/p/zhu/06/nlp/nltk/nltk/util.pyRs  - cCsE|io.|ii}||}||=||fSn tdS(N(RRXR(RRBRC((s/p/zhu/06/nlp/nltk/nltk/util.pytpopitem s   cCs;ti|||||ijo|ii|ndS(N(Rt setdefaultRR@(RRBtfailobj((s/p/zhu/06/nlp/nltk/nltk/util.pyRscCsRti||x;|i|D]*}||ijo|ii|q q WdS(N(RtupdateRRR@(RR2RB((s/p/zhu/06/nlp/nltk/nltk/util.pyRs cCst|i|iS(N(tmapRR(R((s/p/zhu/06/nlp/nltk/nltk/util.pyRs(RRDRR?RRRpRRRRRRRRRR(((s/p/zhu/06/nlp/nltk/nltk/util.pyRs           tAbstractLazySequencecBseZdZdZdZdZdZdZeedZ dZ dZ d Z d Z d Zd Zd ZdZdZRS(sY An abstract base class for read-only sequences whose values are computed as needed. Lazy sequences act like tuples -- they can be indexed, sliced, and iterated over; but they may not be modified. The most common application of lazy sequences in NLTK is for I{corpus view} objects, which provide access to the contents of a corpus without loading the entire corpus into memory, by loading pieces of the corpus from disk as needed. The result of modifying a mutable element of a lazy sequence is undefined. In particular, the modifications made to the element may or may not persist, depending on whether and when the lazy sequence caches that element's value or reconstructs it from scratch. Subclasses are required to define two methods: - L{__len__()} - L{iterate_from()}. cCstddS(se Return the number of tokens in the corpus file underlying this corpus view. s!should be implemented by subclassN(tNotImplementedError(R((s/p/zhu/06/nlp/nltk/nltk/util.pyt__len__:scCstddS(s Return an iterator that generates the tokens in the corpus file underlying this corpus view, starting at the token number C{start}. If C{start>=len(self)}, then this iterator will generate no tokens. s!should be implemented by subclassN(R(RR3((s/p/zhu/06/nlp/nltk/nltk/util.pyt iterate_fromAscCst|to)t||\}}t|||Snx|djo|t|7}n|djotdny|i|iSWntj otdnXdS(s Return the C{i}th token in the corpus file underlying this corpus view. Negative indices and spans are both supported. isindex out of rangeN( RtsliceRtLazySubsequenceRRdRRt StopIteration(RRR3tstop((s/p/zhu/06/nlp/nltk/nltk/util.pyRJs!cCs |idS(sdReturn an iterator that generates the tokens in the corpus file underlying this corpus view.i(R(R((s/p/zhu/06/nlp/nltk/nltk/util.pyRp\scstfd|DS(s7Return the number of times this list contains C{value}.c3s*x#|]}|jo dVqqWdS(iN((Rtelt(RC(s/p/zhu/06/nlp/nltk/nltk/util.pys cs(tsum(RRC((RCs/p/zhu/06/nlp/nltk/nltk/util.pyRascCspt|t||\}}x?tt|||D]%\}}||jo ||Sq7q7WtddS(s Return the index of the first occurance of C{value} in this list that is greater than or equal to C{start} and less than C{stop}. Negative start & stop values are treated like negative slice bounds -- i.e., they count from the end of the list.sindex(x): x not in listN(RRt enumerateRRU(RRCR3RRR((s/p/zhu/06/nlp/nltk/nltk/util.pytindexes  cCst|i|S(s+Return true if this list contains C{value}.(tboolR(RRC((s/p/zhu/06/nlp/nltk/nltk/util.pyt __contains__oscCst||gS(s,Return a list concatenating self with other.(tLazyConcatenation(Rtother((s/p/zhu/06/nlp/nltk/nltk/util.pyt__add__sscCst||gS(s,Return a list concatenating other with self.(R(RR((s/p/zhu/06/nlp/nltk/nltk/util.pyt__radd__wscCst|g|S(s<Return a list concatenating self with itself C{count} times.(R(RR((s/p/zhu/06/nlp/nltk/nltk/util.pyt__mul__{scCst|g|S(s<Return a list concatenating self with itself C{count} times.(R(RR((s/p/zhu/06/nlp/nltk/nltk/util.pyt__rmul__si<cCsg}d}x|D]m}|it||t|dd7}||ijo,t|djoddi|d SqqWddi|SdS(s @return: A string representation for this corpus view that is similar to a list's representation; but if it would be more than 60 characters long, it is truncated. iiis [%s, ...]s, s[%s]N(R@RiRt_MAX_REPR_SIZER7(RtpiecestlengthR((s/p/zhu/06/nlp/nltk/nltk/util.pyt__repr__s#cCs7t|ttfpdSntt|t|S(s Return a number indicating how C{self} relates to other. - If C{other} is not a corpus view or a C{list}, return -1. - Otherwise, return C{cmp(list(self), list(other))}. Note: corpus views do not compare equal to tuples containing equal elements. Otherwise, transitivity would be violated, since tuples do not compare equal to lists. i(RRR1tcmp(RR((s/p/zhu/06/nlp/nltk/nltk/util.pyt__cmp__s cCstd|iidS(sH @raise ValueError: Corpus view objects are unhashable. s%s objects are unhashableN(RURR(R((s/p/zhu/06/nlp/nltk/nltk/util.pyt__hash__s(RRDt__doc__RRRRpRRRRRRRRRRRR(((s/p/zhu/06/nlp/nltk/nltk/util.pyR$s            RcBs8eZdZdZdZdZdZdZRS(s A subsequence produced by slicing a lazy sequence. This slice keeps a reference to its source sequence, and generates its values by looking them up in the source sequence. idcCsR|||ijo$tt|i|||Snti||||SdS(s Construct a new slice from a given underlying sequence. The C{start} and C{stop} indices should be absolute indices -- i.e., they should not be negative (for indexing from the back of a list) or greater than the length of C{source}. N(tMIN_SIZER1RRtobjectt__new__(tclstsourceR3R((s/p/zhu/06/nlp/nltk/nltk/util.pyRs$cCs||_||_||_dS(N(t_sourcet_startt_stop(RRR3R((s/p/zhu/06/nlp/nltk/nltk/util.pyR?s  cCs|i|iS(N(RR(R((s/p/zhu/06/nlp/nltk/nltk/util.pyRscCs3t|ii||itdt||S(Ni(RRRRRR(RR3((s/p/zhu/06/nlp/nltk/nltk/util.pyRs(RRDRRRR?RR(((s/p/zhu/06/nlp/nltk/nltk/util.pyRs   RcBs)eZdZdZdZdZRS(s$ A lazy sequence formed by concatenating a list of lists. This underlying list of lists may itself be lazy. C{LazyConcatenation} maintains an index that it uses to keep track of the relationship between offsets in the concatenated lists and offsets in the sublists. cCs||_dg|_dS(Ni(t_listt_offsets(Rt list_of_lists((s/p/zhu/06/nlp/nltk/nltk/util.pyR?s cCsOt|it|ijo%x"|i|idD]}q6Wn|idS(Ni(RRRR(Rttok((s/p/zhu/06/nlp/nltk/nltk/util.pyRsccsq||idjoti|i|d}nt|id}|i|}t|ito|ii|}nt|i|d}x|D]}|t|idjoF|t||idjp t d|ii |t|n-|i|d|t|jp t dx$|t d||D] }|Vq@W|t|7}|d7}qWdS(Niis!offests not monotonic increasing!s"inconsistent list value (num elts)i( Rtbisectt bisect_rightRRRRRRRRR@R(Rt start_indext sublist_indexRt sublist_itertsublistRC((s/p/zhu/06/nlp/nltk/nltk/util.pyRs( !% (RRDRR?RR(((s/p/zhu/06/nlp/nltk/nltk/util.pyRs  tLazyMapcBs2eZdZdZdZdZdZRS(s A lazy sequence whose elements are formed by applying a given function to each element in one or more underlying lists. The function is applied lazily -- i.e., when you read a value from the list, C{LazyMap} will calculate that value by applying its function to the underlying lists' value(s). C{LazyMap} is essentially a lazy version of the Python primitive function C{map}. In particular, the following two expressions are equivalent: >>> map(f, sequences...) >>> list(LazyMap(f, sequences...)) Like the Python C{map} primitive, if the source lists do not have equal size, then the value C{None} will be supplied for the 'missing' elements. Lazy maps can be useful for conserving memory, in cases where individual values take up a lot of space. This is especially true if the underlying list's values are constructed lazily, as is the case with many corpus readers. A typical example of a use case for this class is performing feature detection on the tokens in a corpus. Since featuresets are encoded as dictionaries, which can take up a lot of memory, using a C{LazyMap} can significantly reduce memory usage when training and running classifiers. cOs|ptdn||_||_|idd|_|idjo h|_n d|_td|Dt|j|_ dS(sJ @param function: The function that should be applied to elements of C{lists}. It should take as many arguments as there are C{lists}. @param lists: The underlying lists. @kwparam cache_size: Determines the size of the cache used by this lazy map. (default=5) s"LazyMap requires at least two argst cache_sizeiicss"x|]}t|tVqWdS(N(RR(Rtlst((s/p/zhu/06/nlp/nltk/nltk/util.pys 4s N( t TypeErrort_listst_funcRt _cache_sizet_cacheRRRt _all_lazy(Rtfunctiontliststconfig((s/p/zhu/06/nlp/nltk/nltk/util.pyR?s      c cs}t|idjoA|io7x,|idi|D]}|i|Vq7WdSn#t|idjoTx toEy|i|id|VWntj o dSnX|d7}qpWn|iog}|iD]}||i|q~}x~tog}x<|D]4}y|i|iWq|idqXqW|dgt|ijodSn|i|V|d7}qWnxtoy,g}|iD]}|||q~}Wntj odgt|i}xEt |iD]4\} }y|||| xs(RR(R((s/p/zhu/06/nlp/nltk/nltk/util.pyRws(RRDRR?RRR(((s/p/zhu/06/nlp/nltk/nltk/util.pyRs   ) tLazyMappedListcBseZdZdZRS(sUse LazyMap instead.cCsti|||dS(N(RR?(RRtfunc((s/p/zhu/06/nlp/nltk/nltk/util.pyR?}s(RRDRR?(((s/p/zhu/06/nlp/nltk/nltk/util.pyR{stLazyZipcBs)eZdZdZdZdZRS(sf A lazy sequence whose elements are tuples, each containing the i-th element from each of the argument sequences. The returned list is truncated in length to the length of the shortest argument sequence. The tuples are constructed lazily -- i.e., when you read a value from the list, C{LazyZip} will calculate that value by forming a C{tuple} from the i-th element of each of the argument sequences. C{LazyZip} is essentially a lazy version of the Python primitive function C{zip}. In particular, the following two expressions are equivalent: >>> zip(sequences...) >>> list(LazyZip(sequences...)) Lazy zips can be useful for conserving memory in cases where the argument sequences are particularly long. A typical example of a use case for this class is combining long sequences of gold standard and predicted values in a classification or tagging task in order to calculate accuracy. By constructing tuples lazily and avoiding the creation of an additional long sequence, memory usage can be significantly reduced. cGsti|d|dS(s\ @param lists: the underlying lists @type lists: C{list} of C{list} cWs|S(((telts((s/p/zhu/06/nlp/nltk/nltk/util.pytsN(RR?(RR((s/p/zhu/06/nlp/nltk/nltk/util.pyR?sccsFti||}x-|t|jo|iV|d7}qWdS(Ni(RRRR(RRR((s/p/zhu/06/nlp/nltk/nltk/util.pyRs  cCstd|iDS(Ncssx|]}t|VqWdS(N(R(RR((s/p/zhu/06/nlp/nltk/nltk/util.pys s(tminR(R((s/p/zhu/06/nlp/nltk/nltk/util.pyRs(RRDRR?RR(((s/p/zhu/06/nlp/nltk/nltk/util.pyRs  t LazyEnumeratecBseZdZdZRS(s< A lazy sequence whose elements are tuples, each ontaining a count (from zero) and a value yielded by underlying sequence. C{LazyEnumerate} is useful for obtaining an indexed list. The tuples are constructed lazily -- i.e., when you read a value from the list, C{LazyEnumerate} will calculate that value by forming a C{tuple} from the count of the i-th element and the i-th element of the underlying sequence. C{LazyEnumerate} is essentially a lazy version of the Python primitive function C{enumerate}. In particular, the following two expressions are equivalent: >>> enumerate(sequence) >>> list(LazyEnumerate(sequence)) Lazy enumerations can be useful for conserving memory in cases where the argument sequences are particularly long. A typical example of a use case for this class is obtaining an indexed list for a long sequence of values. By constructing tuples lazily and avoiding the creation of an additional long sequence, memory usage can be significantly reduced. cCs#ti|tt||dS(sL @param lst: the underlying list @type lst: C{list} N(RR?txrangeR(RR((s/p/zhu/06/nlp/nltk/nltk/util.pyR?s(RRDRR?(((s/p/zhu/06/nlp/nltk/nltk/util.pyRscBseZdZdZRS(sUse LazyMap instead.cCsti|||dS(N(RR?(RRR((s/p/zhu/06/nlp/nltk/nltk/util.pyR?s(RRDRR?(((s/p/zhu/06/nlp/nltk/nltk/util.pyRstLazyMappedChaincBseZdZdZRS(s4Use LazyConcatenation(LazyMap(func, lists)) instead.cCsti|t||dS(N(RR?R(RRR((s/p/zhu/06/nlp/nltk/nltk/util.pyR?s(RRDRR?(((s/p/zhu/06/nlp/nltk/nltk/util.pyRsc Csd|d}t|}d}d}t|doti|iid}n.|idd|id}|idx||jo||f}||d} |i| o|| \} } nd} xt o|it d| d| djo|i n|i} |i } | djoPn|| d} | |djot SqqW||jo| | f|| RPRWRR^RoRuRzR~RRRRRRRRRRRRRRRRRRR(((s/p/zhu/06/nlp/nltk/nltk/util.pyssP             :   %  ,  ['/{*!