gYddlZddlmZddlmZmZddlmZmZGdde Z eGddZ Gd d e Z Gd d e Z Gd de ZGddeZGddeZGdde ZGdde Zy)N)total_ordering)chainislice)raise_unorderable_types slice_boundscfeZdZddZdZdZdZdZdZdZ d Z d Z dd Z d Z dd ZdZdZy) OrderedDictNc |j||jd|_|jd|_|tj |ytj ||y)Nkeysdefault_factory)r get_keys_default_factorydict__init__)selfdatakwargss E/var/www/openai/venv/lib/python3.12/site-packages/nltk/collections.pyrzOrderedDict.__init__sNYYtVZZ%78 & +< = < MM$  MM$ %cftj|||jj|yN)r __delitem__rremoverkeys rrzOrderedDict.__delitem__s$ s# #rcp tj||S#t$r|j|cYSwxYwr)r __getitem__KeyError __missing__rs rrzOrderedDict.__getitem__ s: )##D#. . )##C( ( )s 55c0d|jDS)Nc3 K|]}|ywr).0rs r z'OrderedDict.__iter__..'s+{{s )r rs r__iter__zOrderedDict.__iter__&s+tyy{++rcj|js||jvr t|jSr)rrrrs rr zOrderedDict.__missing__)s-$$DJJ)>* $$&&rctj|||||jvr|jj|yyr)r __setitem__rappend)rritems rr*zOrderedDict.__setitem__.s7 sD) djj JJ  c " !rcbtj||jjyr)rclearrr&s rr.zOrderedDict.clear3s 4 rcRtj|}|j|_|Sr)rcopyr)rds rr0zOrderedDict.copy7s IIdO**rcRt|j|jSr)zipr valuesr&s ritemszOrderedDict.items<s499; ..rc|r|r-t|tsJt|t|k(sJ|St|ts"t|tst|tsJt|tst|tr|j St|tr|Dcgc]\}}| c}}Syd|j vr |jSgScc}}w)Nr) isinstancelistlenrr r __dict__r)rrr rvalues rr zOrderedDict.keys?s !$---4yCI--- tT*!$ 4!$-.dD)Zk-J99;&d+489DLS%CD99,  %:: I :s) Ccz|js t|jj}||}||=||fSr)rrpop)rrr;s rpopitemzOrderedDict.popitemTs;zz* jjnnS  IU|rctj|||||jvr|jj|yyr)r setdefaultrr+)rrfailobjs rr@zOrderedDict.setdefault]s5 c7+ djj JJ  c " !rctj|||j|D],}||jvs|jj |.yr)rupdater rr+)rrrs rrCzOrderedDict.updatebs@ D$99T?C$**$ !!#&#rcBt|j|jSr)mapr rr&s rr4zOrderedDict.valueshs488TZZ((rrNN)__name__ __module__ __qualname__rrrr'r r*r.r0r5r r>r@rCr4r#rrr r sH&) ,' #  /*# ' )rr cveZdZdZdZdZdZdZdZddZ d Z d Z d Z d Z d ZdZdZdZdZdZdZy)AbstractLazySequenceaG An abstract base class for read-only sequences whose values are computed as needed. Lazy sequences act like tuples -- they can be indexed, sliced, and iterated over; but they may not be modified. The most common application of lazy sequences in NLTK is for corpus view objects, which provide access to the contents of a corpus without loading the entire corpus into memory, by loading pieces of the corpus from disk as needed. The result of modifying a mutable element of a lazy sequence is undefined. In particular, the modifications made to the element may or may not persist, depending on whether and when the lazy sequence caches that element's value or reconstructs it from scratch. Subclasses are required to define two methods: ``__len__()`` and ``iterate_from()``. ctd)ze Return the number of tokens in the corpus file underlying this corpus view. !should be implemented by subclassNotImplementedErrorr&s r__len__zAbstractLazySequence.__len__s ""EFFrctd)z Return an iterator that generates the tokens in the corpus file underlying this corpus view, starting at the token number ``start``. If ``start>=len(self)``, then this iterator will generate no tokens. rMrNrstarts r iterate_fromz!AbstractLazySequence.iterate_froms""EFFrct|trt||\}}t|||S|dkr|t |z }|dkr t d t |j|S#t$r}t d|d}~wwxYw)z Return the *i* th token in the corpus file underlying this corpus view. Negative indices and spans are both supported. rindex out of rangeN) r7slicerLazySubsequencer9 IndexErrornextrT StopIteration)rirSstopes rrz AbstractLazySequence.__getitem__s a &tQ/KE4"45 51uSY1u !566 >D--a011  > !56A= >sA++ B4 BBc$|jdS)zdReturn an iterator that generates the tokens in the corpus file underlying this corpus view.r)rTr&s rr'zAbstractLazySequence.__iter__s  ##rc,tfd|DS)z8Return the number of times this list contains ``value``.c3.K|] }|k(s dyw)Nr#)r$eltr;s rr%z-AbstractLazySequence.count..s5u 1s )sumrr;s `rcountzAbstractLazySequence.counts5555rNct|t||\}}tt|||D]\}}||k(s ||zcSt d)aReturn the index of the first occurrence of ``value`` in this list that is greater than or equal to ``start`` and less than ``stop``. Negative start and stop values are treated like negative slice bounds -- i.e., they count from the end of the list.zindex(x): x not in list)rrW enumerater ValueError)rr;rSr]r\rcs rindexzAbstractLazySequence.indexsV #4ud);< ttUD 9:FAse|5y ;233rc6t|j|S)z,Return true if this list contains ``value``.)boolrfres r __contains__z!AbstractLazySequence.__contains__sDJJu%&&rct||gSz,Return a list concatenating self with other.LazyConcatenationrothers r__add__zAbstractLazySequence.__add__s $//rct||gSz,Return a list concatenating other with self.rprrs r__radd__zAbstractLazySequence.__radd__s %//rc t|g|zSz=Return a list concatenating self with itself ``count`` times.rprrfs r__mul__zAbstractLazySequence.__mul__ $%00rc t|g|zSryrprzs r__rmul__zAbstractLazySequence.__rmul__r|r<c g}d}|D]g}|jt||t|ddzz }||jkDsAt|dkDsPddj |ddzcSddj |zS)z Return a string representation for this corpus view that is similar to a list's representation; but if it would be more than 60 characters long, it is truncated. z [%s, ...]z, Nz[%s])r+reprr9_MAX_REPR_SIZEjoin)rpieceslengthrcs r__repr__zAbstractLazySequence.__repr__s C MM$s) $ c&*o) )F+++F a"TYYvcr{%;;;   &)))rcbt|t|k(xrt|t|k(Sr)typer8rrs r__eq__zAbstractLazySequence.__eq__s'DzT%[(FT$Z4;-FFrc||k( Srr#rrs r__ne__zAbstractLazySequence.__ne__s5=  rcxt|t|k7r td||t|t|kS)N<)rrr8rrs r__lt__zAbstractLazySequence.__lt__s1 ;$t* $ #Cu 5DzDK''rcFtd|jjz)zH :raise ValueError: Corpus view objects are unhashable. z%s objects are unhashable)ri __class__rGr&s r__hash__zAbstractLazySequence.__hash__s4t~~7N7NNOOrrF)rGrHrI__doc__rPrTrr'rfrjrmrtrwr{r~rrrrrrr#rrrKrKqsb(GG>($ 6 4'0011N *G!( PrrKc.eZdZdZdZ dZdZdZdZy)rXz A subsequence produced by slicing a lazy sequence. This slice keeps a reference to its source sequence, and generates its values by looking them up in the source sequence. dc||z |jkr'tt|j|||z Stj |S)a Construct a new slice from a given underlying sequence. The ``start`` and ``stop`` indices should be absolute indices -- i.e., they should not be negative (for indexing from the back of a list) or greater than the length of ``source``. )MIN_SIZEr8rrTobject__new__)clssourcerSr]s rrzLazySubsequence.__new__sD %<#,, &v22594%<HI I>>#& &rc.||_||_||_yr)_source_start_stop)rrrSr]s rrzLazySubsequence.__init__s   rc4|j|jz Sr)rrr&s rrPzLazySubsequence.__len__szzDKK''rc t|jj||jzt dt ||z SNr)rrrTrmaxr9rRs rrTzLazySubsequence.iterate_froms< LL % %edkk&9 :C3t9uCTsEE4E2E4N)rGrHrIrrrPrTr#rrrqrqs! rrqc(eZdZdZdZdZdZdZy)LazyMapa A lazy sequence whose elements are formed by applying a given function to each element in one or more underlying lists. The function is applied lazily -- i.e., when you read a value from the list, ``LazyMap`` will calculate that value by applying its function to the underlying lists' value(s). ``LazyMap`` is essentially a lazy version of the Python primitive function ``map``. In particular, the following two expressions are equivalent: >>> from nltk.collections import LazyMap >>> function = str >>> sequence = [1,2,3] >>> map(function, sequence) # doctest: +SKIP ['1', '2', '3'] >>> list(LazyMap(function, sequence)) ['1', '2', '3'] Like the Python ``map`` primitive, if the source lists do not have equal size, then the value None will be supplied for the 'missing' elements. Lazy maps can be useful for conserving memory, in cases where individual values take up a lot of space. This is especially true if the underlying list's values are constructed lazily, as is the case with many corpus readers. A typical example of a use case for this class is performing feature detection on the tokens in a corpus. Since featuresets are encoded as dictionaries, which can take up a lot of memory, using a ``LazyMap`` can significantly reduce memory usage when training and running classifiers. c|s td||_||_|jdd|_|jdkDrind|_t d|Dt|k(|_y)aJ :param function: The function that should be applied to elements of ``lists``. It should take as many arguments as there are ``lists``. :param lists: The underlying lists. :param cache_size: Determines the size of the cache used by this lazy map. (default=5) z"LazyMap requires at least two args cache_sizerrNc3<K|]}t|tywr)r7rKr$lsts rr%z#LazyMap.__init__..s =BcJs0 1Us) TypeError_lists_funcr _cache_size_cacherdr9 _all_lazy)rfunctionlistsconfigs rrzLazyMap.__init__tsq@A A  !::lA6 ,,q0bd  =B  Zrc#"Kt|jdk(rC|jr7|jdj|D]}|j |yt|jdk(r+ |j |jd||dz }*|jr|jDcgc]}|j|}} g}|D]} |j t||dgt|jzk(ry|j||dz }X |jDcgc]}|| }}|j||dz }4#t $rYywxYwcc}w#|j dYxYwcc}w#t $rodgt|jz}t|jD]\}} ||||<#t $rYwxYw|dgt|jzk(rYyYwxYww)Nrbr) r9rrrTrrYr+rZrh)rrjr;r iteratorselementsiteratorr\s rrTzLazyMap.iterate_froms t{{ q T^^Q44U;jj''<   "**T[[^E%:;;  ^^<@KKHKS))%0KIH )H. X7!* vDKK(888 djj(++  6:kkBksE kHB!djj(++ +" I . - C! $vDKK(88H"+DKK"83!*-e*HQK)! !#9  D6C ,<#<<=sA5H8#E# H;E2 HE787H0F> F F H# E/,H.E//H7F  HF=H GH  G'$H &G'' H H H  HcJt|tr2|jDcgc]}|| }}t|jg|S|dkr|t |z }|dkr t d|j||jvr|j|S t|j|}|jKt |j|jkDr|jj||j|<|Scc}w#t$r}t d|d}~wwxYw)NrrV) r7rWrrrr9rYrrZrTr[rr>)rrjr sliced_listsvalr^s rrzLazyMap.__getitem__s eU #26++>+3CJ+L>4::5 5 5qyT"qy !566{{&5DKK+?{{5)) >4,,U34{{&t{{#d&6&66KK'')%( E"J-?! > !56A= >s DD D" DD"c:td|jDS)Nc32K|]}t|ywrr9rs rr%z"LazyMap.__len__..3{3s8{)rrr&s rrPzLazyMap.__len__3t{{333rN)rGrHrIrrrTrrPr#rrrrQs D0.`44rrc"eZdZdZdZdZdZy)LazyZipa A lazy sequence whose elements are tuples, each containing the i-th element from each of the argument sequences. The returned list is truncated in length to the length of the shortest argument sequence. The tuples are constructed lazily -- i.e., when you read a value from the list, ``LazyZip`` will calculate that value by forming a tuple from the i-th element of each of the argument sequences. ``LazyZip`` is essentially a lazy version of the Python primitive function ``zip``. In particular, an evaluated LazyZip is equivalent to a zip: >>> from nltk.collections import LazyZip >>> sequence1, sequence2 = [1, 2, 3], ['a', 'b', 'c'] >>> zip(sequence1, sequence2) # doctest: +SKIP [(1, 'a'), (2, 'b'), (3, 'c')] >>> list(LazyZip(sequence1, sequence2)) [(1, 'a'), (2, 'b'), (3, 'c')] >>> sequences = [sequence1, sequence2, [6,7,8,9]] >>> list(zip(*sequences)) == list(LazyZip(*sequences)) True Lazy zips can be useful for conserving memory in cases where the argument sequences are particularly long. A typical example of a use case for this class is combining long sequences of gold standard and predicted values in a classification or tagging task in order to calculate accuracy. By constructing tuples lazily and avoiding the creation of an additional long sequence, memory usage can be significantly reduced. c4tj|dg|y)zT :param lists: the underlying lists :type lists: list(list) c|Srr#)eltss rz"LazyZip.__init__..sTrN)rr)rrs rrzLazyZip.__init__s 1:E:rc#Ktj||}|t|kr!t||dz }|t|kr!yw)Nrb)rrTr9rZ)rrjrs rrTzLazyZip.iterate_fromsJ''e4c$ix. QJEc$i s AA A c:td|jDS)Nc32K|]}t|ywrrrs rr%z"LazyZip.__len__.. rr)minrr&s rrPzLazyZip.__len__rrN)rGrHrIrrrTrPr#rrrrs>;4rrceZdZdZdZy) LazyEnumeratea A lazy sequence whose elements are tuples, each containing a count (from zero) and a value yielded by underlying sequence. ``LazyEnumerate`` is useful for obtaining an indexed list. The tuples are constructed lazily -- i.e., when you read a value from the list, ``LazyEnumerate`` will calculate that value by forming a tuple from the count of the i-th element and the i-th element of the underlying sequence. ``LazyEnumerate`` is essentially a lazy version of the Python primitive function ``enumerate``. In particular, the following two expressions are equivalent: >>> from nltk.collections import LazyEnumerate >>> sequence = ['first', 'second', 'third'] >>> list(enumerate(sequence)) [(0, 'first'), (1, 'second'), (2, 'third')] >>> list(LazyEnumerate(sequence)) [(0, 'first'), (1, 'second'), (2, 'third')] Lazy enumerations can be useful for conserving memory in cases where the argument sequences are particularly long. A typical example of a use case for this class is obtaining an indexed list for a long sequence of values. By constructing tuples lazily and avoiding the creation of an additional long sequence, memory usage can be significantly reduced. cVtj|tt||y)zI :param lst: the underlying list :type lst: list N)rrranger9)rrs rrzLazyEnumerate.__init__)s uSX4rN)rGrHrIrrr#rrrr s 85rrc0eZdZdZddZdZdZdZdZy) LazyIteratorListz Wraps an iterator, loading its elements on demand and making them subscriptable. __repr__ displays only the first few elements. Nc.||_||_g|_yr)_it_lenr)rit known_lens rrzLazyIteratorList.__init__8s  rc|jr |jS|jt|jD]}t|j|_|jSr)rrTr9rrs rrPzLazyIteratorList.__len__=sN 9999 ""3t{{#34A 5 $ yyrc#Kt|j|krIt|j}|jj |t|j|krI|}|t|jkr/|j||dz }|t|jkr/ t|j}|jj ||5#t $rYywxYww)zBCreate a new iterator over this list starting at the given offset.rbN)r9rrZrr+r[)rrSvr\s rrTzLazyIteratorList.iterate_fromEs$++&TXXA KK  q !$++& #dkk""++a. FA#dkk"" N ""1%   s+A!C3$AC3.6C$$ C0-C3/C00C3c8t|t||Srorrrrs rrtzLazyIteratorList.__add__VstDz%e,--rc8t|t||Srvrrrs rrwzLazyIteratorList.__radd__ZstDz%t,--rr) rGrHrIrrrPrTrtrwr#rrrr1s   "..rrc4eZdZdZdZdfd ZdZdZxZS)Triez!A Trie implementation for stringsTcXt||r|D]}|j|yy)ayBuilds a Trie object, which is built around a ``dict`` If ``strings`` is provided, it will add the ``strings``, which consist of a ``list`` of ``strings``, to the Trie. Otherwise, it'll construct an empty Trie. :param strings: List of strings to insert into the trie (Default is ``None``) :type strings: list(str) N)superrinsert)rstringsstringrs rrz Trie.__init__gs.  ! F#" rcvt|r||dj|ddyd|tj<y)aInserts ``string`` into the Trie :param string: String to insert into the trie :type string: str :Example: >>> from nltk.collections import Trie >>> trie = Trie(["abc", "def"]) >>> expected = {'a': {'b': {'c': {True: None}}}, 'd': {'e': {'f': {True: None}}}} >>> trie == expected True rrbN)r9rrLEAF)rrs rrz Trie.insertxs6 v; O " "6!": .#DOrc&t||<||Sr)rrs rr zTrie.__missing__sFS Cyrr) rGrHrIrrrrr __classcell__)rs@rrrbs+ D$"#,rr)r functoolsr itertoolsrrnltk.internalsrrrr rKrXrqrrrrrr#rrrs$#@V)$V)|APAPAPH& *& R/,/dF4"F4R/4g/4d"5G"5J+.++.b.4.r