Ë Øªg…#ãóŽ—dZddlZddlZddlZddlmZddl­ddlm Z ddl ­ddl ­ddl m Z Gd„de«ZGd „d e«Zy) zN A reader for corpora that contain chunked (and optionally tagged) documents. éN)Ú tagstr2tree)Ú*)ÚBracketParseCorpusReader)ÚTreecó‚—eZdZdZdeedd¬«eddfd„Zdd „Zdd „Z dd „Z dd „Z dd „Z dd„Z dd„Zdd„Zdd„Zd„Zy)ÚChunkedCorpusReadera& Reader for chunked (and optionally tagged) corpora. Paragraphs are split using a block reader. They are then tokenized into sentences using a sentence tokenizer. Finally, these sentences are parsed into chunk trees using a string-to-chunktree conversion function. Each of these steps can be performed using a default function or a custom function. By default, paragraphs are split on blank lines; sentences are listed one per line; and sentences are parsed into chunk trees using ``nltk.chunk.tagstr2tree``. ÚÚ T)ÚgapsÚutf8Nc óJ—tj||||«||||f|_y)z’ :param root: The root directory for this corpus. :param fileids: A list or regexp specifying the fileids in this corpus. N)Ú CorpusReaderÚ__init__Ú_cv_args) ÚselfÚrootÚfileidsÚ extensionÚ str2chunktreeÚsent_tokenizerÚpara_block_readerÚencodingÚtagsets úO/var/www/openai/venv/lib/python3.12/site-packages/nltk/corpus/reader/chunked.pyrzChunkedCorpusReader.__init__&s/€ô ×јd D¨'°8Ô<Ø&¨Ð8IÈ6ÐRˆŒ ð Aócóœ—t|j|d«Dcgc] \}}t||ddddg|j¢­Ž‘Œ"c}}«Scc}}w)z~ :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) Tr©ÚconcatÚabspathsÚChunkedCorpusViewr©rrÚfÚencs rÚwordszChunkedCorpusReader.words:sZ€ô ð!%§ ¡ ¨g°tÔ <ô á <‘HQ˜ô" ! S¨!¨Q°°1ÐE°t·}±}ÔEØ <ò ó ð ùó ó›%A cóœ—t|j|d«Dcgc] \}}t||ddddg|j¢­Ž‘Œ"c}}«Scc}}w)z² :return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(list(str)) Trérr!s rÚsentszChunkedCorpusReader.sentsGóZ€ôð!%§ ¡ ¨g°tÔ <ô á <‘HQ˜ô" ! S¨!¨Q°°1ÐE°t·}±}ÔEØ <ò ó ð ùó r%cóœ—t|j|d«Dcgc] \}}t||ddddg|j¢­Ž‘Œ"c}}«Scc}}w)zÜ :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as lists of word strings. :rtype: list(list(list(str))) Trr'rr!s rÚparaszChunkedCorpusReader.parasUr)r%có¢—t|j|d«Dcgc]#\}}t||ddddg|j¢­d|iŽ‘Œ%c}}«Scc}}w)z¾ :return: the given file(s) as a list of tagged words and punctuation symbols, encoded as tuples ``(word,tag)``. :rtype: list(tuple(str,str)) Tr'rÚ target_tagsetr©rrrr"r#s rÚ tagged_wordsz ChunkedCorpusReader.tagged_wordscól€ôð !%§ ¡ ¨g°tÔ <ô  ñ!=‘HQ˜ô"Øs˜A˜q ! QðØ)-¯©òØFLóð!=ò  ó ð ùó ó›(A có¢—t|j|d«Dcgc]#\}}t||ddddg|j¢­d|iŽ‘Œ%c}}«Scc}}w)z­ :return: the given file(s) as a list of sentences, each encoded as a list of ``(word,tag)`` tuples. :rtype: list(list(tuple(str,str))) Tr'rr-rr.s rÚ tagged_sentsz ChunkedCorpusReader.tagged_sentssr0r1có¢—t|j|d«Dcgc]#\}}t||ddddg|j¢­d|iŽ‘Œ%c}}«Scc}}w)zð :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as lists of ``(word,tag)`` tuples. :rtype: list(list(list(tuple(str,str)))) Tr'rr-rr.s rÚ tagged_parasz ChunkedCorpusReader.tagged_parasƒr0r1có¢—t|j|d«Dcgc]#\}}t||ddddg|j¢­d|iŽ‘Œ%c}}«Scc}}w)av :return: the given file(s) as a list of tagged words and chunks. Words are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). Chunks are encoded as depth-one trees over ``(word,tag)`` tuples or word strings. :rtype: list(tuple(str,str) and Tree) Tr'rr-rr.s rÚ chunked_wordsz!ChunkedCorpusReader.chunked_words“ól€ôð !%§ ¡ ¨g°tÔ <ô  ñ!=‘HQ˜ô"Øs˜A˜q ! QðØ)-¯©òØFLóð!=ò  ó ð ùó r1có¢—t|j|d«Dcgc]#\}}t||ddddg|j¢­d|iŽ‘Œ%c}}«Scc}}w)a6 :return: the given file(s) as a list of sentences, each encoded as a shallow Tree. The leaves of these trees are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). :rtype: list(Tree) Tr'rr-rr.s rÚ chunked_sentsz!ChunkedCorpusReader.chunked_sents¥r8r1có¢—t|j|d«Dcgc]#\}}t||ddddg|j¢­d|iŽ‘Œ%c}}«Scc}}w)ao :return: the given file(s) as a list of paragraphs, each encoded as a list of sentences, which are in turn encoded as a shallow Tree. The leaves of these trees are encoded as ``(word, tag)`` tuples (if the corpus has tags) or word strings (if the corpus has no tags). :rtype: list(list(Tree)) Tr'r-rr.s rÚ chunked_parasz!ChunkedCorpusReader.chunked_paras·r8r1cóP—t|«Dcgc] }t|«‘Œc}Scc}w©N)Úread_blankline_blockr)rÚstreamÚts rÚ _read_blockzChunkedCorpusReader._read_blockÉs%€Ü(<¸VÔ(DÓEÑ(D 1” ˜A•Ð(DÑEÐEùÒEsŽ#r>©NN)Ú__name__Ú __module__Ú __qualname__Ú__doc__rÚRegexpTokenizerr?rr$r(r+r/r3r5r7r:r<rB©rrrrs[„ñ ðØ!Ù& t°$Ô7Ø.ØØóAó(  ó  ó  ó ó  ó  ó  ó$ ó$ ó$Frrcó$—eZdZ dd„Zd„Zd„Zy)r Nc ó²—tj|||¬«||_||_||_||_||_||_| |_| |_ | |_ y)N)r) ÚStreamBackedCorpusViewrÚ_taggedÚ_group_by_sentÚ_group_by_paraÚ_chunkedÚ_str2chunktreeÚ_sent_tokenizerÚ_para_block_readerÚ_source_tagsetÚ_target_tagset) rÚfileidrÚtaggedÚ group_by_sentÚ group_by_paraÚchunkedrrrÚ source_tagsetr-s rrzChunkedCorpusView.__init__Îs_€ô ×'Ñ'¨¨f¸xÐ'ÔH؈Œ Ø+ˆÔØ+ˆÔ؈Œ Ø+ˆÔØ-ˆÔØ"3ˆÔØ+ˆÔØ+ˆÕrcóø—g}|j|«D]ã}g}|jj|«D]’}|j||j|j ¬«}|j s|j|«}|js|j«}|jr|j|«Œ‚|j|«Œ”|jr|j|«ŒÓ|j|«Œå|S)N)r[r-)rSrRÚtokenizerQrTrUrMÚ_untagrPÚleavesrNÚappendÚextendrO)rr@ÚblockÚpara_strÚparaÚsent_strÚsents rÚ read_blockzChunkedCorpusView.read_blockçs߀؈Ø×/Ñ/°Ö7ˆH؈DØ ×0Ñ0×9Ñ9¸(ÖCØ×*Ñ*ØØ"&×"5Ñ"5Ø"&×"5Ñ"5ð+óð—|’|ØŸ;™; tÓ,Dð—}’}ØŸ;™;›=Dð×&Ò&Ø—K‘K Õ%à—K‘K Õ%ð'Dð,×"Ò"Ø— ‘ ˜TÕ"à— ‘ ˜TÕ"ð78ð<ˆ rcó¶—t|«D]J\}}t|t«r|j|«Œ(t|t«r |d||<ŒAt d«‚|S)Nrz"expected child to be Tree or tuple)Ú enumerateÚ isinstancerr^ÚtupleÚ ValueError)rÚtreeÚiÚchilds rr^zChunkedCorpusView._untag sT€Ü! $ž‰HˆAˆuܘ%¤Ô&Ø— ‘ ˜EÕ"ܘE¤5Ô)Ø ™(Q’ä Ð!EÓFÐFð (ðˆ rrC)rDrErFrrgr^rIrrr r Ís„ðØó,ò2 óDrr )rGÚcodecsÚos.pathÚosÚnltkÚ nltk.chunkrÚnltk.corpus.reader.apiÚ nltk.corpus.reader.bracket_parserÚnltk.corpus.reader.utilÚ nltk.tokenizeÚ nltk.treerrrrLr rIrrÚrzsGðñó Ûã Ý"Ü$ÝEÜ%ÜÝôpF˜,ôpFôfDÐ.õDr