g$rdZddlmZddlmZmZmZGddeZd dZGdd e Z Gd d eZ y) zACorpus reader for the XML version of the British National Corpus.)concat) ElementTreeXMLCorpusReader XMLCorpusViewcFeZdZdZd dZd dZd dZd dZd dZd dZ d Z y)BNCCorpusReadera7Corpus reader for the XML version of the British National Corpus. For access to the complete XML data structure, use the ``xml()`` method. For access to simple word lists and tagged word lists, use ``words()``, ``sents()``, ``tagged_words()``, and ``tagged_sents()``. You can obtain the full version of the BNC corpus at https://www.ota.ox.ac.uk/desc/2554 If you extracted the archive to a directory called `BNC`, then you can instantiate the reader as:: BNCCorpusReader(root='BNC/Texts/', fileids=r'[A-K]/\w*/\w*\.xml') c@tj|||||_yN)r__init___lazy)selfrootfileidslazys K/var/www/openai/venv/lib/python3.12/site-packages/nltk/corpus/reader/bnc.pyr zBNCCorpusReader.__init__s  tW5 Nc,|j|dd||S)aT :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. FN_viewsr r strip_spacestems rwordszBNCCorpusReader.words#s{{7E4dCCrc8|rdnd}|j|d|||S)a  :return: the given file(s) as a list of tagged words and punctuation symbols, encoded as tuples ``(word,tag)``. :rtype: list(tuple(str,str)) :param c5: If true, then the tags used will be the more detailed c5 tags. Otherwise, the simplified tags will be used. :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. c5posFrr rrrrtags r tagged_wordszBNCCorpusReader.tagged_words/s$de{{7E3 TBBrc,|j|dd||S)a :return: the given file(s) as a list of sentences or utterances, each encoded as a list of word strings. :rtype: list(list(str)) :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. TNrrs rsentszBNCCorpusReader.sents?s{{7D$ TBBrc:|rdnd}|j|d|||S)a :return: the given file(s) as a list of sentences, each encoded as a list of ``(word,tag)`` tuples. :rtype: list(list(tuple(str,str))) :param c5: If true, then the tags used will be the more detailed c5 tags. Otherwise, the simplified tags will be used. :param strip_space: If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens. :param stem: If true, then use word stems instead of word strings. rrT)sentrrrrrs r tagged_sentszBNCCorpusReader.tagged_sentsLs.de{{ $C[t  rc |jrtn |j}t|j |Dcgc]}||||||c}Scc}w)zPA helper function that instantiates BNCWordViews or the list of words/sentences.)r BNCWordView_wordsrabspaths)r rr#rrrffileids rrzBNCCorpusReader._views]sX::K4;;#mmG4 4F&$[$74    sAc 8g}tj|j}|jdD]}g} t |D]} | j } | sd} |s|r| j } |r| jd| } |dk(r| | jdf} n(|dk(r#| | jd| jdf} | j| |r)|jt|jd| |j| d|vsJ|S)a Helper used to implement the view methods -- returns a list of words or a list of sentences, optionally tagged. :param fileid: The name of the underlying file. :param bracket_sent: If true, include sentence bracketing. :param tag: The name of the tagset to use, or None for no tags. :param strip_space: If true, strip spaces from word tokens. :param stem: If true, then substitute stems for words. z.//shwrrnN) rparsegetrootfindall_all_xmlwords_intextstripgetappend BNCSentenceattribextend) r r* bracket_sentrrrresultxmldocxmlsentr#xmlwordwords rr'zBNCCorpusReader._wordsgs ""6*224~~f-GD+G4||D$::||_tj||yr )numlistr )r rLitemss rr zBNCSentence.__init__s dE"rN)r@rArBrCr rDrrr7r7s  #rr7c8eZdZdZhdZ dZdZdZdZdZ y) r&zN A stream backed corpus view specialized for use with the BNC corpus. >pbgapaligneventpauseshiftvocalunclearcP|rd}nd}||_||_||_||_d|_d|_d|_d|_tj||||j|j|jd|j|jddi|_y)aG :param fileid: The name of the underlying file. :param sent: If true, include sentence bracketing. :param tag: The name of the tagset to use, or None for no tags. :param strip_space: If true, strip spaces from word tokens. :param stem: If true, then substitute stems for words. z.*/sz.*/s/(.*/)?(c|w)Nz .*/teiHeader$rrD)_sent_tag _strip_space_stemtitleauthoreditorrespsrr _open read_block_stream handle_headerclose _tag_context)r r*r#rrrtagspecs rr zBNCWordView.__init__s G(G  '     tVW5   ot7I7IJ Grc|jd}|rdjd|D|_|jd}|rdjd|D|_|jd}|rdjd|D|_|jd}|rd jd |D|_yy) NztitleStmt/title c3PK|]}|jj ywr r3r4).0r]s r z,BNCWordView.handle_header..s"J6%5::#3#3#56$&ztitleStmt/authorc3PK|]}|jj ywr rk)rlr^s rrmz,BNCWordView.handle_header..#NgFFKK$5$5$7grnztitleStmt/editorc3PK|]}|jj ywr rk)rlr_s rrmz,BNCWordView.handle_header..rprnztitleStmt/respStmtz c3LK|]}djd|Dyw)ric3PK|]}|jj ywr rk)rlresp_elts rrmz6BNCWordView.handle_header...sEH(----/rnN)join)rlresps rrmz,BNCWordView.handle_header..s$%RW$ EEERWs"$)r1rur]r^r_r`)r rHcontexttitlesauthorseditorsr`s rrdzBNCWordView.handle_headers./ "J6"JJDJ++01 ))#Ng#NNDK++01 ))#Ng#NNDK 01 %RW%DJ rc^|jr|j|S|j|Sr )rY handle_sent handle_word)r rHrws r handle_eltzBNCWordView.handle_elts+ ::##C( (##C( (rc^|j}|sd}|js |jr|j}|jr|j d|}|j dk(r||j df}|S|j dk(r#||j d|j df}|S)Nr,r-rr)r3r[r\r4r5rZ)r rHr?s rr}zBNCWordView.handle_wordsxxD    ::rsBH*RR|o|~#$#f2-f2r