g.dZddlZddlddlejdZejdZejdZejdZGdd Z Gd d Z Gd d e Z y)a CorpusReader for reviews corpora (syntax based on Customer Review Corpus). Customer Review Corpus information ================================== Annotated by: Minqing Hu and Bing Liu, 2004. Department of Computer Science University of Illinois at Chicago Contact: Bing Liu, liub@cs.uic.edu https://www.cs.uic.edu/~liub Distributed with permission. The "product_reviews_1" and "product_reviews_2" datasets respectively contain annotated customer reviews of 5 and 9 products from amazon.com. Related papers: - Minqing Hu and Bing Liu. "Mining and summarizing customer reviews". Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-04), 2004. - Minqing Hu and Bing Liu. "Mining Opinion Features in Customer Reviews". Proceedings of Nineteeth National Conference on Artificial Intelligence (AAAI-2004), 2004. - Xiaowen Ding, Bing Liu and Philip S. Yu. "A Holistic Lexicon-Based Appraoch to Opinion Mining." Proceedings of First ACM International Conference on Web Search and Data Mining (WSDM-2008), Feb 11-12, 2008, Stanford University, Stanford, California, USA. Symbols used in the annotated reviews: :[t]: the title of the review: Each [t] tag starts a review. :xxxx[+|-n]: xxxx is a product feature. :[+n]: Positive opinion, n is the opinion strength: 3 strongest, and 1 weakest. Note that the strength is quite subjective. You may want ignore it, but only considering + and - :[-n]: Negative opinion :##: start of each sentence. Each line is a sentence. :[u]: feature not appeared in the sentence. :[p]: feature not appeared in the sentence. Pronoun resolution is needed. :[s]: suggestion or recommendation. :[cc]: comparison with a competing product from a different brand. :[cs]: comparison with a competing product from the same brand. Note: Some of the files (e.g. "ipod.txt", "Canon PowerShot SD500.txt") do not provide separation between different reviews. This is due to the fact that the dataset was specifically designed for aspect/feature-based sentiment analysis, for which sentence-level annotation is sufficient. For document- level classification and analysis, this peculiarity should be taken into consideration. N)*z ^\[t\](.*)$z%((?:(?:\w+\s)+)?\w+)\[((?:\+|\-)\d)\]z\[(?!t)(p|u|s|cc|cs)\]z##(.*)$c0eZdZdZddZdZdZdZdZy) Reviewz> A Review is the main block of a ReviewsCorpusReader. Nc4||_|g|_y||_y)z :param title: the title of the review. :param review_lines: the list of the ReviewLines that belong to the Review. N)title review_lines)selfrrs O/var/www/openai/venv/lib/python3.12/site-packages/nltk/corpus/reader/reviews.py__init__zReview.__init__Rs!    "D  ,D c^t|tsJ|jj|y)z Add a line (ReviewLine) to the review. :param review_line: a ReviewLine instance that belongs to the Review. N) isinstance ReviewLinerappendr review_lines r add_linezReview.add_line]s( +z222   -r cbg}|jD]}|j|j|S)a Return a list of features in the review. Each feature is a tuple made of the specific item feature and the opinion strength about that feature. :return: all features of the review as a list of tuples (feat, score). :rtype: list(tuple) )rextendfeatures)r rrs r rzReview.featuresfs0,,K OOK00 1-r cT|jDcgc]}|jc}Scc}w)z Return all tokenized sentences in the review. :return: all sentences of the review as lists of tokens. :rtype: list(list(str)) )rsentrs r sentsz Review.sentsss*594E4EF4E[   4EFFFs%cNdj|j|jS)Nz#Review(title="{}", review_lines={}))formatrrr s r __repr__zReview.__repr__|s$4;; JJ))  r NN) __name__ __module__ __qualname____doc__r rrrrr r rrMs! -. G r rceZdZdZddZdZy)rz A ReviewLine represents a sentence of the review, together with (optional) annotations of its features and notes about the reviewed item. NcV||_|g|_n||_|g|_y||_yNrrnotes)r rrr(s r r zReviewLine.__init__s0  DM$DM =DJDJr cddj|j|j|jS)Nz*ReviewLine(features={}, notes={}, sent={}))rrr(rrs r rzReviewLine.__repr__s(;BB MM4::tyy  r r)rr r!r"r rr#r r rrs  r rcbeZdZdZeZedfdZd dZd dZ d dZ d dZ d Z d Z d Zd Zy)ReviewsCorpusReadera Reader for the Customer Review Data dataset by Hu, Liu (2004). Note: we are not applying any sentence tokenization at the moment, just word tokenization. >>> from nltk.corpus import product_reviews_1 >>> camera_reviews = product_reviews_1.reviews('Canon_G3.txt') >>> review = camera_reviews[0] >>> review.sents()[0] # doctest: +NORMALIZE_WHITESPACE ['i', 'recently', 'purchased', 'the', 'canon', 'powershot', 'g3', 'and', 'am', 'extremely', 'satisfied', 'with', 'the', 'purchase', '.'] >>> review.features() # doctest: +NORMALIZE_WHITESPACE [('canon powershot g3', '+3'), ('use', '+2'), ('picture', '+2'), ('picture quality', '+1'), ('picture quality', '+1'), ('camera', '+2'), ('use', '+2'), ('feature', '+1'), ('picture quality', '+3'), ('use', '+1'), ('option', '+1')] We can also reach the same information directly from the stream: >>> product_reviews_1.features('Canon_G3.txt') [('canon powershot g3', '+3'), ('use', '+2'), ...] We can compute stats for specific product features: >>> n_reviews = len([(feat,score) for (feat,score) in product_reviews_1.features('Canon_G3.txt') if feat=='picture']) >>> tot = sum([int(score) for (feat,score) in product_reviews_1.features('Canon_G3.txt') if feat=='picture']) >>> mean = tot / n_reviews >>> print(n_reviews, tot, mean) 15 24 1.6 utf8cPtj||||||_d|_y)ad :param root: The root directory for the corpus. :param fileids: a list or regexp specifying the fileids in the corpus. :param word_tokenizer: a tokenizer for breaking sentences or paragraphs into words. Default: `WordPunctTokenizer` :param encoding: the encoding that should be used to read the corpus. z README.txtN) CorpusReaderr _word_tokenizer_readme)r rootfileidsword_tokenizerencodings r r zReviewsCorpusReader.__init__s' dD'8<-# r Nc | |j}nt|tr|g}t|j |dDcgc]#\}}|j ||j |%c}}Scc}}w)au Return a list of features. Each feature is a tuple made of the specific item feature and the opinion strength about that feature. :param fileids: a list or regexp specifying the ids of the files whose features have to be returned. :return: all features for the item(s) in the given file(s). :rtype: list(tuple) Tr4)_fileidsrstrconcatabspaths CorpusView_read_featuresr r2fileidencs r rzReviewsCorpusReader.featuressv ?mmG  %iG&*]]7D%A %AMVS(;(;cJ%A    s(A- c | |j}t|j|dDcgc]#\}}|j||j|%c}}Scc}}w)aS Return all the reviews as a list of Review objects. If `fileids` is specified, return all the reviews from each of the specified files. :param fileids: a list or regexp specifying the ids of the files whose reviews have to be returned. :return: the given file(s) as a list of reviews. Tr6)r7r9r:r;_read_review_blockr=s r reviewszReviewsCorpusReader.reviewsse ?mmG&*]]7D%A %AMVS(?(?#N%A    s(A c t|j|ddDcgc]$\}}}|j||j|&c}}}Scc}}}w)aY Return all sentences in the corpus or in the specified files. :param fileids: a list or regexp specifying the ids of the files whose sentences have to be returned. :return: the given file(s) as a list of sentences, each encoded as a list of word strings. :rtype: list(list(str)) Tr6)r9r:r;_read_sent_blockr r2pathr?r>s r rzReviewsCorpusReader.sents\,0==$+M +M'T3d&;&;cJ+M    )A c t|j|ddDcgc]$\}}}|j||j|&c}}}Scc}}}w)aK Return all words and punctuation symbols in the corpus or in the specified files. :param fileids: a list or regexp specifying the ids of the files whose words have to be returned. :return: the given file(s) as a list of words and punctuation symbols. :rtype: list(str) Tr6)r9r:r;_read_word_blockrEs r wordszReviewsCorpusReader.wordsrGrHcg}tdD]A}|j}|s|cS|jtjt |C|S)N)rangereadlinerrefindallFEATURES)r streamrilines r r<z"ReviewsCorpusReader._read_featuressGrA??$D OOBJJx6 7  r c~ |j}|sgStjt|}|r*t |j dj }n[ |j}|j}|s|gStjt|r|j||gStjt|}tjt|}tjt|}|r|jj|d}t|||} |j!| )N)rrr')rOrPmatchTITLErgroupstriptellseekrQrRNOTESSENTr/tokenizerr) r rSrU title_matchreviewoldposfeatsr(rrs r rAz&ReviewsCorpusReader._read_review_blocks ??$D ((5$/K%++A.446[[]F??$Dxxxt$ F#xJJx.EJJud+E::dD)D++44T!W=$$eLK OOK (%r cg}|j|D]/}|j|jDcgc]}|c}1|Scc}wr&)rArr)r rSrrbrs r rDz$ReviewsCorpusReader._read_sent_block>sG--f5F LL6<<>:>4$>: ;6 ;s A cg}tdD]\}|j}tjt|}|s0|j |j j|d^|S)NrMr)rNrOrPrQr_rr/r`)r rSrKrTrUrs r rJz$ReviewsCorpusReader._read_word_blockDs\rA??$D::dD)D T11::47CD   r r&)rr r!r"StreamBackedCorpusViewr;WordPunctTokenizerr rrBrrKr<rArDrJr#r r r+r+sF>(J-?,@6 $ * $ " ")B r r+) r"rPnltk.corpus.reader.api nltk.tokenizecompilerYrRr^r_rrr.r+r#r r rls6p $ >" 2::,   ,-rzz*2 2 j  0q,qr