Aspect extraction (A survey)

Aspect-Level Sentiment Classification for Arabic Language

Work Plan

By: Mahmoud El-RazzazISSR, Cairo UniversityCairo, Egypt

Work Plan

Mining Aspects Collect Dataset Apply Aspect Mining to dataset collected Conduct aspect-level sentiment classifier Preview results & compare it with same

classifiers conducted for other languages Conclusion Future work

Mining Aspects Collect Dataset Apply Aspect Mining to dataset collected Conduct aspect-level sentiment classifier Preview results & compare it with same

classifiers conducted for other languages Conclusion Future work

Work Plan

Aspect Mining

Vocabulary:› Aspect[1] and feature[2]

› The two terms are used in the literature as synonyms and represents the opinion target.

› Simply aspect here means a feature of a product e.g. “cast” and “script” are a features of a movie

[1] Na, J.-C., Khoo, C. S. G.. Aspect-based sentiment analysis of movie reviews on discussion boards. 2010.[2] Hu, Minqing and Bing Liu. mining and summarization customer reviews. In proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2004). 2004.

References

Aspect Mining

Aspect mining or Aspect Extraction: For example “ the voice quality of this

phone is amazing” The aspect is “voice quality” of entity

represented by “this phone”

it is possible that in an application the opinion targets are given because the user is onlyInterested in these particular targets (e.g., the BMW and Ford brands)

note

Aspect Mining

An opinion typically always has a target.

The target is often the aspect to be extracted from a sentence.

Thus it is important to recognize each opinion expression and its target from a sentence.

some opinion expressions can play two rules, indicating a sentiment and implying an (implicit) aspect (target). For example, in “this car is expensive” is a sentiment word also indicates the aspect “price”

note

Aspect Mining

There are four main approaches for aspect extraction:

1. Extraction based on frequent nouns and

noun phrases.

2. Extraction by exploiting opinion and

target relations.

3. Extraction using supervised learning.

4. Extraction using topic modeling.

Aspect Mining


1. Extraction based on frequent nouns

and noun phrases.


target relations.



Aspect Extraction based on frequent nouns

This method finds explicit expressions that are nouns and noun phrases from a large number of reviews in a given domain.

Hu and Liu (2004) used a data mining algorithm.

Nouns and noun phrases were identified by a part-of-peach (POS) tagger.

Their occurrence frequency is counted and only frequent ones are kept.


The reason that this approach works is that when people comment on different aspects of an entity, the vocabulary that they use usually converges.

Irrelevant content in reviews are often diverse.

The precision of this algorithm was improved in (Popescu and Etzioni, 2005)[1]

[1] N Popescu, Ana-Maria and Oren Etzioni. Extracting product features and opinions from reviews. In proceedings of Conference on Empirical Methods in Natural Language Processing (EMNLP-2005). 2005.

References


More references for aspect extraction based on frequent nouns:› Blair-Goldensohn et al. (2008)[1]

In this approach several filters were applied to remove unlikely aspects, e.g., dropping aspects which do not have sufficient mentions along-side down sentiment words.

Also they collapsed aspects at the word stem level.

[1] Blair-Goldensohn, Sasha, Kerry Hannan, Ryan Mcdonald, Tyler Neylon, George A. Reis, and Jeff Reyner. Building a sentiment summarizer for local service reviews. In proceedings of WWW-2008 workshop on NLP in the information Explosion Era. 2008.

References


More references for aspect extraction based on frequent nouns:› Ku, Liang and Chen, (2006)[1]

The authors made use of TF-IDF scheme considering terms at the document level and the paragraph level.

[1] Ku, Lun-Weim Yu-Ting Liang, and Hsin-His Chen. Opinion extraction, summarization andTracking in news and blog corpora. In proceedings of AAAI-CAAW’06. 2006.

References


More references for aspect extraction based on frequent nouns:› Moghaddam and Ester, (2010)[1]

The authors augmented the frequency-based approach with an additional filter to remove some non-aspect nouns.

Their work also predicted aspect ratings.

[1] Moghaddam, Samaneh and Martin Ester. ILDA: interdependent LDA model for learning latent aspects and their ratings from online product reviews. in Proceedings of the Annual ACM SIGIR International conference on Research and Development in Information Retrieval (SIGIR- 2011). 2011.

References


More references for aspect extraction based on frequent nouns:› Scaffidi et al., (2007)[1]

The authors compared the frequency of extracted frequent nouns in a review corpus with their occurrence rates in generic English corpus to identify true aspects.

[1] Scaffidi, Christopher, Kevin Bierhoff, Eric Chang, Mikhael Felker, Herman Ng, and Chun Jin. Red Opal: product-feature scoring from reviews. in Proceedings of Twelfth ACM Conference on Electronic Commerce (EC-2007). 2007.

References


More references for aspect extraction based on frequent nouns:› Zhu et al.,(2009)[1]

Proposed a method based on the Cvalue measure from (Frantzi, Ananiadou and Mima, 2000)[2] for exracting multi-word aspects.

[1] Zhu, Jingbo, Huizhen Wang, Benjamin K. Tsou, and Muhua Zhu. Multiaspect opinion polling from textual reviews. in Proceedings of ACM International Conference on Information and Knowledge Management (CIKM-2009). 2009.

[2] Frantzi, Katerina, Sophia Ananiadou, and Hideki Mima. Automatic recognition of multi- word terms:. the C-value/NC-value method. International Journal on Digital Libraries, 2000. 3(2): p. 115-130.

References


More references for aspect extraction based on frequent nouns:› Long, Zhang and Zhu,(2010)[1]

Extracted aspects based on frequency and information distance.

Their method first finds the core aspect words using the frequency-based method.

It then uses the information distance in (Cilibrasi and Vitanyi, 2007) to find other related words to an aspect, e.g., for aspect price, it may find “$” and “dollars”.

[1] Long, Chong, Jie Zhang, and Xiaoyan Zhu. A review selection approach for accurate feature rating estimation. in Proceedings of Coling 2010: Poster Volume. 2010.

[2] Cilibrasi, Rudi L. and Paul M. B. Vitanyi. The google similarity distance. IEEE Transactions on Knowledge and Data Engineering, 2007. 19(3): p. 370-383.

References

Aspect Mining



and noun phrases.


target relations.



Aspect Extraction using Opinion Target Relations

Since opinions have targets, they are obviously related. Their relationships can be exploited to extract aspects which are opinion targets because sentiment words are often known.

This method was used in (Hu and Liu, 2004) for extracting infrequent aspects.

For example “The software is amazing.” if we know that “amazing” is a sentiment word, then “software” is extracted as an aspect.

Aspect Extraction using Opinion Target Relations

References for literature used this methid:› Zhuang, Jingm and Zhu, 2006[1]

› Somasundaran and Wiebe, 2009[2]

› Kobayashi et al., 2006[3]

In previous literature a dependency parser was used to identify such dependency relations for aspect extraction.

[1] Zhuang, Li, Feng Jing, and Xiaoyan Zhu. Movie review mining and summarization. in Proceedings of ACM International Conference on Information and Knowledge Management (CIKM-2006). 2006.[2] Somasundaran, S., J. Ruppenhofer, and J. Wiebe. Discourse level opinion relations: An annotation study. in Proceedings of the 9th SIGdial Workshop on Discourse and Dialogue. 2008.[3] Kobayashi, Nozomi, Ryu Iida, Kentaro Inui, and Yuji Matsumoto. Opinion mining on the Web by extracting subject-attribute-value relations. In Proceedings of AAAI-CAAW'06.2006.

References

Aspect Mining



and noun phrases.


target relations.



Aspect Extraction using Supervised Learning

Many algorithms based on supervised learning have been proposed in the past for information extraction (Hobbs and Riloff, 2010[1]; Mooney and Bunescu, 2005[2]; Sarawagi, 2008[3])

[1] Hobbs, Jerry R. and Ellen Riloff. Information Extraction, in in Handbook of Natural Language Processing, 2nd Edition, N. Indurkhya and F.J. Damerau, Editors. 2010, Chapman & Hall/CRC Press.[2] Mooney, Raymond J. and Razvan Bunescu. Mining knowledge from text using information extraction. ACM SIGKDD Explorations Newsletter, 2005. 7(1): p. 3-10.[3] Sarawagi, Sunita. Information extraction. Foundations and Trends in Databases, 2008. 1(3): p. 261-377..

References


The most dominant methods are based on sequential learning.

The current state of the art sequential learning methods are Hidden Markov Models (HMM) (Rabiner, 1989)[1] and Conditional Random Fields (CRF) (Lafferty, McCallum and Pereira, 2001)[2]

[1] Rabiner, Lawrence R. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 1989. 77(2): p. 257-286.

[2] Lafferty, John, Andrew McCallum, and Fernando Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. in Proceedings of International Conference on Machine Learning (ICML-2001). 2001.

References


The most dominant methods are based on sequential learning.

The current state of the art sequential learning methods are Hidden Markov Models (HMM) (Rabiner, 1989)[1] and Conditional Random Fields (CRF) (Lafferty, McCallum and Pereira, 2001)[2]

[1] Rabiner, Lawrence R. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 1989. 77(2): p. 257-286.

[2] Lafferty, John, Andrew McCallum, and Fernando Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. in Proceedings of International Conference on Machine Learning (ICML-2001). 2001.

References


Yu et al. (2012)[1] used a partially supervised learning method called one class SVM (Manevitz and Yousef, 2002)[2] to extract aspects.

In their case they only extracted aspects from Pos and Cons of review format 2 as in (Liu, Hu and Cheng, 2005)[3]

They also clustered those synonym aspects and ranked aspects based on their frequency and their contributions to the overall review rating of reviews.

[1] Yu, Jianxing, Zheng-Jun Zha, Meng Wang, and Tat-Seng Chua. Aspect ranking: identifying important product aspects from online consumer reviews. in Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics. 2011.[2] Manevitz, Larry M. and Malik Yousef. One-class SVMs for document classification. The Journal of Machine Learning Research, 2002. 2: p. 139- 154.[3] Liu, Bing, Minqing Hu, and Junsheng Cheng. Opinion observer: Analyzing and comparing opinions on the web. in Proceedings of International Conference on World Wide Web (WWW-2005). 2005.

References


Ghani et al. (2006)[1] used both traditional supervised learning and semi-supervised learning for aspect extraction.

Kovelamudi et al., (2011)[2] used a supervised method but also exploited some relevant information from Wikipedia.

[1] Ghani, Rayid, Katharina Probst, Yan Liu, Marko Krema, and Andrew Fano. Text mining for product attribute extraction. ACM SIGKDD Explorations Newsletter, 2006. 8(1): p. 41-48.

[2] Kovelamudi, Sudheer, Sethu Ramalingam, Arpit Sood, and Vasudeva Varma. Domain Independent Model for Product Attribute Extraction from User Reviews using Wikipedia. in Proceedings of the 5th International Joint Conference on Natural Language Processing (IJCNLP-2010). 2011.

References

Aspect Mining



and noun phrases.


target relations.



Aspect Extraction using Topic Models

Topic modeling is an unsupervised learning method that assumes each document consists of a mixture of topics and each topic is a probability distribution.

There were two main basic models, pLSA (Probabilistic Latent Semantic Analysis) (Hofmann, 1999)[1] and LDA (Latent Dirichlet allocation) (Blei, Ng and Jordan, 2003; Griffiths and Steyvers, 2003; Steyvers and Griffiths, 2007).

[1] Hofmann, Thomas. Probabilistic latent semantic indexing. in Proceedings of Conference on Uncertainty in Artificial Intelligence (UAI-1999). 1999.[2] Blei, David M., Andrew Y. Ng, and Michael I. Jordan. Latent dirichlet allocation. The Journal of Machine Learning Research, 2003. 3: p. 993- 1022.[3] Steyvers, Mark and Thomas L. Griffiths. Probabilistic topic models. Handbook of latent semantic analysis, 2007. 427(7): p. 424-440.

References


In the sentiment analysis context, one can design a joint model to model both sentiment words and topics at the same time, due to the observation that every opinion has a target.

For readers who are not familiar with topic models, a part from reading the topic modeling literature, the “pattern recognition and machine learning” book by Christopher M. Bishop.


Mei et al. (2007)[1] proposed an aspect sentiment mexture model, which was based on aspect (topic) model, positive and negative sentiment models learned with the help of external training data. And their model was based on pLSA.

Some researchers showed that global topic models are not suitable for detecting aspects as in (Titov and McDonald, 2008)[2].

[1] Mei, Qiaozhu, Xu Ling, Matthew Wondra, Hang Su, and ChengXiang Zhai. Topic sentiment mixture: modeling facets and opinions in weblogs. In Proceedings of International Conference on World Wide Web (WWW-2007). 2007.[2] Titov, Ivan and Ryan McDonald. Modeling online reviews with multi-grain topic models. in Proceedings of International Conference on World Wide Web (WWW-2008). 2008.

References


Later Brody and El Hadad (2010) [1] proposed to first identify aspects using topic models and then identify aspect-specific sentiment words by considering adjectives only.

In (Mukherjee and Liu, 2012), a semi-supervised joint model was proposed, which allows the user to provide some seed aspect terms for some topics in order to guide the inference to produce aspect distributions that conform to the user’s need.

[1] Brody, Samuel and Noemie Elhadad. An Unsupervised Aspect-Sentiment Model for Online Reviews. in Proceedings of The 2010 Annual Conference of the North American Chapter of the ACL. 2010.[2] Mukherjee, Arjun and Bing Liu. Aspect Extraction through Semi- Supervised Modeling. in roceedings of 50th Anunal Meeting of Association for Computational Linguistics (ACL- 2012) (Accepted for publication). 2012.

References

Aspect Mining

Some other used techniques for aspect extraction:

› Meng and Wang (2009)[1] extracted

aspects from product specifications,

which are structured data.

[1] Meng, Xinfan and Houfeng Wang. Mining user reviews: from specification to summarization. in Proceedings of the ACL-IJCNLP 2009 Conference Short Papers. 2009.

References

Next Step

Identify which of those methods is better and more reliable.

Study the applicability of each of these methods for Arabic Language based on the language dependent factor of each.

End Of Presentation

Thank you

Education

Aspect extraction (A survey)