Upload
others
View
13
Download
0
Embed Size (px)
Citation preview
Convolutional Neural Networks for Sentiment Analysis in Persian
Social Media
Morteza Rohanian, MSc1, Mostafa Salehi, PhD2, Ali Darzi, PhD3, Vahid Ranjbar, PhD
1- Faculty of New Sciences and Technologies, University of Tehran, Tehran, Iran, Email: [email protected]
2- Faculty of New Sciences and Technologies, University of Tehran, Tehran, Iran, Email: [email protected]
3- Faculty of Literature and Humanities, University of Tehran, Tehran, Iran, Email: [email protected]
1- Department of Computer Enigneering, Yazd University, Yazd, Iran, Email: [email protected]
Abstract: With the social media engagement on the rise, the resulting data can be used as a rich resource for analyzing and understanding
different phenomena around us. A sentiment analysis system employs these data to find the attitude of social media users towards certain
entities in a given document. In this paper we propose a sentiment analysis method for Persian text using Convolutional Neural Network
(CNN), a feedforward Artificial Neural Network, that categorize sentences into two and five classes (considering their intensity) by
applying a layer of convolution over input data through different filters. We evaluated the method on three different datasets of Persian
social media texts using Area under Curve metric. The final results show the advantage of using CNN over earlier attempts at developing
traditional machine learning methods for Persian texts sentiment classification especially for short texts.
Keywords: Sentiment Analysis, Social Media, Convolutional Neural Network, Sentiment Intensity , Short Texts
سنده مسئول:ينام نو
سنده مسئول: ينو ينشان
-
:
3
4
5)CNN(6
7
-
-
-
8
9
1011-
12SVM
SVMيبنددسته
استخراج يهايژگيمختلف با استفاده از و يهاا در کالسيکردن اش
شده است
-
13RNN
RNN
RNN
14
15
[28 ]
SVM
-
LSTM
16
17
-
(CNN)
18
19
20
21
𝑥 ∈ 𝑅𝑠
𝑠𝑑 ∈
𝑅𝑛×𝑠𝑛
𝑊 ∈ 𝑅ℎ×𝑠ℎ
*
𝑊 ∗ 𝑑𝑗:𝑗+ℎ−1 = ∑ ∑ 𝑊𝑖,𝑘𝑑𝑖,𝑘𝑠−1𝑘=0
𝑗+ℎ−1𝑖=𝑗 .
ℎ𝑑𝑗:𝑗+ℎ−1
𝑓𝑐𝑗
𝑐𝑗 = 𝑓(𝑑𝑗:𝑗+ℎ−1 + 𝑏),
𝑏22
𝑊
𝑐(𝑊) = [𝑐1, 𝑐2, … , 𝑐𝑛−ℎ+1].
23
24
25
�̂�𝑊 = max 𝑐(𝑊)𝑖.
26
�̂�𝑊 = [�̂�𝑊1 , … , �̂�𝑊𝑘]𝑇,
𝑇 = {1, … , 𝑘}
يه الحاق، ما عمل الحاق حداکثريدر ال
ه کانولوشن و به يمختلف حاصل از ال يهايژگيب ويترک يرا برا
دن به يرس يم. برايدهيک بردار با بعد ثابت انجام ميآوردن وجود
يارهيوند زنجيه کانولوشن را پيال يهايصاف يبردار جمله خروج
يهاتعداد نگاشت ،شتر باشديها بيم. هرچه تعداد صافيدهيم
ن مقدار در يشتريب ،يژگيشود و از هر نگاشت ويشتر ميب يژگيو
يسراسر يژگيبردار و شود و بهيانتخاب م يمرحله الحاق حداکثر
توان گفت کهيپس م شود.ياضافه م
،
𝑈 ∈ 𝑅2×𝑘𝑏𝑈 ∈ 𝑅2
𝑦𝑗 = 𝑈𝑗 �̂�𝑊 + 𝑏𝑗𝑈
𝑐𝑤𝑈𝑗𝑗 𝑈𝑏𝑗𝑈𝑗𝑏𝑈𝑌𝑗
𝑗𝑑
𝑃(𝑌𝑗 = 1|𝑑, 𝑊, 𝑏𝑈) =𝑒
𝑦𝑗
∑ 𝑒𝑦𝑖𝑖,
𝐷
− ∑ log( 𝑃(𝑌𝑃𝑂𝑆𝑑 − 1|𝑑, 𝑊, 𝑏𝑈))𝑑∈𝐷
− ∑ log( 𝑃(𝑌𝑃𝑂𝑆𝑑 − 2|𝑑, 𝑊, 𝑏𝑈))𝑑∈𝐷
𝑑, 𝑊, 𝑏𝑈
CNN
27
𝑠
-
-
مجموعه داده -4-1
@username
http://t.co/urlURLs
2007
#tweet
/\
يسازادهيپ -4-2
GPU
KerasTheano][
scikit-learn]25[
معيار ارزيابي -4-3
Y
X
31AUC
CNN
RNN
يآنتروپ
نهيشيب
SVM
-
-
-
-
-
[1] Greenwood, S., A. Perrin, and M. Duggan. "Social
media update 2016: Facebook usage and engagement
is on the rise, while adoption of other platforms holds
steady." Pew Research Center (2016).
[2] Mander, Jason. "Daily time spent on social networks
rises to 1.72 hours." London: Global Web Index (2015).
[3] Liu, Bing. "Sentiment analysis and opinion mining."
Synthesis lectures on human language technologies 5,
no. 1 (2012): 1-167.
[4] Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan.
"Thumbs up?: sentiment classification using machine
learning techniques." In Proceedings of the ACL-02
conference on Empirical methods in natural language
processing-Volume 10, pp. 79-86. Association for
Computational Linguistics, 2002.
[5] Tripathy, Abinash, Ankit Agrawal, and Santanu Kumar
Rath. "Classification of sentiment reviews using n-
gram machine learning approach." Expert Systems with
Applications 57 (2016): 117-126.
[6] Mullen, Tony, and Nigel Collier. "Sentiment Analysis
using Support Vector Machines with Diverse
Information Sources." In EMNLP, vol. 4, pp. 412-418.
2004.
[7] Agarwal, Basant, and Namita Mittal. "Machine
learning approach for sentiment analysis." In
Prominent Feature Extraction for Sentiment Analysis,
pp. 21-45. Springer International Publishing, 2016.
[8] Poria, Soujanya, Haiyun Peng, Amir Hussain, Newton
Howard, and Erik Cambria. "Ensemble application of
convolutional neural networks and multiple kernel
learning for multimodal sentiment analysis."
Neurocomputing (2017).
[9] Dong, Li, Furu Wei, Chuanqi Tan, Duyu Tang, Ming
Zhou, and Ke Xu. "Adaptive Recursive Neural
Network for Target-dependent Twitter Sentiment
Classification." In ACL (2), pp. 49-54. 2014.
[10] Kim, Yoon. "Convolutional neural networks for
sentence classification." arXiv preprint
arXiv:1408.5882 (2014).
[11] Zhang, Ye, and Byron Wallace. "A sensitivity analysis
of (and practitioners' guide to) convolutional neural
networks for sentence classification." arXiv preprint
arXiv:1510.03820 (2015).
[12] Jaynes, Edwin T. "Information theory and statistical
mechanics." Physical review 106, no. 4 (1957): 620.
[13] Neethu, M. S., and R. Rajasree. "Sentiment analysis in
twitter using machine learning techniques." In
Computing, Communications and Networking
Technologies (ICCCNT), 2013 Fourth International
Conference on, pp. 1-5. IEEE, 2013.
[14] Cortes, Corinna, and Vladimir Vapnik. "Support-
vector networks." Machine learning 20, no. 3 (1995):
273-297.
[15] Dos Santos, Cícero Nogueira, and Maira Gatti. "Deep
Convolutional Neural Networks for Sentiment
Analysis of Short Texts." In COLING, pp. 69-78. 2014.
[16] Zhang, Yu, Mengdong Chen, Lianzhong Liu, and
Yadong Wang. "An effective convolutional neural
network model for Chinese sentiment analysis." In AIP
Conference Proceedings, vol. 1836, no. 1, p. 020085.
AIP Publishing, 2017.
[17] Cieliebak, Mark, Jan Deriu, Dominic Egger, and Fatih
Uzdilli. "A Twitter Corpus and Benchmark Resources
for German Sentiment Analysis." SocialNLP 2017
(2017): 45.
[18] Socher, Richard, Alex Perelygin, Jean Y. Wu, Jason
Chuang, Christopher D. Manning, Andrew Y. Ng, and
Christopher Potts. "Recursive deep models for
semantic compositionality over a sentiment treebank."
In Proceedings of the conference on empirical methods
in natural language processing (EMNLP), vol. 1631,
p. 1642. 2013.
[19] Collobert, Ronan, Jason Weston, Léon Bottou,
Michael Karlen, Koray Kavukcuoglu, and Pavel
Kuksa. "Natural language processing (almost) from
scratch." Journal of Machine Learning Research 12,
no. Aug (2011): 2493-2537.
[20] Wang, Keze, Xiaolong Wang, Liang Lin, Meng Wang,
and Wangmeng Zuo. "3D human activity recognition
with reconfigurable convolutional neural networks." In
Proceedings of the 22nd ACM international
conference on Multimedia, pp. 97-106. ACM, 2014.
[21] Mikolov, Tomas, Ilya Sutskever, Kai Chen, Greg S.
Corrado, and Jeff Dean. "Distributed representations of
words and phrases and their compositionality." In
Advances in neural information processing systems,
pp. 3111-3119. 2013.
[22] Bengio, Yoshua, Réjean Ducharme, Pascal Vincent,
and Christian Jauvin. "A neural probabilistic language
model." Journal of machine learning research 3, no.
Feb (2003): 1137-1155.
[23] Bojanowski, Piotr, Edouard Grave, Armand Joulin,
and Tomas Mikolov. "Enriching word vectors with
subword information." arXiv preprint
arXiv:1607.04606 (2016).
[24] Bagheri, Ayoub, and Mohamad Saraee. "Persian
Sentiment Analyzer: A Framework based on a Novel
Feature Selection Method." International Journal of
Artificial Intelligence™ 12, no. 2 (2014): 115-129.
[25] Pedregosa, Fabian, Gaël Varoquaux, Alexandre
Gramfort, Vincent Michel, Bertrand Thirion, Olivier
Grisel, Mathieu Blondel et al. "Scikit-learn: Machine
learning in Python." Journal of Machine Learning
Research 12, no. Oct (2011): 2825-2830.
[26] Bergstra, James, Olivier Breuleux, Frédéric Bastien,
Pascal Lamblin, Razvan Pascanu, Guillaume
Desjardins, Joseph Turian, David Warde-Farley, and
Yoshua Bengio. "Theano: A CPU and GPU math
compiler in Python." In Proc. 9th Python in Science
Conf, pp. 1-7. 2010.
[27] Hajmohammadi, Mohammad Sadegh, and Roliana
Ibrahim. "A svm-based method for sentiment analysis
in persian language." In International Conference on
Graphic and Image Processing (ICGIP 2012), vol.
8768, p. 876838. International Society for Optics and
Photonics, 2013.
[28] Basiri, Mohammad Ehsan, Ahmad Reza Naghsh-
Nilchi, and Nasser Ghassem-Aghaee. "A framework
for sentiment analysis in persian." Open Transactions
on Information Processing 1, no. 3, pp 1-14 ,2014.
[29] Roshanfekr, Behnam, Shahram Khadivi, and
Mohammad Rahmati. "Sentiment analysis using deep
learning on Persian texts." In 2017 Iranian Conference
on Electrical Engineering (ICEE), pp. 1503-1508.
IEEE, 2017.
[]
.
[]
[]
1 Subjective Opinions
2 Objective Opinions 3 Supervised Learning 4 Generic
5 Convolutional Neural Networks 6 Feedforward 7 Word Embedding 8 Naïve Bayes classifier 9 Maximum Entropy Classifier (MEC) 1 0 Exponential Model 1 1 Principle of Maximum Entropy 1 2 Support Vector Machines (SVMs) 1 3 Recurrent Neural Networks 1 4 Semantic Role Labeling 1 5 Chunking 1 6 Representation Learning
[]
RTDGPS
RNNPSO
اهسيرنويز
1 7 Unsupervised Learning 1 8 Grid 1 9 Filters 2 0 Feature Maps 2 1 Max-pooling 2 2 Bias 2 3 Channel 2 4 Sentence Vector 2 5 Softmax 2 6 Global Feature Vector
2 7 Distributed Representation of Words 2 8 Validation Loss 2 9 Dropout 3 0 Adadelta 3 1 Area Under Curve