View
376
Download
0
Category
Preview:
Citation preview
Outline MD Classification MDSSL Application to SA Conclusions Future Work
DEA - Diploma of Advanced StudiesSemi-supervised Learning of Multi-dimensional Class Bayesian
Network Classifiers: Application to Sentiment Analysis
Jonathan Ortigosa-Hernandez
advised by
Jose A. Lozano and Inaki Inza
Intelligent Systems GroupComputer Science and Artificial Intelligence Department
University of the Basque Country
November 4th, 2010
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Curriculum Vitae
Undergraduate Education2003-2008: Masters of Science Degree in ComputerEngineering, University of the Basque Country.2007-2008: Bachelor of Science Degree in Informatics,Coventry University.
Postgraduate Education2008-Present: PhD Student, ISG Group, University of theBasque Country (Four-Year MEC-FPU Grant).Doctorate Program: Probabilistic Graphical Models forArtificial Intelligence and Data Mining.Doctorate Lectures:
Fundamentals of Probabilistic Graphical ModelsInference in PGMsLearning PGMsBioinformatic Applications of PGMsScientific Research MethodologyStatistical and Computational Basis for PGMs
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Research Interest
MethodologyMulti-dimensional ClassificationMulti-dimensional Class Bayesian Network ClassifiersSemi-supervised Learning
ApplicationsOpinion Mining and Sentiment AnalysisAffect Analysis
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
1 Multi-dimensional Supervised Classification
2 Multi-dimensional Semi-supervised Learning
3 Application to Sentiment Analysis
4 Conclusions
5 Current and Future Work
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Supervised Classification
It consists of building a classifier Ψ from a given labelledtraining dataset D, by using an induction algorithm A(A(D) = Ψ),
X1 X2 ... Xn C
x(1)1 x
(1)2 ... x
(1)n c (1)
x(2)1 x
(2)2 ... x
(2)n c (2)
... ... ... ... ...
x(N)1 x
(N)2 ... x
(N)n c (N)
in order to predict the value of a class variable C for any newunlabelled instance x (Ψ(x) = c).
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Uni-dimensional and Multi-dimensional Classification
Uni-dimensional classification tries to predict a single classvariable based on a dataset composed of a set of labelledexamples.
(Uni-dimensional Class) Bayesian Network Classifiers(Larranaga et al, 2005).
Multi-dimensional classification is the generalisation of thesingle-class classification task to the simultaneous predictionof a set of class variables.
Multi-dimensional Class Bayesian Network Classifiers (v.d.Gaag and d. Waal, 2006).Do not confuse with multi-class and multi-label classification.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Multi-dimensional Supervised Learning
A typical supervised training dataset
X1 X2 ... Xn C1 C2 ... Cm
x(1)1 x
(1)2 ... x
(1)n c
(1)1 c
(1)2 ... c
(1)m
x(2)1 x
(2)2 ... x
(2)n c
(2)1 c
(2)2 ... c
(2)m
... ... ... ... ... ... ... ...
x(N)1 x
(N)2 ... x
(N)n c
(N)1 c
(N)2 ... c
(N)m
Each instance of the dataset contains both the values of theattributes and m labels which characterise the attributes.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Bayesian Network Classifiers
X1 X2 X3 X4 X5 X6
C
Figure: A (uni-dimensional) naive Bayes structure.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Multi-dimensional Class Bayesian Network Classifiers(MDBNC)
X1 X2 X3 X4 X5 X6
C1 C2 C3
Figure: A multi-dimensional naive Bayes structure.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
MDBNC Structure
X1 X2 X3 X4 X5
C1 C2 C3
X1 X2 X3 X4 X5
C1 C2 C3
X1 X2 X3 X4 X5
(a) Complete graph
(c) Class subgraph
(b) Feature selection subgraph
(d) Feature subgraph
C1 C2 C3
Figure: A MDNBC structure and its division.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Sub-families of MDBNC
(a) Multi-dimensional naive Bayes
(c) Multi-dimensional J/K dependence Bayesian (2/3)
(b) Multi-dimensional tree-augmented network
!"
!#
!$
!%
!& !'
!(
!)
*# *$ *%
!+ !",
*"
!" !# !$ !% !& !' !( !) !* !"+
," ,# ,$
!" !# !$ !% !&
'" '#
Figure: Different subfamilies of MDBNC.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Sub-families of MDBNC - MDnB
Multi-dimensional naive Bayes (MDnB)
!" !# !$ !% !& !' !( !) !* !"+
," ,# ,$The class and featuresubgraphs are empty.
Each class variable isparent of all thefeatures.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Sub-families of MDBNC - MDnB
Multi-dimensional naive Bayes (MDnB)
!" !# !$ !% !& !' !( !) !* !"+
," ,# ,$ It has a fixed structure.
Thus, it has nostructural learning (v.d.Gaag and d. Waal,2006).
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Sub-families of MDBNC - MDnB
Multi-dimensional tree-augmented network classifier (MDTAN)
!" !# !$ !% !&
'" '#
The class and featuresubgraphs are trees.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Sub-families of MDBNC - MDnB
Multi-dimensional tree-augmented network classifier (MDTAN)
!" !# !$ !% !&
'" '#
A wrapper structurallearning algorithm isproposed in (v.d. Gaagand d. Waal, 2006).
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Sub-families of MDBNC - MDnB
Multi-dimensional J/K dependence Bayesian classifier (MD J/K )
!"
!#
!$
!%
!& !'
!(
!)
*# *$ *%
!+ !",
*"
The class subgraph is aJ-dependence graph.
The feature subgraph isa K -dependence graph.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Sub-families of MDBNC - MDnB
Multi-dimensional J/K dependence Bayesian classifier (MD J/K )
!"
!#
!$
!%
!& !'
!(
!)
*# *$ *%
!+ !",
*"
There was not a specificstructural learningalgorithm.
So, we proposed alearning algorithm in(Ortigosa-Hernandez etal, 2010).
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 0 - Initialisation
Establish the maximumnumber of parents inboth class and featuresubgraphs, i.e. J = 2and K = 2.
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 1 - Learn the structure between the class variables (Ac )
Calculate the mutualinformation MI (Ci ,Cj )for each pair of classvariables.
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 1 - Learn the structure between the class variables (Ac )
Calculate the p-values(significance of eachmutual information)using independence test.
C1 C2 C3
C4 0.36 0.57 0.01C3 0.27 0.63C2 0.06
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 1 - Learn the structure between the class variables (Ac )
Remove the p-valuesgreater than thethreshold α = 0.1.
C1 C2 C3
C4 0.36 0.57 0.01C3 0.27 0.63C2 0.06
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 1 - Learn the structure between the class variables (Ac )
From the lowest value,start adding arcs to thegraph fulfilling theconditions of no cyclesand no more thanJ-parents per classvariable.
C1 C2 C3
C4 x x 0.01C3 x xC2 0.06
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 1 - Learn the structure between the class variables (Ac )
From the lowest value,start adding arcs to thegraph fulfilling theconditions of no cyclesand no more thanJ-parents per classvariable.
C1 C2 C3
C4 x x xC3 x xC2 0.06
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 2 - Learn the structure between the class variables and thefeatures (ACF )
Calculate the mutualinformation MI (Ci ,Xj )for each pair Ci and Xj . X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 2 - Learn the structure between the class variables and thefeatures (ACF )
Calculate the p-value ofeach mutualinformation.
C1 C2 C3 C4
X1 0.64 0.00 0.77 0.98X2 0.82 0.03 0.11 0.37X3 0.00 0.06 0.00 0.01X4 0.68 0.09 0.00 0.55X5 0.81 0.12 0.81 0.65X6 0.57 0.24 0.00 0.00X7 0.25 0.26 0.00 0.00X8 0.32 0.15 0.00 0.44
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 2 - Learn the structure between the class variables and thefeatures (ACF )
Remove the p-valuesgreater than thethreshold α = 0.1.
C1 C2 C3 C4
X1 0.64 0.00 0.77 0.98X2 0.82 0.03 0.11 0.37X3 0.00 0.06 0.00 0.01X4 0.68 0.09 0.00 0.55X5 0.81 0.12 0.81 0.65X6 0.57 0.24 0.00 0.00X7 0.25 0.26 0.00 0.00X8 0.32 0.15 0.00 0.44
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 2 - Learn the structure between the class variables and thefeatures (ACF )
Add all the arcs to thestructure.
C1 C2 C3 C4
X1 x 0.00 x xX2 x 0.03 x xX3 0.00 0.06 0.00 0.01X4 x 0.09 0.00 xX5 x x x xX6 x x 0.00 0.00X7 x x 0.00 0.00X8 x x 0.00 x
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 3 - Learn the structure between the features(AF )
Calculate theconditional mutualinformationMI (Xi ,Xj ||Pac(Xj )).
Calculate thep-values.
Remove thep-values greaterthan the thresholdα = 0.1.
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
A supervised method to learn a MD J/K structure(Ortigosa-Hernandez et al, 2010)
Step 3 - Learn the structure between the features(AF )
Add arcs between thefeatures fulfilling theconditions of no cyclesbetween the featuresand no more thanK -parents per feature.
X1 X2 X3 X4 X5 X6 X7 X8
C1 C2 C3 C4
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Major Problem of Supervised Learning
However, in many real world problems, obtaining data isrelatively easy, while labelling is difficult, expensive or laborintensive (usually done by an external mechanism, e.g. humanbeings).
This problem is accentuated when using multiple targetvariables.
DESIRE: Learning algorithms able to incorporate a largenumber of unlabelled data with a small number of labeleddata when learning competitive classifiers.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Multi-dimensional Semi-supervised Learning
A typical semi-supervised training dataset
X1 X2 ... Xn C1 C2 ... Cm
x(1)1 x
(1)2 ... x
(1)n c
(1)1 c
(1)2 ... c
(1)m
x(2)1 x
(2)2 ... x
(2)n c
(2)1 c
(2)2 ... c
(2)m
... ... ... ... ... ... ... ...
x(L)1 x
(L)2 ... x
(L)n c
(L)1 c
(L)2 ... c
(L)m
x(L+1)1 x
(L+1)2 ... x
(L+1)n ? ? ... ?
x(L+2)1 x
(L+2)2 ... x
(L+2)n ? ? ... ?
... ... ... ... ... ... ... ...
x(N)1 x
(N)2 ... x
(N)n ? ? ... ?
Semi-supervised Learning fulfils this desire.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
The Expectation-Maximisation Algorithm
The EM algorithm (Dempster et al, 1977)
Learn an initial model.Repeat until convergence:(a) Expectation step: Using the current model, estimate themissing values of the data.(b) Maximisation step: Using the whole data and the previousestimations, learn a new current model.
Any MDBNC learning algorithm can be used as model in thisalgorithm.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Artificial Experimentation
20 different mult-dimensional datasets are sampled.
Training subset: 10, 000 instances (L:100/U:10, 000)Test subset: 5, 000
Feat. Class V.
Num. 5 to 20 2 to 4Card. 2 to 4 2 to 3
Learning algorithms:
4 uni-dimensional (nB, TAN, 2-DB and 3-DB)4 multi-dimensional (MDnB, MDTAN, MD 2/2 and MD 2/3)
Scenario: Supervised / Semi-supervised
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Artificial Experimentation
Numerical results can be found in:
http://www.sc.ehu.es/ccwbayes/members/jonathan/home/News_and_Notables/Entries/2010/11/30_
Artificial_Experiments_2010.html
(Semi-supervised) Multi-dimensional algorithms
V(Supervised) Multi-dimensional algorithms
VUni-dimensional algorithms
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Application to Sentiment Analysis
Sentiment Analysis (AKA Opinion Mining) is thecomputational study of opinions, sentiments and emotionsexpressed in text (Liu, 2010).
When treating Sentiment Analysis as a classification problem,several different (but related) problems appear. For example:
1 Subjectivity Classification. Its aim is to classify a text assubjective or objective.
2 Sentiment Classification. It classifies an opinionated text asexpressing a positive, neutral, or negative opinion.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Motivation for Using Semi-supervised Learning ofMulti-dimensional Classifiers
1 Up to now, these subproblems have been studied in isolationdespite of being closely related. So, probably it would behelpful to use multi-dimensional classifiers.
2 Obtaining enough labeled examples for a classifier may becostly and time consuming. This motivates us to deal withunlabelled examples in a semi-supervised framework.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Hypothesis Formulated
First Hypothesis
The explicit use of the relationships between different classvariables can be beneficial to improve their recognition rates.
Second Hypothesis
Multi-dimensional techniques can work with unlabelled data inorder to improve the classification rates in Sentiment Analysis.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Properties of the Dataset
Collected by Socialware Company S.A., from the ASOMOservice of mobilised opinion analysis.It consists of 2, 542 Spanish reviews extracted from a blog:
150 documents have been labeled in isolation by an expert.2, 392 posts are left unlabelled.
Figure: The ASOMO corpus.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Properties of Each Document
Each document is represented as:
14 features
Obtained by using an open source morphological analyser(Carreras et al, 2006).Each feature provide different information related topart-of-speech (POS).Eg. First Persons, Agreement Expressions, Imperatives,Prediction Verbs (future), Questions, Positive Adjectives, etc.Represented as a real number between 0 and 1.
3 class variables
Will to Influence: {declarative sentence, soft WI, medium WI,strong WI}Sentiment: {very negative, negative, neutral, positive, verypositive}Subjectivity: {Yes (subjective), No (objective)}
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Experiment 1 - Set Up I
First Hypothesis
The explicit use of the relationships between different classvariables can be beneficial to improve their recognition rates.
The ASOMO corpus has been used to learn:
3 (uni-dimensional) naive Bayes classifiers, one per each classvariable.A (uni-dimensional) naive Bayes classifier with a compoundclass variable.A multi-dimensional naive Bayes classifier.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Experiment 1 - Set Up II
First Hypothesis
The explicit use of the relationships between different classvariables can be beneficial to improve their recognition rates.
Features from ASOMO dataset are discretised into 3 valuesusing equal frequency.
In addition to the ASOMO feature set, two state-of-the-artfeature sets are used:
UnigramsUnigrams + Bigrams
Results averaged over 5 × 5 fold cross validation.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Experiment 1- JOINT Accuracies
Figure: JOINT accuracies on ASOMO corpus using three differentfeature sets in both uni and multi-dimensional scenarios (5× 5cv)
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Experiment 1 - Computation Time
Figure: Computational times of the learning algorithms using differentfeature sets.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Experiment 2 - Set Up
Second Hypothesis
Multi-dimensional techniques can work with unlabelled data inorder to improve the classification rates in Sentiment Analysis.
The ASOMO dataset has been used to learn:
3 (uni-dimensional) Bayesian network classifiers: nB, TAN and2DB.5 MDBNC: MDnB, MDTAN, MD 2/2, MD 2/3 and MD 2/4.
In both Supervised and Semi-supervised (EM algorithm)learning frameworks.
Features from ASOMO dataset are discretised into 3 valuesusing equal frequency.
Results averaged over 5 × 5 fold cross validation.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Experiment 2 - JOINT Accuracy
5
10
15
20
25
nB TAN 2DB MDnB MDTAN MD 2/2 MD 2/3 MD 2/4
Figure: JOINT accuracies on ASOMO dataset in the supervised andsemi-supervised learning frameworks.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Experiment 2 - Will to Influence
30
40
50
60
70
nB TAN 2DB MDnB MDTAN MD 2/2 MD 2/3 MD 2/4
Figure: Accuracies for the Will to Influence class variable on ASOMOdataset in the supervised and semi-supervised learning frameworks.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Experiment 2 - Sentiment Polarity
20
25
30
35
40
nB TAN 2DB MDnB MDTAN MD 2/2 MD 2/3 MD 2/4
Figure: Accuracies for the Sentiment Polarity class variable on ASOMOdataset in the supervised and semi-supervised learning frameworks.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Experiment 2 - Subjectivity
50
60
70
80
90
nB TAN 2DB MDnB MDTAN MD 2/2 MD 2/3 MD 2/4
Figure: Accuracies for the Subjectivity class variable on ASOMO datasetin the supervised and semi-supervised learning frameworks.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Conclusions I - Methodology
Multi-dimensional classification and semi-supervised learningare two different branches of machine learning.
With this research, we have established a bridge betweenthem showing that:
Uni-dimensional approaches cannot capture the real nature ofmulti-dimensional problems.More accurate classifiers can be found using themulti-dimensional learning approaches.The use of large amounts of unlabelled data can be beneficialto improve recognition rates.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Conclusions II - Application
With respect to the Sentiment Analysis application, we haveproposed a novel perspective to solve the problem.Experimental results demonstrate that the use ofmulti-dimensional classification, as well as the use ofunlabelled data, can lead us to more accurate classifiers.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Conclusions III - Publications
Publication
J. Ortigosa-Hernandez, J.D. Rodriguez, L. Alzate, I. Inza, J.A.Lozano. (2010). A Semi-supervised Approach to Multi-dimensionalClassification with Application to Sentiment Analysis. CEDI 2010,V Simposio de Teoria y Aplicaciones de Mineria de Datos(TAMIDA2010), Valencia, Spain.
We are writing the final draft of a paper for the Special Issue onData Mining Applications and Case Studies at Neurocomputing.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Short Term Future Work
Title: Application to Affect AnalysisDescription: Use the methodology proposed in this presentation to dealwith the problem of Affect Analysis.Collaboration: Socialware S.A.
Motivation (Abbasi et al., 2008)
Affect Analysis is concerned with the analysis of text containing emotionsand it tries to extract a large number of potential emotions, e.g.happiness, sadness, anger, hate, violence, excitement, etc.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Short Term Future Work
We want to take a step forward in this problem taking advantageof the potential possibilities of the MDBNC to model complexrelationships between the class variables.
(a) Plutchik’s affect model (b) Chromatic affect model
Figure: Psychological Human Affection models considered for this project.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Long Term Future Work
Wrapper Structural Search Algorithm.
Adapt other semi-supervised learning approaches to themulti-dimensional classification domain, e.g. Co-training,Active Learning, ...
The scalability of MDBNC is a problem that has to be studied(high dimensionality of multi-label problems).
“As discussed in the literature, currently there is no coherentstrategy for handling unlabelled data, so some creativity must beexercised.” (Cohen’s thesis, 2003)
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
Questions
THANK YOUjonathan.ortigosa@ehu.es
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Outline MD Classification MDSSL Application to SA Conclusions Future Work
References
Abbasi, A. and Chen, H. and Thoms, S. and Fu, T. (2008). Affect Analysis of Web Forums and Blogs usingCorrelation Ensembles. IEEE Transactions on Knowledge and Data Engineering, Vol. 20(9), pp. 1168–1180.
Carreras X., Chao I., Padro L. and Padro M. (2006). An Open-Source Suite of Language Analyzers. In Proceedingsof the 4th Int. Conference on Language Resources and Evaluation, Vol. 10, pp. 239–342.
Cozman, F. and Cohen, I. (2006). Risk of Semi-Supervised Learning. In: Chapelle, O. Scholkopf, B. and Zien, A.Semi-Supervised Learning. The MIT Press. pp 57-72.
Dempster A., Laird N. and Rubin D. (1977). Maximum Likelihood from Incomplete Data via the EM Algorithm.Journal of the Royal Statistical Society. Series B, 39(1): 1–38.
Friedman, N. (1998.) The Bayesian Structural EM algorithm. In Proc. 14th Conf. on Uncertainty in ArtificialIntelligence. Morgan Kaufmann, San Francisco, CA, pp. 129–138.
van der Gaag L. and d. Waal P. (2006). Multi-dimensional Bayesian Classifiers. In Proceedings of the ThirdEuropean Workshop in Probabilistic Graphical Models, pages 107–114.
Larranaga, P., Lozano, J.A., Pena, J.M. and Inza, I. (2005). Special Issue on Probabilistic Graphical Models forClassification. Machine Learning, 59(3).
Liu B. (2010). Sentiment Analysis and Subjectivity. In: Indurkhya N. and Damerau F.J. Handbook of NaturalLanguage Processing, Chapman & Hall, 2nd Ed.
Ortigosa-Hernandez J., Rodriguez J.D., Alzate L., Inza I. and Lozano J.A. (2010). A Semi-supervised Approach toMulti-dimensional Classification with Application to Sentiment Analysis. In Proc. of the V Simposio de Teoria yAplicaciones de Mineria de Datos (TAMIDA2010), CEDI 2010, Valencia, Spain.
Rodriguez, J.D. and Lozano, J.A. (2008). Multi-objective learning of multi-dimensional Bayesian classifiers. InProceedings of the Eighth International Conference on Hybrid Intelligent Systems, HIS 2008, Barcelona, Spain. pp.501-506.
Jonathan Ortigosa-Hernandez DEA - Diploma of Advanced Studies
Recommended