Upload
phoebe-jones
View
223
Download
1
Tags:
Embed Size (px)
Citation preview
Support Vector Machine Data Mining
Olvi L. Mangasarian
with
Glenn M. Fung, Jude W. Shavlik
& Collaborators at ExonHit – Paris
Data Mining Institute
University of Wisconsin - Madison
What is a Support Vector Machine?
An optimally defined surface Linear or nonlinear in the input space Linear in a higher dimensional feature space Implicitly defined by a kernel function K(A,B) C
What are Support Vector Machines Used For?
Classification Regression & Data Fitting Supervised & Unsupervised Learning
Principal TopicsKnowledge-based classification
Incorporate expert knowledge into a classifierBreast cancer prognosis & chemotherapy
Classify patients on basis of distinct survival curves Isolate a class of patients that may benefit from chemotherapy
Multiple Myeloma detection via gene expression measurementsDrug discovery based on gene macroarray expression
Joint work with ExonHit
Support Vector MachinesMaximize the Margin between Bounding Planes
x0w= í +1
x0w= í à 1
A+
A-
jjwjj2
w
Principal Topics
Knowledge-based classification (NIPS*2002)
Conventional Data-Based SVM
Knowledge-Based SVM via Polyhedral Knowledge Sets
Incoporating Knowledge Sets Into an SVM Classifier
This implication is equivalent to a set of constraints that can be imposed on the classification problem.
Suppose that the knowledge set: belongs to the class A+. Hence it must lie in the halfspace :
èx??Bx 6 b
é
èxjx0w>í +1
é
Bx6b ) x0w>í +1
We therefore have the implication:
Numerical TestingThe Promoter Recognition Dataset
Promoter: Short DNA sequence that precedes a gene sequence.
A promoter consists of 57 consecutive DNA nucleotides belonging to {A,G,C,T} .
Important to distinguish between promoters and nonpromoters
This distinction identifies starting locations of genes in long uncharacterized DNA sequences.
The Promoter Recognition DatasetNumerical Representation
Simple “1 of N” mapping scheme for converting nominal attributes into a real valued representation:
Not most economical representation, but commonly used.
The Promoter Recognition DatasetNumerical Representation
Feature space mapped from 57-dimensional nominal space to a real valued 57 x 4=228 dimensional space.
57 nominal values
57 x 4 =228binary values
Promoter Recognition Dataset Prior Knowledge Rules
Prior knowledge consist of the following 64 rules:
R1orR2orR3orR4
2
66666664
3
77777775
V
R5orR6orR7orR8
2
66666664
3
77777775
V
R9orR10orR11orR12
2
66666664
3
77777775
=) PROMOTER
Promoter Recognition Dataset Sample Rules
R8 : (pà 12= T) ^(pà 11= A) ^(pà 07= T);
R4 : (pà 36= T) ^(pà 35= T) ^(pà 34=G)^(pà 33= A) ^(pà 32= C);
R10 : (pà 45= A) ^(pà 44= A) ^(pà 41= A);where denotes position of a nucleotide, with respect to a meaningful reference point starting at position and ending at positionpà 50
pj
p7:Then:
R4^R8^R10=) PROMOTER
The Promoter Recognition DatasetComparative Algorithms
KBANN Knowledge-based artificial neural network [Shavlik et al] BP: Standard back propagation for neural networks [Rumelhart et al] O’Neill’s Method Empirical method suggested by biologist O’Neill [O’Neill] NN: Nearest neighbor with k=3 [Cost et al] ID3: Quinlan’s decision tree builder[Quinlan] SVM1: Standard 1-norm SVM [Bradley et al]
The Promoter Recognition DatasetComparative Test Results
Principal Topics
Breast cancer prognosis & chemotherapy
Kaplan-Meier Curves for Overall Patients:With & Without Chemotherapy
Breast Cancer Prognosis & ChemotherapyGood, Intermediate & Poor Patient Groupings
(6 Input Features : 5 Cytological, 1 Histological)(Clustering: Utilizes 2 Histological Features &Chemotherapy)
253 Patients(113 NoChemo, 140 Chemo)
Cluster 113 NoChemo PatientsUse k-Median Algorithm with Initial Centers:
Medians of Good1 & Poor1
69 NoChemo Good 44 NoChemo Poor 67 Chemo Good 73 Chemo Poor
Good PoorIntermediate
Cluster 140 Chemo PatientsUse k-Median Algorithm with Initial Centers:
Medians of Good1 & Poor1
Good1:Lymph=0 AND Tumor<2
Compute Median Using 6 Features
Poor1:Lymph>=5 OR Tumor>=4
Compute Median Using 6 Features
Compute InitialCluster Centers
Kaplan-Meier Survival Curvesfor Good, Intermediate & Poor Patients
82.7% Classifier Correctness via 3 SVMs
Kaplan-Meier Survival Curves for Intermediate Group Note Reversed Role of Chemotherapy
Multiple Myeloma DetectionMultiple Myeloma is cancer of the plasma cell
Plasma cells normally produce antibodiesOut of control plasma cells produce tumorsWhen tumors appear in multiple sites they are called Multiple Myeloma
Dataset105 patients: 74 with MM, 31 healthyEach patient is represented by 7008 gene measurements taken from plasma cell samplesFor each one of the 7008 gene measurements
Absolute Call (AC):Absent (A), Marginal (M) or Present (P)
Average Difference (AD):Positive or negative number
Multiple Myeloma Data Representation
A 1 0 0
M 0 1 0
P 0 0 1
AMP 7008 X 3 = 21024AD 7008 Total = 28,032 per patient104 Patients: 74 MM + 31 Healthy104 X 28,032 Data Matrix A
Multiple Myeloma 1-Norm SVM Linear Classifier
Leave-one-out-correctness (looc) = 100%Average number of features used = 7 per foldTotal computing time for 105 folds = 7892 sec. Overall number of features used in 105 folds= 7
Breast Cancer Treatment ResponseJoint with ExonHit - Paris (Curie Dataset)
35 patients treated by a drug cocktail 9 partial responders; 26 nonresponders25 gene expressions out of 692, selected by Arnaud Zeboulon Most patients had 3 replicate measurements1-Norm SVM classifier selected 14 out of 25 gene expressions
Leave-one-out correctness was 80%Greedy combinatorial approach selected 5 genes out of 14Separating plane obtained in 5-dimensional gene-expression space
Replicates of all patients except one used in trainingAverage of replicates of patient left out used for testing
Leave-one-out correctness was 33 out of 35, or 94.2%
Separation of Convex Hull of Replicates of:10 Synthetic Nonresponders & 4 Synthetic Partial Responders
Linear Classifier in 3-Gene Space35 Patients with 93 Replicates
26 Nonresponders & 9 Partial Responders
Conclusion
New approaches for SVM-based classification Algorithms capable of classifying data with few examples in very large dimensional spaces
Typical of microarray classification problemsClassifiers based on both abstract prior knowledge as well as conventional datasetsIdentification of breast cancer patients that can benefit from chemotherapy Useful tool for drug discovery