Upload
imogen-banks
View
223
Download
0
Tags:
Embed Size (px)
Citation preview
CS558 Project
Local SVM Classification based on triangulation
(on the plane)
Glenn Fung
Outline of Talk
Classification problem on the plane All of the recommended stages were applied:
Sampling Ordering:
Clustering Triangulation
Interpolation (Classification)SVM: Support vector Machines
Optimization: Number of training points increased Evaluation:
Checkerboard datasetSpiral dataset
Classification Problem in
Given m points in 2 dimensional space Represented by an m-by-2 matrix A Membership of each in class +1 or –1A i
R 2
SAMPLING:
1000 randomly sampled points
ORDERING:
Clustering A Fuzzy-logic based clustering algorithm was used. 32 cluster centers were obtained
-50 0 50 100 150 200 250-50
0
50
100
150
200
250
0 50 100 150 200
0
50
100
150
200
ORDERING:
Delaunay Triangulation Algorithms to triangulate and to get the Delaunay triangulation from HWKs 3 and 4 were used. Given a point,the random point approach is used to localize the triangle that contains it.
Interpolation:
SVM SVM : Support Vector Machine Classifiers A different nonlinear Classifier is used for each triangle
The triangle structure is efficiently used for both training and testing phases and for defining a “simple” and fast nonlinear classifier.
What is a Support Vector Machine?
An optimally defined surface Typically nonlinear in the input space Linear in a higher dimensional space Implicitly defined by a kernel function
What are Support Vector Machines Used For?
Classification Regression & Data Fitting Supervised & Unsupervised Learning
(Will concentrate on classification)
Support Vector MachinesMaximizing the Margin between Bounding
Planes
x0w= í +1
x0w= í à 1
A+
A-
jjwjj22
w
The Nonlinear Classifier
K (A;A0) : Rmân â Rnâm7à! Rmâm
K (x0;A0)Du = í
The nonlinear classifier:
Where K is a nonlinear kernel, e.g.: Gaussian (Radial Basis) Kernel :
"àökA iàA jk22; i; j = 1;. . .;mK (A;A0)ij =
The ij -entry of K (A;A0) represents the “similarity” of data points A i A jand
Reduced Support Vector Machine AlgorithmNonlinear Separating Surface: K (x0;Aö0)Döuö= í
(i) Choose a random subset matrix ofA 2 Rmân
entire data matrix A 2 Rmân
(ii) Solve the following problem by the Newtonmethod with corresponding D ú D :
2÷kp(eà D(K (A;A0)Döuöà eí );ë)k22+ 2
1kuö; í k22min(u; í ) 2 Rm+1
K (x0;Aö0)Döuö= í
(iii) The separating surface is defined by the optimal(u;í )solution in step (ii):
How to Choose in RSVM?A
A is a representative sample of the entire dataset Need not be a subset of A
A good selection of A may generate a classifier usingvery small m
Possible ways to chooseA :
Choose random rows from the entire datasetm A Choose such that the distance between its rows A
exceeds a certain tolerance Use k cluster centers of Aas AàA+ and
Obtained Bizarre “Checkerboard”
Optimization: More sampled pointsTraining parameters adjusted
Result: Improved Checkerboard
Nonlinear PSVM: Spiral Dataset94 Red Dots & 94 White Dots
Next:Bascom Hill
Some Questions
Would it work for B&W pictures (regression instead of classification?
Aplications?