View
83
Download
9
Category
Tags:
Preview:
DESCRIPTION
In the Name of God. Machine Learning. Classification and Linear Classifiers. Mohammad Ali Keyvanrad. Thanks to: M . Soleymani (Sharif University of Technology ) R. Zemel (University of Toronto ) p. Smyth (University of California, Irvine). Fall 1392. Outline. Classification - PowerPoint PPT Presentation
Citation preview
Consistency of Learning Processes
Classification and Linear ClassifiersMohammad Ali KeyvanradMachine LearningFall 1392In the Name of GodThanks to: M. Soleymani (Sharif University of Technology)R. Zemel (University of Toronto)p. Smyth (University of California, Irvine)OutlineClassificationLinear classifiersPerceptronMulti-class classificationGenerative approachNave Bayes classifier
2Classification: Oranges and Lemons3
Classification: Oranges and Lemons4
Classification problem5Linear classifiers6
Linear classifiers7
7Decision boundary8Linear Decision boundary (Perceptron)9
Linear Decision boundary (Decision Tree)10t1t3t2IncomeLinear Decision boundary (K Nearest Neighbor)11OOOxxxFeature 1Feature 2Non-Linear Decision boundary12
Decision BoundaryDecision Region 1Decision Region 2Decision boundaryLinear classifier13
Non-linear decision boundaryChoose non-linear featuresClassifier still linear in parameters 14
Linear boundary: geometry15
SSE cost function for classification SSE cost function is not suitable for classificationSum of Squared Errors loss penalizes too correct predictionsSSE also lack robustness to noise16
SSE cost function for classification 17
Perceptron algorithm18
Perceptron criterion19
Batch gradient for descent PerceptronGradient Descent to solve the optimization problem
Batch Perceptron converges in finite number of steps for linearly separable data
20
Stochastic gradient descent for Perceptron21
Convergence of Perceptron22
Convergence of Perceptron23
Multi-class classification24Multi-class classificationOne-vs-all (one-vs-rest)25
Multi-class classificationOne-vs-one26
Multi-class classification: ambiguityConverting the multi-class problem to a set of two-class problems can lead to regions in which the classification is undefined27
Probabilistic approachBayes theorem
28
Bayes theorem29
Bayes decision theory30
Probabilistic classifiersProbabilistic classification approaches can be divided in two main categoriesGenerativeDiscriminative31Discriminative vs. generative approach32
Generative approach33Discriminative approach34Nave Bayes classifier35Nave Bayes classifier36
Nave Bayes: discrete example37
38
Recommended