Machine LearningCS 165B

Spring 2012

1

Course outline

2

Eager versus Lazy Learners

• Eager learner– Learn the decision boundary at training time

– Simple distributions

• Lazy learners– Learn the decision boundary at testing time

– High cost of classification

– Can learn complex local distributions

– How to define distance?

– Dimensionality reduction techniques

3

K-nearest neighbor

• Classify based on the majority label of k-nearest neighbors

• Voronoi diagram defines the decision boundary for 1-nearest neighbor

• Distance-weighted nearest neighbor

• Possible to scale each dimension based on cross-validation

4

Locally weighted regression

• Approximate target function by considering weighted contribution of points close to the query point

• Generalization of k-nearest neighbor classification

• Radial basis functions– Choose a set of kernel functions

– Use their weighted combination to learn the target function

– Usually Gaussian kernels

– Can be achieved by a 2-layer ANN

5