31
Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Embed Size (px)

DESCRIPTION

Local Likelihood and Other Models Example: Likelihood The probability of the parameters fitting the data, as opposed to the other way around [Wikipedia]

Citation preview

Page 1: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Local Likelihood & other models, Kernel Density Estimation & Classification, Radial

Basis Functions

Page 2: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Local Likelihood and Other Models• Other parametric models can be made local• Any such model that supports observation weights is easily adapted.

Page 3: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Local Likelihood and Other Models• Example: Likelihood• The probability of the parameters fitting the data, as opposed to the other

way around• [Wikipedia]

Page 4: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Local Likelihood and Other Models• Example: Likelihood

Page 5: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Local Likelihood and Other Models• Example: Likelihood

Page 6: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Local Likelihood and Other Models• Example: Likelihood• Many likelihood models involve the covariates in a linear fashion.• This lets you take advantage of this while only requiring local linearity of the

model.

Page 7: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Local Likelihood and Other Models• Example: Log Likelihood

Page 8: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Local Likelihood and Other Models• Example: Log Likelihood

Page 9: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Local Likelihood and Other Models• Example: Log Likelihood• Log is monotonically increasing

• Same local max/min etc as original function• Often Easier to find these maximas from log likelihood

Page 10: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

• What are the differences among density estimation, classification, and regression? What are the uses of kernels in each of them? (-Grace)• What is Kernel density? (-Yifang)

Page 11: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Estimation and Classification

Page 12: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Estimation

Page 13: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Estimation• Natural local estimate

Page 14: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Estimation• Parzen estimate

Page 15: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Estimation• Parzen Estimate, choice of K_{lambda}

Page 16: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Estimation• Parzen Estimate, choice of K_{lambda}

Page 17: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Estimation• Parzen Estimate, choice of K_{lambda}

• How do we get [the second equation above]? (-Brendan)

Page 18: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Classification

Page 19: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Classification

Page 20: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Classification

Page 21: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Kernel Density Classification

Page 22: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Idiot’s Bayes

Page 23: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Idiot’s Bayes• What is the relationship between Naive bayes classifier and kernel

functions that we study in this chapter? (Grace)

Page 24: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Radial Basis Functions• Basis Functions

[image from brnt.eu]

Page 25: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Radial Basis Functions• Basis Functions• Kernel Methods

Page 26: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Radial Basis Functions

Page 27: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Radial Basis Functions

Page 28: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Radial Basis Functions

Page 29: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Radial Basis Functions

Page 30: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Radial Basis Functions

Page 31: Local Likelihood & other models, Kernel Density Estimation & Classification, Radial Basis Functions

Radial Basis Functions