EECS738 Xue-wen Chen

EECS 738: Machine Learning

Fall 2011, Prof. Xue-wen ChenThe University of Kansas

EECS738 Xue-wen Chen

Machine Learning

• Predict the unknown from uncertain information

EECS738 Xue-wen Chen

Why Machine Learning?

EECS738 Xue-wen Chen

Speech Recognition

Hidden Markov models and their generalizations

EECS738 Xue-wen Chen

Tracking and Robot Localization

[Fox et al.] [Funiak et al.]

Kalman Filters

EECS738 Xue-wen Chen

Evolutionary Biology

[Friedman et al.]

Bayesian networks, Sequence alignment …

EECS738 Xue-wen Chen

Modeling Sensor DataUndirected graphical models

[Guestrin et al.]

EECS738 Xue-wen Chen

Planning Under Uncertainty

F’

E’

G’

P’ Peasant

Footman

Enemy

Gold

R

t t+1TimeAPeasant

ABuild

AFootman

P(F’|F,G,AB,AF)

[Guestrin et al.]

Dynamic Bayesian networksFactored Markov decision problems

EECS738 Xue-wen Chen

Images and Text Data

[Barnard et al.]

Hierarchical Bayesian models

EECS738 Xue-wen Chen

Structured Data (text, webpage,…)

[Koller et al.]Probabilistic relational models

EECS738 Xue-wen Chen

And many

many

many

many

manymore…

EECS738 Xue-wen Chen

Syllabus• About me the course (see the syllabus)• Covers a wide range of machine learning

topics (if time permits): from basic to state-of-the-art– Fundamentals– Supervised and unsupervised– SVM, NN, DTs– Bayesian networks– MCMC, Gibbs, EM– Gaussian and hybrid models, discrete and continuous variables– temporal and template models, hidden Markov Models, – Forwards-Backwards, Viterbi, Baum-Welch, Kalman filter,

• Covers algorithms, theory and application

EECS738 Xue-wen Chen

Prerequisites• Mathematical maturity:

– Vector/Matrix– Probabilities: distributions, densities, marginalization…– Basic statistics: moments, typical distributions, regression…– Optimization– Ability to deal with “abstract mathematical concepts”

• Programming– Experienced in at least one language (C, C++, Java, R, Matlab …)

• It’s going to be fun and hard work– Think before you decide: for credit only or for learning something– The class will be fast paced– Willing to spending time and efforts (in classroom and out…)– Dealing with mathematical formulas … CANNOT emphasize it

more– Understand it, program it– Fun only if you enjoy it …

EECS738 Xue-wen Chen

Text Books– Machine Learning: an algorithm perspective, Stephen

Marsland, CRC Press.– (optional) Tom Mitchell. Machine Learning, 1997,

WCB/McGraw-Hill.– (optional) Pattern Recognition and Machine Learning,

Christopher Bishop, Springer– Additional handouts will be provided as needed.

EECS738 Xue-wen Chen

Grades

• Exam: 40%• Final Project: 60%• Participation 10%

• The cutoffs for grades will be roughly as follows:

A: 90 – 100 B: 80 – 89 C: 70 – 79 D: 60 – 69 F: 0 – 59

EECS738 Xue-wen Chen

Exam

• To test – if you are ready!! – If you will survive

• Include but not limited to– Linear algebra– Matrix calculus– Probability ad Statistics– Optimization

EECS738 Xue-wen Chen

Project• Choose a topic that is related to your research interest and

pertains to the course material.

• The proposal should include the following sessions (Due: October 24) – the project goal, – the problems to be studied, – overview of current methods, – proposed methods, – expected results, and – references (about 4 pages: single space, fond size = 12, references

are not counted). References should be cited in the proposal.

• A written final report in the style of a journal article is also required. Final project is due by Dec. 07 (no late written project).

• Each student will give classroom presentation about the final project.

• Details: see the syllabus

EECS738 Xue-wen Chen

Some Important Dates

• October 24 – Project Proposal Due

• December 07 – Final project (written) due

EECS738 Xue-wen Chen

Tentative Lectures

• See syllabus• Preliminaries:

– matrix, statistics, optimization • Questions?