General InformationGeneral Information

Course Id: COSC6342 Machine Learning

Time: TU/TH 10a-11:30a

Instructor: Christoph F. Eick

Classroom: AH123

E-mail: ceick@aol.com

Homepage: http://www2.cs.uh.edu/~ceick/

22

What is Machine Learning?What is Machine Learning? Machine Learning Machine Learning is theis the

• study of algorithms thatstudy of algorithms that• improve their performanceimprove their performance• at some taskat some task• with experiencewith experience

Role of Statistics: Inference from a sampleRole of Statistics: Inference from a sample Role of Computer science: Efficient algorithms toRole of Computer science: Efficient algorithms to

• Solve optimization problemSolve optimization problemss• Representing and evaluating the model for Representing and evaluating the model for

inferenceinference

33

ApplicationsApplications of Machine Learning of Machine Learning Supervised LearningSupervised Learning

• ClassificationClassification• Prediction Prediction

Unsupervised LearningUnsupervised Learning• Association Analysis Association Analysis • Clustering Clustering

Preprocessing and Summarization of DataPreprocessing and Summarization of Data Reinforcement LearningReinforcement Learning Activities Related to Models Activities Related to Models

• Learning parameters of modelsLearning parameters of models• Choosing/Comparing modelsChoosing/Comparing models• ……

Prerequisites Prerequisites BackgroundBackground ProbabilitiesProbabilities

• Distributions, densities, marginalization…Distributions, densities, marginalization… Basic statisticsBasic statistics

• Moments, typical distributions, regression Moments, typical distributions, regression Basic knowledge of optimization techniquesBasic knowledge of optimization techniques AlgorithmsAlgorithms

• basic data structures, complexity…basic data structures, complexity… Programming skillsProgramming skills We provide some background, but the class will be fast We provide some background, but the class will be fast

pacedpaced Ability to deal with “abstract mathematical concepts”Ability to deal with “abstract mathematical concepts”

TextbooksTextbooks

Textbook:Textbook: Ethem Alpaydin, Introduction to Machine Learning, MIT Press, 2004.

Recommended Textbooks:Recommended Textbooks:

1.Christopher M. Bishop, Pattern Recognition and Machine Learning, 2006.

2.Tom Mitchell, Machine Learning, McGraw-Hill, 1997.

GradingGrading

3 Exams 67-70%Project 18-24%Homeworks 10-15%Attendance 1-2%

 

NOTE: PLAGIARISM IS NOT TOLERATED.

Remark: Weights are subject to change

Topics Covered in 2009 (Based on Alpaydin)Topics Covered in 2009 (Based on Alpaydin)Topic 1: Introduction Topic 2: Supervised Learning Topic 3: Bayesian Decision Theory (excluding Belief Networks) Topic 4: Using Curve Fitting as an Example to Discuss Major Issues in ML Topic 5: Parametric Model Selection Topic 6: Dimensionality Reduction Centering on PCA Topic 7: Clustering1: Mixture Models, K-Means and EM Topic 8: Non-Parametric Methods Centering on kNN and Density Estimation Topic 9: Clustering2: Density-based ApproachesTopic 10: Decision Trees Topic 11: Comparing Classifiers Topic 12: Combining Multiple Learners Topic 13: Linear Discrimination Topic 14: More on Kernel Methods Topic 15: Naive Bayes' and Belief Networks Topic 16: Hidden Markov Models Topic 17: Sampling Topic 18: Reinforcement Learning Topic 19: Neural Networks Topic 20: Computational Learning Theory

Remark: Topics 14, 16, 17, 19, and 20 likely will be only briefly covered or skipped---due to the lack of time.

Course ProjectCourse Project

The project will center on the application of machine learning techniques The project will center on the application of machine learning techniques to a challenging problem. It will be conducted in the window Feb. 12-April 11.to a challenging problem. It will be conducted in the window Feb. 12-April 11.

You can either conduct some novel experiments by applying machine learningYou can either conduct some novel experiments by applying machine learning algorithm(s) to a challenging machine learning task or attempt a theoretical algorithm(s) to a challenging machine learning task or attempt a theoretical analysis. analysis.

Findings of the project will be summarized in a report and in a brief presentation.Findings of the project will be summarized in a report and in a brief presentation. The report must include a short survey of related work with the corresponding The report must include a short survey of related work with the corresponding list list of references. of references.

Tentative Tentative ML Spring 2009 ScheduleML Spring 2009 Schedule

Week Topic

Jan 20 Introduction

Jan 27 Supervised Learning/Bayesian Decision Theory

Feb. 3 Curve Fitting/Model Estimation---Parametric Approaches

Feb. 10 Model Estimation---Parametric Approaches

Feb. 17 Parametric Approaches/Clustering1

Feb. 24 Clustering1/Non-param Methods

March 3 Non-Param Methods/Exam1

March 10 Clustering2/Dim. Reduction,Decision Trees

March 24 Dim. Reduction; DecisionTrees /Exam2

March 31 SVMs/Kernel Methods; Ensemble Methods

April 7 Comparing Classifiers/Group1 Presentations

April 14 Group2 Presentations/TBDL

April 21 Reinforcement Learning/possibly Belief Networks

April 28 Review/Exam3

March 31, 2009

Course ElementsCourse Elements

Total: 25-26 classes Total: 25-26 classes • 18 lectures18 lectures• 2-3 classes for review and discussing homework problems 2-3 classes for review and discussing homework problems • 2 classes will be allocated for student presentations2 classes will be allocated for student presentations• 3 exams3 exams• homeworkshomeworks

• individual gradedindividual graded• group graded group graded • not-graded (solutions will be discussed in lecture 7-9 not-graded (solutions will be discussed in lecture 7-9 days later). days later).

Dates to RememberDates to Remember

Dates to remember Events

March 5, March 26, April 30 Exams

April 9 and 14 Student Project Presentations

March 17 /19 No class (Spring Break)

April 13(Group1)/April 15(Group2) 11p

Submit Project Report /Software/…

ExamsExams

Will be open notes/textbookWill be open notes/textbook Will get a review list before the examWill get a review list before the exam Exams will center (80% or more) on material that was covered in the lectureExams will center (80% or more) on material that was covered in the lecture There will be a review prior to the second and third exam; first exam will mostly There will be a review prior to the second and third exam; first exam will mostly center on basics. center on basics. Exam scores will be immediately converted into number gradesExam scores will be immediately converted into number grades No sample exams; sorry I haven’t taught this course for a long time… No sample exams; sorry I haven’t taught this course for a long time…

Other UH-CS Courses with Overlapping ContentsOther UH-CS Courses with Overlapping Contents

1.1. COSC 6368: COSC 6368: Artificial IntelligenceArtificial Intelligence Strong Overlap: Decision Trees, Bayesian Belief NetworksStrong Overlap: Decision Trees, Bayesian Belief Networks Medium Overlap: Reinforcement LearningMedium Overlap: Reinforcement Learning

COSC 6335: COSC 6335: Data MiningData Mining Strong Overlap: Decision trees, SVM, kNN, Density- Strong Overlap: Decision trees, SVM, kNN, Density- based Clusteringbased Clustering Medium Overlap: K-means, Decision Trees, Medium Overlap: K-means, Decision Trees, Preprocessing/Exploratory DA, AdaBoostPreprocessing/Exploratory DA, AdaBoost

COSC 6343: Pattern ClassificationMedium Overlap: all classification algorithms, feature Medium Overlap: all classification algorithms, feature selection—discusses those topics taking selection—discusses those topics taking a different perspective. a different perspective.