Upload
colleen-dickerson
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Data Mining
Joyeeta Dutta-MoscatoJuly 10, 2013
Wherever we have large amounts of data, we have the need for building systems capable of learning information from the data
– predictions in medicine– text and web page classification– speech recognition
Learning underlying patterns useful to– to predict the presence of a disease for future
patients,– describe the dependencies between diseases
andsymptoms
Data Mining
Data Mining focuses on the discovery of (previously) unknown properties from data, using techniques from Machine Learning.
• 4 attributes / features• Each attribute has values
• 3 × 3 × 2 × 2 = 36 possible combinations• 14 combinations present in this example
Data
If outlook = sunny and humidity = high then play = noIf outlook = rainy and windy = true then play = noIf outlook = overcast then play = yesIf humidity = normal then play = yesIf none of the above then play = yes
A set of rules to predict whether we will get to play could look like this:
A decision list
Data Prediction
F = { <Outlook, Humidity, Wind, Temp> Play Tennis? }
Decision Tree Learning
The goal is to create a model that predicts the value of a target variable based on several input variables.
Problem Setting• Set of possible instances X
Each instance x in X is a feature vector x = < x1, x2, ... xn>• Unknown target function f: XY
Y is discrete valued• Set of function hypotheses H = { h | h : X Y }
Each hypothesis h is a decision tree Input: • Training examples {<x(i),y(i)>} of unknown target function f Output • Hypothesis h ∈ H that best approximates target function f
Decision Tree Learning
Supervised Learning
Given a set of training examples of the form:{(x1, y1), … (xn, yn)}
a learning algorithm seeks a function: g : X Y
Where X is the input space and Y is the output space. Example: - Classify the universe of music into ‘like’ & ‘dislike’ for one person - Training set: A list of songs that the person heard, and marked as ‘like’ or ‘dislike’ - Task: Infer a function of features (of these songs) to predict what other songs the person will like
Supervised Learning
Given a model family, we are interested in finding the best model parameters, such that the misfit (measured by an error function) between the data and the model is minimized.
An optimal scenario will allow for the algorithm to correctly determine the class labels for unseen instances.
Supervised LearningConsiderations:
• The learning algorithm must generalize from the training data to unseen situations in a "reasonable" way: Avoid overfitting
• Bias-variance tradeoff
• Number of training examples versus model complexity
Supervised LearningCommon methods of supervised learning:
• RegressionX discrete or continuous → Y continuous
Examples:– debt, equity, orders, sales → stock price– age, height, weight, race, VKORC1 genotype, CYP2C9
genotype → warfarin dose
• ClassificationX discrete or continuous → Y discreteExamples:- family history, history of head trauma, age, gender,
race,APOE status → Alzheimer’s disease
- arrangement of pixels in handwritten digit → “3”
• Linear Regression
Fitting the data to the model
Object: Minimize mean square error
Supervised Learning
Regression
Is a mean square error of 0 (i.e. no difference between prediction and target) mean this is the best model?
OverfittingReal test of ‘best model’ is performance on data it has not been trained on
Regression
What does this mean about the relationship between x and y?
• Linear classifier
Classification
• Logistic regressionHard threshold
Soft threshold
Uses the logistic function, which goes between 0 and 1
• Support Vector machines
Other common methods in Supervised Learning
• Artificial Neural Networks (can also be unsupervised)
• K-nearest neighbor
• Graphical models, Bayesian models
More sophisticated algorithms are needed for data that are not linearly separable
Unsupervised Learning
Learn relationships among the inputs, x1 , … xn .
No y is given.
Clustering – Group inputs based on some measure of
similarity- Common “first pass” exploratory data
mining technique
Hierarchical ClusteringA method of cluster analysis which aims to partition into groups that are “close” to each other according to some distance metric.
k-means ClusteringA method of cluster analysis which aims to partition the data into k clusters in which each observation belongs to the cluster with the nearest mean.
Acknowledgments
Shyam Visweswaran, Dept. of Biomedical Informatics
Tom Mitchell, Dept. of Machine Learning, CMU
“Data Mining: Practical Machine Learning Tools and Techniques” Ian H. Witten, Eibe Frank, Mark A. Hall