View
217
Download
0
Category
Preview:
Citation preview
STATISTICAL LEARNING(FROM DATA TO DISTRIBUTIONS)
AGENDA
Learning a discrete probability distribution from data
Maximum likelihood estimation (MLE) Maximum a posteriori (MAP) estimation
MOTIVATION
Previously studied how to infer characteristics of a distribution, given a fully-specified Bayes net
This lecture: where does the specification of the Bayes net come from? Specifically: estimating the parameters of the
CPTs from fully-specified data
LEARNING IN GENERAL
Agent has made observations (data) Now must make sense of it (hypotheses) Why?
Hypotheses alone may be important (e.g., in basic science)
For inference (e.g., forecasting) To take sensible actions (decision making)
A basic component of economics, social and hard sciences, engineering, …
MACHINE LEARNING VS. STATISTICS
Machine Learning automated statistics This lecture
Statistical learning (aka Bayesian learning) Maximum likelihood (ML) learning Maximum a posteriori (MAP) learning Learning Bayes Nets (R&N 20.1-3)
Future lectures try to do more with even less data Decision tree learning Neural nets Support vector machines …
PUTTING GENERAL PRINCIPLES TO PRACTICE
Note: this lecture will cover general principles of statistical learning on toy problems
Grounded in some of the most theoretically principled approaches to learning But the techniques are far more general Practical applications to larger problems requires
a bit of mathematical savvy and “elbow grease”
BEAUTIFUL RESULTS IN MATH GIVE RISE TO SIMPLE IDEAS
OR
HOW TO JUMP THROUGH HOOPS IN ORDER TO JUSTIFY SIMPLE IDEAS …
CANDY EXAMPLE
• Candy comes in 2 flavors, cherry and lime, with identical wrappers
• Manufacturer makes 5 indistinguishable bags
• Suppose we draw• What bag are we holding? What flavor will we draw
next?
h1C: 100%L: 0%
h2C: 75%L: 25%
h3C: 50%L: 50%
h4C: 25%L: 75%
h5C: 0%L: 100%
BAYESIAN LEARNING
Main idea: Compute the probability of each hypothesis, given the data
Data d: Hypotheses: h1,…,h5
h1C: 100%L: 0%
h2C: 75%L: 25%
h3C: 50%L: 50%
h4C: 25%L: 75%
h5C: 0%L: 100%
BAYESIAN LEARNING
Main idea: Compute the probability of each hypothesis, given the data
Data d: Hypotheses: h1,…,h5
h1C: 100%L: 0%
h2C: 75%L: 25%
h3C: 50%L: 50%
h4C: 25%L: 75%
h5C: 0%L: 100%
P(hi|d)
P(d|hi)
We want this…
But all we have is this!
USING BAYES’ RULE
P(hi|d) = a P(d|hi) P(hi) is the posterior
(Recall, 1/a = P(d) = Si P(d|hi) P(hi))
P(d|hi) is the likelihood
P(hi) is the hypothesis prior
h1C: 100%L: 0%
h2C: 75%L: 25%
h3C: 50%L: 50%
h4C: 25%L: 75%
h5C: 0%L: 100%
LIKELIHOOD AND PRIOR
Likelihood is the probability of observing the data, given the hypothesis model
Hypothesis prior is the probability of a hypothesis, before having observed any data
COMPUTING THE POSTERIOR
Assume draws are independent Let P(h1),…,P(h5) = (0.1, 0.2, 0.4, 0.2, 0.1) d = { 10 x }
P(d|h1) = 0P(d|h2) = 0.2510
P(d|h3) = 0.510
P(d|h4) = 0.7510
P(d|h5) = 110
P(d|h1)P(h1)=0P(d|h2)P(h2)=9e-8P(d|h3)P(h3)=4e-4P(d|h4)P(h4)=0.011P(d|h5)P(h5)=0.1
P(h1|d) =0P(h2|d) =0.00P(h3|d) =0.00P(h4|d) =0.10P(h5|d) =0.90
Sum = 1/a = 0.1114
POSTERIOR HYPOTHESES
PREDICTING THE NEXT DRAW
P(X|d) = Si P(X|hi,d)P(hi|d)
= Si P(X|hi)P(hi|d)
P(h1|d) =0P(h2|d) =0.00P(h3|d) =0.00P(h4|d) =0.10P(h5|d) =0.90
H
D X
P(X|h1) =0P(X|h2) =0.25P(X|h3) =0.5P(X|h4) =0.75P(X|h5) =1
Probability that next candy drawn is a lime
P(X|d) = 0.975
P(NEXT CANDY IS LIME | D)
PROPERTIES OF BAYESIAN LEARNING
If exactly one hypothesis is correct, then the posterior probability of the correct hypothesis will tend toward 1 as more data is observed
The effect of the prior distribution decreases as more data is observed
HYPOTHESIS SPACES OFTEN INTRACTABLE To learn a probability distribution, a
hypothesis would have to be a joint probability table over state variables 2n entries => hypothesis space is 2n-1-
dimensional! 2^(2n) deterministic hypotheses
6 boolean variables => over 1022 hypotheses And what the heck would a prior be?
LEARNING COIN FLIPS Let the unknown fraction of cherries be q
(hypothesis) Probability of drawing a cherry is q Suppose draws are independent and
identically distributed (i.i.d) Observe that c out of N draws are cherries
(data)
LEARNING COIN FLIPS Let the unknown fraction of cherries be q
(hypothesis) Intuition: c/N might be a good hypothesis (or it might not, depending on the draw!)
MAXIMUM LIKELIHOOD
Likelihood of data d={d1,…,dN} given q
P(d|q) = Pj P(dj|q) = qc (1-q)N-c
i.i.d assumption Gather c cherry terms together, then N-c lime terms
MAXIMUM LIKELIHOOD
Likelihood of data d={d1,…,dN} given q P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
1/1 cherry
q
P(d
ata
|q)
MAXIMUM LIKELIHOOD
Likelihood of data d={d1,…,dN} given q P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.2
0.4
0.6
0.8
1
1.2
2/2 cherry
q
P(d
ata
|q)
MAXIMUM LIKELIHOOD
Likelihood of data d={d1,…,dN} given q P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
2/3 cherry
q
P(d
ata
|q)
MAXIMUM LIKELIHOOD
Likelihood of data d={d1,…,dN} given q P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.01
0.02
0.03
0.04
0.05
0.06
0.07
2/4 cherry
q
P(d
ata
|q)
MAXIMUM LIKELIHOOD
Likelihood of data d={d1,…,dN} given q P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
2/5 cherry
q
P(d
ata
|q)
MAXIMUM LIKELIHOOD
Likelihood of data d={d1,…,dN} given q P(d|q) = qc (1-q)N-c
0 0.10.20.30.40.50.60.70.80.9 10
0.0000002
0.0000004
0.0000006
0.0000008
0.000001
0.0000012
10/20 cherry
q
P(d
ata
|q)
MAXIMUM LIKELIHOOD
Likelihood of data d={d1,…,dN} given q P(d|q) = qc (1-q)N-c
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
1E-31
2E-31
3E-31
4E-31
5E-31
6E-31
7E-31
8E-31
9E-31
50/100 cherry
q
P(d
ata
|q)
MAXIMUM LIKELIHOOD
Peaks of likelihood function seem to hover around the fraction of cherries…
Sharpness indicates some notion of certainty…
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
1E-31
2E-31
3E-31
4E-31
5E-31
6E-31
7E-31
8E-31
9E-31
50/100 cherry
q
P(d
ata
|q)
MAXIMUM LIKELIHOOD
P(d|q) be the likelihood function The quantity argmaxq P(d|q) is known as the
maximum likelihood estimate (MLE)
MAXIMUM LIKELIHOOD
The maximum of P(d|q) is obtained at the same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
So the MLE is the same as maximizing log likelihood… but: Multiplications turn into additions We don’t have to deal with such tiny numbers
MAXIMUM LIKELIHOOD
The maximum of P(d|q) is obtained at the same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
l(q) = log P(d|q) = log [ qc (1-q)N-c]
MAXIMUM LIKELIHOOD
The maximum of P(d|q) is obtained at the same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
l(q) = log P(d|q) = log [ qc (1-q)N-c]= log [ qc ] + log [(1-q)N-c]
MAXIMUM LIKELIHOOD
The maximum of P(d|q) is obtained at the same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
l(q) = log P(d|q) = log [ qc (1-q)N-c]= log [ qc ] + log [(1-q)N-c]= c log q + (N-c) log (1-q)
MAXIMUM LIKELIHOOD
The maximum of P(d|q) is obtained at the same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
l(q) = log P(d|q) = c log q + (N-c) log (1-q) At a maximum of a function, its derivative is 0 So, dl/dq( )q = 0 at the maximum likelihood
estimate
MAXIMUM LIKELIHOOD
The maximum of P(d|q) is obtained at the same place that the maximum of log P(d|q) is obtained Log is a monotonically increasing function
l(q) = log P(d|q) = c log q + (N-c) log (1-q) At a maximum of a function, its derivative is 0 So, dl/dq( )q = 0 at the maximum likelihood
estimate => 0 = c/q – (N-c)/(1-q)
=> q = c/N
OTHER CLOSED-FORM MLE RESULTS
Multi-valued variables: take fraction of counts for each value
Continuous Gaussian distributions: take average value as mean, standard deviation of data as standard deviation
MAXIMUM LIKELIHOOD FOR BN
For any BN, the ML parameters of any CPT can be derived by the fraction of observed values in the data, conditioned on matched parent values
Alarm
Earthquake Burglar
E: 500 B: 200
N=1000
P(E) = 0.5 P(B) = 0.2
A|E,B: 19/20A|E,B: 188/200A|E, B: 170/500A|E,B : 1/380
E B P(A|E,B)
T T 0.95
F T 0.95
T F 0.34
F F 0.003
BAYES NET ML ALGORITHM
Input: Bayes net with nodes X1,…,Xn, dataset D=(d[1],…,d[N]) Each d[i] = (x1[i],…,xn[i]) is a sample of the full
state of the world
For each node X with parents Y1,…,Yk: For all y1Val(Y1),…, ykVal(Yk)
For all xVal(X) Count the number of times (X=x, Y1=y1,…, Yk=yk) is
observed in D. Let this be mx
Count the number of times (Y1=y1,…, Yk=yk) is observed in D. Let this be m. (note m=xmx)
Set P(x|y1,…,yk) = mx / m for all xVal(X)
MAXIMUM LIKELIHOOD PROPERTIES
As the number of data points approaches infinity, the MLE approaches the true estimate
With little data, MLEs can vary wildly
MAXIMUM LIKELIHOOD IN CANDY BAG EXAMPLE
hML = argmaxhi P(d|hi)
P(X|d) P(X|hML)
hML = undefined h5
P(X|hML)
P(X|d)
BACK TO COIN FLIPS
The MLE is easy to compute… but what about those small sample sizes?
Motivation You hand me a coin from your pocket 1 flip, turns up tails Whats the MLE?
A particularly acute problem for BN nodes with many parents!
(the data fragmentation problem)
MAXIMUM A POSTERIORI ESTIMATION
Maximum a posteriori (MAP) estimation Idea: use the hypothesis prior to get a better
initial estimate than ML, without resorting to full Bayesian estimation “Most coins I’ve seen have been fair coins, so I
won’t let the first few tails sway my estimate much”
“Now that I’ve seen 100 tails in a row, I’m pretty sure it’s not a fair coin anymore”
MAXIMUM A POSTERIORI
P(q|d) is the posterior probability of the hypothesis, given the data
argmaxq P(q|d) is known as the maximum a posteriori (MAP) estimate
Posterior of hypothesis q given data d={d1,…,dN} P(q|d) = 1/a P(d|q) P(q) Max over q doesn’t affect a So MAP estimate is argmaxq P(d|q) P(q)
MAXIMUM A POSTERIORI
hMAP = argmaxhi P(hi|d)
P(X|d) P(X|hMAP)
hMAP = h3 h4 h5
P(X|hMAP)
P(X|d)
ADVANTAGES OF MAP AND MLE OVER BAYESIAN ESTIMATION Involves an optimization rather than a large
summation Local search techniques
For some types of distributions, there are closed-form solutions that are easily computed
BACK TO COIN FLIPS
Need some prior distribution P(q) P(q|d) P(d|q)P(q) = qc (1-q)N-c P(q)
Define, for all q, the probability that I believe in q
10 q
P(q)
MAP ESTIMATE
Could maximize qc (1-q)N-c P(q) using some optimization
Turns out for some families of P(q), the MAP estimate is easy to compute
10 q
P(q)
Beta distributions
BETA DISTRIBUTION
Betaa,b(q) = g qa-1 (1-q)b-1
a, b hyperparameters > 0 g is a normalization
constant a=b=1 is uniform
distribution
POSTERIOR WITH BETA PRIOR
Posterior qc (1-q)N-c P(q)= g qc+a-1 (1-q)N-c+b-1
= Betaa+c,b+N-c(q)
Prediction = meanE[ ]q =(c+a)/(N+a+b)
POSTERIOR WITH BETA PRIOR
What does this mean? Prior specifies a “virtual
count” of a=a-1 heads, b=b-1 tailsSee heads, increment aSee tails, increment b
MAP estimate is a/(a+b)Effect of prior diminishes
with more data
MAP FOR BN
For any BN, the MAP parameters of any CPT can be derived by the fraction of observed values in the data + the prior virtual counts
Alarm
Earthquake Burglar
E: 500+100 B: 200+200
N=1000+1000
P(E) = 0.5 0.3 P(B) = 0.2 0.2
A|E,B: (19+100)/(20+100)A| E,B: (188+180)/(200+200)A|E, B: (170+100)/(500+400)A|E,B : (1+3)/(380 + 300)
E B P(A|E,B)
T T 0.95 0.99
F T 0.94 0.92
T F 0.34 0.3
F F 0.0030.006
EXTENSIONS OF BETA PRIORS
Multiple discrete values: Dirichlet prior Mathematical expression more complex, but in
practice still takes the form of “virtual counts” Mean, standard deviation for Gaussian
distributions: Gamma prior Conjugate priors preserve the representation
of prior and posterior distributions, but do not necessary exist for general distributions
RECAP
Bayesian learning: infer the entire distribution over hypotheses Robust but expensive for large problems
Maximum Likelihood (ML): optimize likelihood Good for large datasets, not robust for small
ones Maximum A Posteriori (MAP): weight
likelihood by a hypothesis prior during optimization A compromise solution for small datasets Like Bayesian learning, how to specify the prior?
HOMEWORK
Read R&N 18.1-3
APPLICATION TO STRUCTURE LEARNING
So far we’ve assumed a BN structure, which assumes we have enough domain knowledge to declare strict independence relationships
Problem: how to learn a sparse probabilistic model whose structure is unknown?
BAYESIAN STRUCTURE LEARNING
But what if the Bayesian network has unknown structure?
Are Earthquake and Burglar independent? P(E,B) = P(E)P(B)?
Earthquake Burglar
?B E
0 0
1 1
0 1
1 0
… …
BAYESIAN STRUCTURE LEARNING
Hypothesis 1: dependent P(E,B) as a probability table has 3 free
parameters Hypothesis 2: independent
Individual tables P(E),P(B) have 2 total free parameters
Maximum likelihood will always prefer hypothesis 1!
Earthquake Burglar
B E
0 0
1 1
0 1
1 0
… …
BAYESIAN STRUCTURE LEARNING
Occam’s razor: Prefer simpler explanations over complex ones
Penalize free parameters in probability tables
Earthquake Burglar
B E
0 0
1 1
0 1
1 0
… …
ITERATIVE GREEDY ALGORITHM
Fit a simple model (ML), compute likelihood Repeat:
Earthquake Burglar
B E
0 0
1 1
0 1
1 0
… …
ITERATIVE GREEDY ALGORITHM
Fit a simple model (ML), compute likelihood Repeat:
Pick a candidate edge to add to the BN
Earthquake Burglar
B E
0 0
1 1
0 1
1 0
… …
ITERATIVE GREEDY ALGORITHM
Fit a simple model (ML), compute likelihood Repeat:
Pick a candidate edge to add to the BN Compute new ML probabilities
Earthquake Burglar
B E
0 0
1 1
0 1
1 0
… …
ITERATIVE GREEDY ALGORITHM
Fit a simple model (ML), compute likelihood Repeat:
Pick a candidate edge to add to the BN Compute new ML probabilities If new likelihood exceeds old likelihood +
threshold, Add the edge
Otherwise, repeat with a different edge Earthquake Burglar
B E
0 0
1 1
0 1
1 0
… …
(Usually log likelihood)
OTHER TOPICS IN LEARNING DISTRIBUTIONS
Learning continuous distributions using kernel density estimation / nearest neighbor smoothing
Expectation Maximization for hidden variables Learning Bayes nets with hidden variables Learning HMMs from observations Learning Gaussian Mixture Models of continuous
data
NEXT TIME
Introduction to machine learning R&N 18.1-2
Recommended