Upload
randell-flowers
View
220
Download
0
Embed Size (px)
Citation preview
Bayesian approaches to knowledge representation and reasoning
Part 1(Chapter 13)
Bayesianism vs. Frequentism
• Classical probability: Frequentists
– Probability of a particular event is defined relative to its frequency in a sample space of events.
– E.g., probability of “the coin will come up heads on the next trial” is defined relative to the frequency of heads in a sample space of coin tosses.
• Bayesian probability:
– Combine measure of “prior” belief you have in a proposition with your subsequent observations of events.
• Example: Bayesian can assign probability to statement “The first e-mail message ever written was not spam” but frequentist cannot.
Bayesian Knowledge Representation and Reasoning
• Question: Given the data D and our prior beliefs, what is the probability that h is the correct hypothesis? (spam example)
• Bayesian terminology (example -- spam recognition)
– Random variable X: returns one of a set of values
{x1, x2, ...,xm},
or a continuous value in interval [a,b] with probability distribution D(X).
– Data D: {v1, v2, v3, ...} Set of observed values of random variables X1, X2, X3, ...
– Hypothesis h: Function taking instance j and returning classification of j (e.g., “spam” or “not spam”).
– Space of hypotheses H: Set of all possible hypotheses
– Prior probability of h:
• P(h): Probability that hypothesis h is true given our prior knowledge
• If no prior knowledge, all h H are equally probable
– Posterior probability of h:
• P(h|D): Probability that hypothesis h is true, given the data D.
– Likelihood of D:
• P(D|h): Probability that we will see data D, given hypothesis h is true.
Recall definition of conditional probability:
)(
)()|(
YP
YXPYXP
X Yevent space
Event space = all e-mail messages
X = all spam messages
Y = all messages containing word “v1agra”
Bayes Rule:
)(
)()|()|(
YP
XPXYPYXP
)()|()()()|(
:Proof
XPXYPYXPYPYXP
X Yevent space
Example: Using Bayes Rule
Hypotheses:
h = “message m is spam”
h = “message m is not spam”
Data:
+ = message m contains “viagra”
– = message m does not contain “viagra”
Prior probability:
P(h) = 0.1 P(h) = 0.9
Likelihood:
P(+ | h) = 0.6 P(– | h) = 0.4
P(+ | h) = 0.03, P(– | h) = 0.97
P(+) = P(+ | h) P(h) + P(+ | h)P(h)
= 0.6 * .1 + .03 * .9 = 0.09
P(–) = 0.91
P(h | +) = P(+ | h) P(h) / P(+)
= 0.6 * 0.1 / .09 = .67
How would we learn these prior probabilities and likelihoods from past examples of spam and not spam?
Full joint probability distribution(CORRECTED)
“viagra” “viagra”
Spam .06 .04
Spam .027 .873
Notation: P(h,D) P(h D)
P (h +) = P(h | +) P(+)
P(h -) = P(h | -) P(-)
etc.
Now suppose there is a second feature examined: does message contain the word “offer”?
viagra “viagra” “viagra” “viagra”
spam
spam
offer offer
Full joint distribution scales exponentially with number of parameters
P(m=spam, viagra, offer)
• Bayes optimal classifier for spam:
where fi is a feature (here, could be a “keyword”)
• In general, intractable.
),...,,|(maxarg 21},{
nspamspamh
fffhP
• Classification using “naive Bayes”
• Assumes that all features are independent of one another.
• How do we learn the naive Bayes model from data?
• How do we apply the naive Bayes model to a new instance?
)|()(),...,,|( 21 hfPhPfffhPi
in
Example: Training and Using Naive Bayes for Classification
• Features:
– CAPS: Boolean (longest contiguous string of capitalized letters in message is longer than 3)
– URL: Boolean (0 if no URL in message, 1 if at least one URL in message)
– $: Boolean (0 if $ does not appear at least once in message; 1 otherwise)
• Training data:
M1: “DON’T MISS THIS AMAZING OFFER $$$!” spam
M2: “Dear mm, for more $$, check this out: http://www.spam.com” spam
M3: “I plan to offer two sections of CS 250 next year” not spam
M4: “Hi Mom, I am a bit short on $$ right now, can you not spam
send some ASAP? Love, me”
Training a Naive Bayes Classifier
• Two hypotheses: spam or not spam
• Estimate: P(spam) = .5 P(spam) = .5
P(CAPS | spam) = .5 P(CAPS | spam) = .5
P(URL | spam) = .5 P(URL | spam) = .5
P($ | spam) = .75 P($ | spam) = .25
P(CAPS | spam )= .5 P(CAPS | spam) = .5
P(URL | spam) = .25 P(URL | spam) = .75
P($ | spam) = .5 P($ | spam) =.5
)|()(),...,,,( 21 hfPhPfffhPi
in
• m-estimate of probability (to fix cases where one of the terms in the product is 0):
.2 and 2
1set we typicallyfeatures,Boolean For
. weight heavily to how determineshat constant t a is
for estimateprior is
true,is for which examples training ofnumber theis
trueis for which examples trainingofnumber theis where
,)|(
i
mp
pm
fhp
ftruehn
hn
mn
mpnhfP
ic
ci
• Now classify new message:
M4: “This is a ONE-TIME-ONLY offer that will get you BIG $$$, just click on http://www.spammers.com”
Information Retrieval
• Most important concepts:
– Defining features of a document
– Indexing documents according to features
– Retrieving documents in response to a query
– Ordering retrieved documents by relevance
• Early search engines:
– Features: List of all terms (keywords) in document (minus “a”, “the”, etc.)
– Indexing: by keyword
– Retrieval: by keyword match with query
– Ordering: by number of keywords matched
• Problems with this approach
Naive Bayesian Document retrieval• Let D be a document (“bag of words”), Q be a query (“bag of words”), and r be the event that D is
relevant to Q.
• In document retreival, we want to compute:
• Or, “odds ratio”:
• In the book, they show (via a lot of algebra) that
• Chain rule: P(A,B) = p(A|B) p(B)
),|( QDrP
),|(
),|(
QDrP
QDrP
document" theof relevance" intrinsic" ),|(
)|(
)|(),|(
),|(
),|(
rDQP
DrP
DrPrDQP
QDrP
QDrP
Naive Bayesian Document retrieval• Let D be a document (“bag of words”), Q be a query (“bag of words”), and r be the event that D is
relevant to Q.
• In document retreival, we want to compute:
• Or, “odds ratio”:
• In the book, they show (via a lot of algebra) that
• Chain rule: P(A,B) = p(A|B) p(B)
),|( QDrP
),|(
),|(
QDrP
QDrP
document" theof relevance" intrinsic" ),|(
)|(
)|(),|(
),|(
),|(
rDQP
DrP
DrPrDQP
QDrP
QDrP
Naive Bayesian Document retrieval
• Where Qj is the jth keyword in the query.
• The probability of a query given a relevant document D is estimated as the product of the probabilities of each keyword in the query, given the relevant document.
• How to learn these probabilities?
j
j rDQPtRDQP ),|(),|(
Evaluating Information Retrieval Systems• Precision and Recall
• Example: Out of corpus of 100 documents, query has following results:
• Precision: Fraction of relevant documents in results set = 30/40 = .75 “How precise is results set?”
• Recall: Fraction of relevant documents in whole corpus that are in results set = 30/50 = .60 “How many relevant documents were recalled?”
In results set Not in results set
Relevant 30 20
Not relevant 10 40
• Tradeoff between recall and precision:
If we want to ensure that recall is high, just recall a lot of documents. Then precision may be low. If we recall 100% of documents, but only 50% are relevant, then recall is 1, but precision is 0.5.
If we want high chance for precision to be high, just recall the single document judged most relevant (“I’m feeling lucky” in Google.) Then precision will (likely) be 1.0, but recall will be low.
When do you want high precision? When do you want high recall?
Bayesian approaches to knowledge representation and reasoning
Part 2(Chapter 14, sections 1-4)
• Recall Naive Bayes method:
• This can also be written in terms of “cause” and “effect”:
)|()()...,|( ,,1 hfPhPffhPi
in
)|()()...,|( ,,1 CauseeffectPCausePeffecteffectCausePi
in
Spam
v1agra
stockoffer
cause
effects
Naive Bayes
Bayesian network
Spam
v1agra stock
offer
Each node has a “conditional probability table” that gives its dependencies on its parents.
Spam
v1agra stock
offer
P(Spam)
0.1
Spam P(v1agra)
t 0.6
f 0.03
Spam P(stock)
t 0.2
f 0.3 stock P(offer)
t 0.6
f 0.1
Semantics of Bayesian networks
• If network is correct, can calculate full joint probability distribution from network.
where parents(Xi) denotes specific values of parents of Xi.
))(|(),...,(
)...(
11
11
i
n
iin
nn
XparentsxPxxP
xXxXP
offer offer offer offer
Spam .012 .008 .008 .072
Spam .162 .108 .063 .567
stock stock
Sum of all boxesis 1.
Example from textbook
• I'm at work, neighbor John calls to say my alarm is ringing, but neighbor Mary doesn't call. Sometimes it's set off by minor earthquakes. Is there a burglar?
• Variables: Burglary, Earthquake, Alarm, JohnCalls, MaryCalls
• Network topology reflects "causal" knowledge:– A burglar can set the alarm off– An earthquake can set the alarm off– The alarm can cause Mary to call– The alarm can cause John to call
Example continued
Complexity of Bayesian Networks
• For n random Boolean variables:
– Full joint probability distribution: 2n entries
– Bayesian network with at most k parents per node:
• Each conditional probability table: at most 2k entries
• Entire network: n 2k entries
Exact inference in Bayesian networks
Query:
What is P(Burglary | JohnCalls=true ^ MaryCalls = true)?
Notation: Capital letters are distributions; lower case letters are values or variables, depending on context.
We have:
},{ },{
)(
)()()(
)(),|(
falsetruee falsetruea
mjaeBP
mjBPmPjP
mjBPmjBP
e a
amPajPebaPePbPmjbP )|()|(),|()()( ),|(
Let’s calculate this for b = “Burglary = true”:
Worse case complexity: O(n 2n), where n is number of Boolean
variables.
We can simplify:
e a
amPajPebaPePbPmjbP )|()|(),|()()( ),|(
A. Onisko et al., A Bayesian network model for diagnosis of liver disorders
Can speed up further via “variable elimination”.
However, bottom line on exact inference:
In general, it’s intractable. (Exponential in n.)
Solution:
Approximate inference, by sampling.
Bayesian approaches to knowledge representation and reasoning
Part 3(Chapter 14, section 5)
What are the advantages of Bayesian networks?
• Intuitive, concise representation of joint probability distribution (i.e., conditional dependencies) of a set of random variables.
• Represents “beliefs and knowledge” about a particular class of situations.
• Efficient (?) (approximate) inference algorithms
• Efficient, effective learning algorithms
Review of exact inference in Bayesian networks
General question: What is P(x|e)?
Example Question: What is P(c| r,w)?
General question: What is P(x|e)?
networks)Bayesian of (semantics ))(|(
) variablesevidence-non theare (where ),,(
factor)ion normalizat a is ( ),(
y)probabilit lconditiona ofn (definitio )(
),()|(
},,{
y ye
y
YYe
e
e
ee
ZparentszP
xP
xP
P
xPxP
xz
Event space
Event space
Cloudy
Event space
Cloudy
Rain
Event space
Cloudy
Sprinkler
Rain
Event space
Cloudy
Sprinkler
Wet Grass
Rain
Event space
Cloudy
Sprinkler
Wet Grass
Rain
)3636(.
)9.9.8.5(.99.1.8.5.
),|()|()|()(
))(|(
),,,(),|(
},,,{
s
s swrcz
s
rswPcsPcrPcP
ZparentszP
swrcPwrcPEvent space
Cloudy
Sprinkler
Wet Grass
Rain
)0945(.),|( wrcP
206.,794.
0945.3636.
0945.,
0945.3636.
3636.
0945.,3636.),|(
wrCP
• Draw expression tree for
• Worst-case complexity is exponential in n (number of nodes)
• Problem is having to enumerate all possibilities for many variables.
s
rswPcsPcrPcP ),|()|()|()(
Issues in Bayesian Networks
• Building / learning network topology
• Assigning / learning conditional probability tables
• Approximate inference via sampling
Real-World Example 1: The Lumière Project at Microsoft Research
• Bayesian network approach to answering user queries about Microsoft Office.
• “At the time we initiated our project in Bayesian information retrieval, managers in the Office division were finding that users were having difficulty finding assistance efficiently.”
• “As an example, users working with the Excel spreadsheet might have required assistance with formatting “a graph”. Unfortunately, Excel has no knowledge about the common term, “graph,” and only considered in its keyword indexing the term “chart”.
• Networks were developed by experts from user modeling studies.
• Offspring of project was Office Assistant in Office 97.
Real-World Example 2:Diagnosing liver disorders with Bayesian
networks• Variables: “disorder class” (16 possibilities) plus 93
features from existing database of patient records.
• Data: 600 patient records, which used those features
• Network structure: designed by “domain experts” (30 hours)
A. Onisko et al., A Bayesian network model for diagnosis of liver disorders
• Prior and conditional probability distributions were learned from data in liver-disorders database.
• Problem: Data doesn’t give enough samples for good conditional probability estimates.
• For combinations of parent values that are not adequately sampled, assume uniform distribution over those values.
Results
“number of observations” = number of evidence variables in query
window = n means that classification is counted as correct if it is in the n most probable diagnoses given by the network for the given evidence values.
Approximate inference in Bayesian networks
Instead of enumerating all possibilities, sample to estimate probabilities.
X1 X2 X3 Xn
...
Direct Sampling
• Suppose we have no evidence, but we want to determine P(c,s,r,w) for all c,s,r,w.
• Direct sampling:
– Sample each variable in topological order, conditioned on values of parents.
– I.e., always sample from P(Xi | parents(Xi))
1. Sample from P(Cloudy). Suppose returns true.
2. Sample from P(Sprinkler | Cloudy = true). Suppose returns false.
3. Sample from P(Rain | Cloudy = true). Suppose returns true.
4. Sample from P(WetGrass | Sprinkler = false, Rain = true). Suppose returns true.
Here is the sampled event: [true, false, true, true]
Example
• Suppose there are N total samples, and let NS (x1, ..., xn) be the observed frequency of the specific event x1, ..., xn.
• Suppose N samples, n nodes. Complexity O(Nn).
• Problem 1: Need lots of samples to get good probability estimates.
• Problem 2: Many samples are not realistic; low likelihood.
),...,(),...,(
lim 11
nnS
NxxP
N
xxN
),...,(),...,(
11
nnS xxP
N
xxN
Likelihood weighting
• Now suppose we have evidence e. Thus values for the evidence variables E are fixed.
• We want to estimate P(X | e)
• Need to sample X and Y, where Y is the set of non-evidence variables.
• Each event sampled is weighted by the likelihood that that event accords to the evidence.
• I.e., events in which the actual evidence appears unlikely should be given less weight.
Example:
Estimate P(Rain | Sprinkler = true, WetGrass = true).
WeightedSample algorithm:
1. Set weight w = 1.0
2. Sample from Cloudy. Suppose it returns true.
3. Sprinkler is an evidence variable with value true. Update likelihood weighting:
Low likelihood for sprinkler if cloudy is true, so this sample gets lower weight.
1.0)|( trueCloudytrueSprinklerPww
4. Sample from P(Rain | Cloudy = true). Suppose this returns true.
5. WetGrass is an evidence variable with value true. Update likelihood weighting:
6. Return event [true, true, true, true] with weight 0.099.
Weight is low because cloudy = true, so sprinkler is unlikely to be true.
099.0),|( trueRaintrueSprinklertrueWetGrassPww
Problem with likelihood sampling
• As number of evidence variables increases, performance degrades. This is because most samples will have very low weights, so weighted estimate will be dominated by fraction of samples that accord more than an infinitesimal likelihood to the evidence.
Markov Chain Monte Carlo Sampling
• One of most common methods used in real applications.
• Uses idea of “Markov blanket” of a variable Xi:
– parents, children, children’s parents
• Recall that: By construction of Bayesian network, a node is conditionaly independent of its non-descendants, given its parents.
• Proposition: A node Xi is conditionally independent of all other nodes in the network, given its Markov blanket.
– Example.
– Need to show that Xi is conditionally independent of nodes outside its Markov blanket.
– Need to show that Xi can be conditionally dependent on children’s parents.
A C
B
E
F
Example:
The proposition says: B is conditionally independent of F given A, C, E.
This can only be true if
P(B | A,C,E,F) = P(B | A, C, E)
Prove:
We know, by definition of conditional probability:
From tree we have:
),,|(),,,|( ECABPFECABP
),,,(
),,,,(),,,|(
FECAP
FECABPFECABP
),(),|()|()()(
)|(),(),|()()(),,,()2(
)|()|(),|()()(),,,,()1(
bEPCAbPEFPCPAP
EFPbEPCAbPCPAPFECAP
EFPBEPCABPCPAPFECABP
b
b
Thus:
b
b
bEPCAbP
BEPCABP
BEPCAbPEFPCPAP
EFPBEPCABPCPAPFECABP
)|(),|(
)|(),|(
)|(),|()|()()(
)|()|(),|()()(),,,|(
Now compute P(B | A, C, E):
Thus:
Q.E.D.
b
bEPCAbPCPAPECAP
BEPCABPCPAPECABP
ECAP
ECABPECABP
)|(),|()()(),,()2(
)|(),|()()(),,,()1(
),,(
),,,(),,|(
).,,,|(
),(),|(
)|(),|(),,|(
FECABP
bEPCAbP
BEPCABPECABP
b
Markov Chain Monte Carlo Sampling
• Start with random sample from variables: (x1, ..., xn). This is the current “state” of the algorithm.
• Next state: Randomly sample value for one non-evidence variable Xi , conditioned on current values in “Markov Blanket” of Xi.
Example
• Query: What is P(Rain | Sprinkler = true, WetGrass = true)?
• MCMC: – Random sample, with evidence variables fixed:
[true, true, false, true]
– Repeat:
1. Sample Cloudy, given current values of its Markov blanket: Sprinkler = true, Rain = false. Suppose result is false. New state:
[false, true, false, true]
2. Sample Rain, given current values of its Markov blanket:
Cloudy = false, Sprinkler = true, WetGrass = true. Suppose
result is true. New state: [false, true, true, true].
• Each sample contributes to estimate for query
P(Rain | Sprinkler = true, WetGrass = true)
• Suppose we perform 100 such samples, 20 with Rain = true and 80 with Rain = false.
• Then answer to the query is
Normalize (20,80) = .20,.80
• Claim: “The sampling process settles into a dynamic equilibrium in which the long-run fraction of time spent in each state is exactly proportional to its posterior probability, given the evidence.”
• Proof of claim is on pp. 517-518.
Claim (again)
• Claim: MCMC settles into behavior in which each state is sampled exactly according to its posterior probability, given the evidence.
• That is: for all variables Xi, the probability of the value xi of Xi appearing in a sample is equal to P(xi | e).
Proof of Claim (outline)
First, give example of Markov chain.
Now:
Let x be a state, with x = (x1, ..., xn).
Let q (x x) be the transition probability from state x to state x.
Let t(x) be the probability that the system will be in state x after t time steps, starting from state x0.
Let t+1(x’) be the probability that the system will be in state x after t+1 time steps, starting from state x0.
We have:
x
xxxx )'()()'(1 qtt
Definition:
Result from Markov chain theory: Given q, there is exactly one such stationary distribution (assuming q is “ergodic”).
is called the Markov process’s stationary distribution if t = t+1 for all x.
Defining equation for stationary distribution:
(1) )'()()'( xxxxxx
q
One way to satisfy equation 1 is:
Called property of detailed balance.
Detailed balance implies stationarity:
',)'()'()'()( xxxxxxxx qq
).'()'()'(
)'()'()'()(
xxxx
xxxxxxxx
x
q
Proof of claim: [Replace old version of this slide with this version]
Show that transition probability q (x x) defined by MCMC sampling satisfies detailed balance equation, with a stationary distribution equal to P(x | e).
Let Xi be the variable to be sampled. Let e be the values of the evidence variables and let Y be the other non-evidence variables.
Current sample: x = vector(xi, y), with fixed evidence variable values e.
We have, by definition of MCMC algorithm:
),|()),,(),,'(()'( eyeyeyxx iii xPxxqq ),|'()),,'(),,(()'( eyeyeyxx iii xPxxqq
Now, show this transition probability produces detailed balance.
We want to show:
)|()(then
)'()'()'()( If
exx
xxxxxx
P
...
)|()|,()(
have we),'(),|'( Since
3 and 2 terms
on rulechain backwardsby )|,(),|'(
),|()|(),|'(
)'()'()'()(
)'()'(
DEQ
ePxP
qxP
xPxP
xPPxP
i
i
q
ii
xeyx
xxey
eyey
eyeyey
xxxxxx
xxx
Speech Recognition(Section 15.6)
• Task: Identify sequence of words uttered by speaker, given acoustic signal.
• Uncertainty introduced by noise, speaker error, variation in pronunciation, homonyms, etc.
• Thus speech recognition is viewed as problem of probabilistic inference.
Speech Recognition
• So far, we’ve looked at probabilistic reasoning in static environments.
• Speech: Time sequence of “static environments”.
– Let X be the “state variables” (i.e., set of non-evidence variables) describing the environment (e.g., Words said during time step t)
– Let E be the set of evidence variables (e.g., S = features of acoustic signal).
– The E values and X joint probability distribution changes over time.
t1: X1, e1
t2: X2 , e2
etc.
• At each t, we want to compute P(Words | S).
• We know from Bayes rule:
• P(S | Words), for all words, is a previously learned “acoustic model”.
– E.g. For each word, probability distribution over phones, and for each phone, probability distribution over acoustic signals (which can vary in pitch, speed, volume).
• P(Words), for all words, is the “language model”, which specifies prior probability of each utterance.
– E.g. “bigram model”: probability of each word following each other word.
)()|()|( WordsPWordsPWordsP SS
• Speech recognition typically makes three assumptions:
1. Process underlying change is itself “stationary”
i.e., state transition probabilities don’t change
2. Current state X depends on only a finite history of previous states (“Markov assumption”).
– Markov process of order n: Current state depends only on n previous states.
3. Values et of evidence variables depend only on current state Xt. (“Sensor model”)
Example: “I’m firsty, um, can I have something to dwink?”