Probabilistic Reasoning With Bayes’ Rule

CSE 415 -- (c) S. Tanimoto, 2008 Bayes Nets

1

Probabilistic Reasoning With Bayes’ Rule

Outline:MotivationGeneralizing Modus PonensBayes’ RuleApplying Bayes’ RuleOddsOdds-Likelihood Formulation of Bayes’ RuleCombining independent items of evidenceGeneral combination of evidenceBenefits of Bayes nets for expert systems


2

MotivationLogical reasoning has limitations:

It requires that assumptions be considered “certain”.

It typically uses general rules. General rules that are reliable may be difficult to come by.

Logical reasoning can be awkward for certain structured domains such as time and space.


3

Generalizing Modus PonensModus Ponens: P -> Q P ---------- Q

Bayes’ Rule: (general idea)

If P then sometimes QP-------------------------------Maybe Q

(Bayes’ rule lets us calculate the probability of Q, taking P into account.)


4

Bayes’ Rule

E: Some evidence exists, i.e., a particular condition is trueH: some hypothesis is true.

P(E|H) = probability of E given H.P(E|~H) = probability of E given not H.P(H) = probability of H, independent of E.

P(E|H) P(H)P(H|E) = ----------------- P(E)

P(E) = P(E|H) P(H) + P(E|~H)(1 - P(H))


5

Applying Bayes’ RuleE: The patient’s white blood cell count exceeds 110% of average.

H: The patient is infected with tetanus.

P(E|H) = 0.8 class-conditional probabilityP(E|~H) = 0.3 “P(H) = 0.01 prior probability

posterior probability:

P(E|H) P(H) (0.8) (0.01) 0.008P(H|E) = ----------------- = -------------------------------- = ------- = 0.0262 P(E) (0.8) (0.01) + (0.3)(0.99) 0.305

P(E) = P(E|H) P(H) + P(E|~H)(1 - P(H))


6

OddsOdds are 10 to 1 it will rain tomorrow.

10 10P(rain) = ---------- = ------- 10 + 1 11

Suppose P(A) = 1/4Then O(A) = (1/4) / (3/4) = 1/3

P(A) P(A)in general: O(A) = -------- = ----------- P(~A) 1 - P(A)


7

Bayes’ Rule reformulated... P(E|H) P(H)P(H|E) = ----------------- P(E)_______ ______________

P(E|~H) P(~H)P(~H|E) = -------------------- P(E)

P(E|H)O(H|E) = ------------ O(H) P(E|~H)


8

Odds-LikelihoodForm of Bayes’ Rule

E: The patient’s white blood cell count exceeds 110% of average.


O(H) = 0.01/0.99

O(H|E) = λ O(H) lambda is called the sufficiency factor.

O(H|~E) = λ’ O(H) lambda prime is called the necessity factor.


9

The Monty Hall Problem

From the Wikipedia


10

The Monty Hall ProblemThere are three doors: a red door, green door, and blue door.Behind one is a car, and behind the other two are goats.You get to keep whatever is behind the door you choose.

You choose a door (say, red).

The host opens one of the other doors (say, green), which reveals a goat.

The host says, “Would you like to select the OTHER door?”

Should you switch?


11

DiscussionA: car is behind red doorB: car is behind green doorC: car is behind blue door

P(A) = P(B) = P(C) = 1/3

Suppose D: you choose the red door, and the host opens the green door revealing a goat.

Is P(C|D) = ½ ????

(No)

Why not? What is P(C|D)?


12

Bayes Nets

A practical way to manage probabilistic inference when multiple variables (perhaps many) are involved.


13

Why Bayes Networks?

Reasoning about events involving many parts or contingencies generally requires that a joint probability distribution be known. Such a distribution might require thousands of parameters. Modeling at this level of detail is typically not practical.

Bayes Nets require making assumptions about the relevance of some conditions to others. Once the assumptions are made, the joint distribution can be “factored” so that there are many fewer separate parameters that must be specified.


14

Review of Bayes’ Rule

E: Some evidence exists, i.e., a particular condition is trueH: some hypothesis is true.

P(E|H) = probability of E given H.P(E|~H) = probability of E given not H.P(H) = probability of H, independent of E.

P(E|H) P(H)P(H|E) = ----------------- P(E)

P(E) = P(E|H) P(H) + P(E|~H)(1 - P(H))


15

Combining Independent Items of Evidence

E1: The patient’s white blood cell count exceeds 110% of average.E2: The patient’s body temperature is above 101oF.


O(H) = 0.01/0.99

O(H|E1) = λ1 O(H) sufficiency factor for high white cell count.

O(H|E2) = λ2 O(H) sufficiency factor for high body temp.

Assuming E1 and E2 are independent:

O(H|E1 E2) = λ1 λ2 O(H)


16

Bayes Net Example

A: Accident (An accident blocked traffic on the highway.)B: Barb Late (Barbara is late for work).C: Chris Late (Christopher is late for work).

B C

A P(A) = 0.2

P(B|A) = 0.5

P(B|~A) = 0.15

P(C|A) = 0.3

P(C|~A) = 0.1


17

Forward Propagation(from causes to effects)

B C

A P(A) = 0.2

P(B|A) = 0.5

P(B|~A) = 0.15

P(C|A) = 0.3

P(C|~A) = 0.1

Suppose A (there is an accident): Then P(B|A) = 0.5 P(C|A) = 0.3Suppose ~A (no accident):Then P(B|~A) = 0.15 P(C|A) = 0.1(These come directly from the given information.)


18

Marginal Probabilities(using forward propagation)

B C

A P(A) = 0.2

P(B|A) = 0.5

P(B|~A) = 0.15

P(C|A) = 0.3

P(C|~A) = 0.1

Then P(B) = probability Barb is late in any situation = P(B|A) P(A) + P(B|~A) P(~A) = (0.5)(0.2) + (0.15)(0.8) = 0.22

Similarly P(C) = probability Chris is late in any situation = P(C|A) P(A) + P(C|~A) P(~A) = (0.3)(0.2) + (0.1)(0.8) = 0.14

Marginalizing means eliminating a contingency by summing the probabilities for its different cases (here A and ~A).


19

Backward Propagation: “diagnosis”(from effects to causes)

B C

A P(A) = 0.2

P(B|A) = 0.5

P(B|~A) = 0.15

P(C|A) = 0.3

P(C|~A) = 0.1

Suppose B (Barb is late)What’s the probability of an accident on the highway?Use Bayes’ rule:Then P(A|B) = P(B|A) P(A) / P(B) = 0.5 * 0.2 / (0.5 * 0.2 + 0.15 * 0.8)= 0.1 / 0.22= 0.4545


20

Revising Probabilities of Consequences

B C

A P(A|B) = 0.4545

P(B|A) = 0.5

P(B|~A) = 0.15

P(C|A) = 0.3

P(C|~A) = 0.1

P(C|B) = ???Suppose B (Barb is late).What’s the probability that Chris is also late, given this information?We already figured that P(A|B) = 0.4545P(C|B) = P(C|A) P(A|B) + P(C|~A) P(~A|B) = (0.3)(0.4545) + (0.1)(0.5455) = 0.191 somewhat higher than P(C)=0.14


21

Handling Multiple Causes

B C

A

P(B|A^D) = 0.9 P(B|A^~D) = 0.45P(B|~A^D) = 0.75P(B|~A^~D) = 0.1

D: Disease (Barb has the flu). P(D) = 0.111

(These values are consistent with P(B|A) = 0.5. )

D


22

Explaining Away

B C

A P(B|A^D) = 0.9 P(B|A^~D) = 0.45P(B|~A^D) = 0.75P(B|~A^~D) = 0.1

Suppose B (Barb is late). This raises the probability for each cause: P(A|B) = 0.4545,

P(D|B) = P(B|D) P(D)/ P(B) = 0.3935

Now, in addition, suppose C (Chris is late).C makes it more likely that A is true,“And this explains B.” D is now a less probable.

P(B|D) = P(B|A^D)P(A) + P(B|~A^D)P(~A) = 0.78

D


23

Benefits of Bayes Nets

The joint probability distribution normally requires 2n – 1 independent parameters.

With Bayes Nets we only specify these parameters:1. “root” node probabilities.

e. g., P(A=true) = 0.2; P(A=false)=0.8.

2. For each non-root node, a table of 2k values, where k is the number of parents of that node.Typically k < 5.

3. Propagating probabilities happens along the paths in the net. With a full joint prob. dist., many more computations may be needed.

Documents

Probabilistic Reasoning With Bayes’ Rule