Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing Conditional Distributions Summary

  • View
    216

  • Download
    3

Embed Size (px)

Text of Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing...

  • Slide 1

Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing Conditional Distributions Summary Slide 2 Bayesian Belief Networks (BBN) A Bayesian Belief Network is a method to describe the joint probability distribution of a set of variables. Let x1, x2, , xn be a set of random variables. A Bayesian Belief Network or BBN will tell us the probability of any combination of x1, x2,.., xn. Slide 3 Representation A BBN represents the joint probability distribution of a set of variables by explicitly indicating the assumptions of conditional independence through the following: a)Nodes representing random variables b)Directed links representing relations. c)Conditional probability distributions. d) The graph is a directed acyclic graph. Slide 4 Example 1 Weather Cavity Toothache Catch Slide 5 Example Slide 6 Representation Each variable is independent of its non-descendants given its predecessors. We say x1 is a descendant of x2 if there is a direct path from x2 to x1. Example: Predecessors of Alarm: Burglary, Earthquake. Slide 7 Joint Probability Distribution To compute the joint probability distribution of a set of variables given a Bayesian Belief Network we simply use the following formula: P(x1,x2,,xn) = P(xi | Parents(xi)) Where parents are the immediate predecessors of xi. Slide 8 Joint Probability Distribution Example: P(John, Mary,Alarm,~Burglary,~Earthquake) : P(John|Alarm) P(Mary|Alarm) P(Alarm|~Burglary ^ ~Earthquake) P(~Burglary) P(~Earthquake) = 0.00062 Slide 9 Conditional Probabilities Alarm Burglary Earthquake B E P(A) t t 0.95 t f 0.94 f t 0.29 f f 0.001 Slide 10 Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing Conditional Distributions Summary Slide 11 Constructing Bayesian Networks Choose the right order from causes to effects. P(x1,x2,,xn) = P(xn|xn-1,..,x1)P(xn-1,,x1) = P(xi|xi-1,,x1) -- chain rule Example: P(x1,x2,x3) = P(x1|x2,x3)P(x2|x3)P(x3) Slide 12 How to construct BBN P(x1,x2,x3) x3 x2 x1 root cause leaf Correct order: add root causes first, and then leaves, with no influence on other nodes. Slide 13 Compactness BBN are locally structured systems. They represent joint distributions compactly. Assume n random variables, each influenced by k nodes. Size BBN: n2 k Full size: 2 n Slide 14 Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing Conditional Distributions Summary Slide 15 Representing Conditional Distributions Even if k is small O(2 k ) may be unmanageable. Solution: use canonical distributions. Example: U.S. Canada Mexico North America simple disjunction Slide 16 Noisy-OR Cold Flu Malaria Fever A link may be inhibited due to uncertainty Slide 17 Noisy-OR Inhibitions probabilities: P(~fever | cold, ~flu, ~malaria) = 0.6 P(~fever | ~cold, flu, ~malaria) = 0.2 P(~fever | ~cold, ~flu, malaria) = 0.1 Slide 18 Noisy-OR Now the whole probability can be built: P(~fever | cold, ~flu, malaria) = 0.6 x 0.1 P(~fever | cold, flu, ~malaria) = 0.6 x 0.2 P(~fever | ~cold, flu, malaria) = 0.2 x 0.1 P(~fever | cold, flu, malaria) = 0.6 x 0.2 x 0.1 P(~fever | ~cold, ~flu, ~malaria) = 1.0 Slide 19 Continuous Variables Continuous variables can be discretized. Or define probability density functions Example: Gaussian distribution. A network with both variables is called a Hybrid Bayesian Network. Slide 20 Continuous Variables Subsidy Harvest Cost Buys Slide 21 Continuous Variables P(cost | harvest, subsidy) P(cost | harvest, ~subsidy) Normal distribution x P(x) Slide 22 Probabilistic Reasoning Bayesian Belief Networks Constructing Bayesian Networks Representing Conditional Distributions Summary Slide 23 Bayesian networks are directed acyclic graphs that concisely represent conditional independence relations among random variables. BBN specify the full joint probability distribution of a set of variables. BBN can by hybrid, combining categorical variables with numeric variables.