Probabilistic Reasoning

• Bayesian Belief Networks• Constructing Bayesian Networks• Representing Conditional Distributions• Summary

Bayesian Belief Networks (BBN)

A Bayesian Belief Network is a method to describe the joint probability distribution of a set of variables.

Let x1, x2, …, xn be a set of random variables. A Bayesian Belief Network or BBN will tell us the probability of any combination of x1, x2 , .., xn.

Representation

A BBN represents the joint probability distribution of a set of variables by explicitly indicating the assumptions of conditional independence through the following:

a) Nodes representing random variables b) Directed links representing relations.c) Conditional probability distributions.d) The graph is a directed acyclic graph.

Example 1

Weather Cavity

Toothache Catch

Example

Representation

Each variable is independent of its non-descendants given its predecessors. We say x1 is a descendant of x2 if there is a direct path from x2 to x1.

Example:

Predecessors of Alarm: Burglary, Earthquake.

Joint Probability Distribution

To compute the joint probability distribution of a set of variables given a Bayesian Belief Network we simply use the following formula:

P(x1,x2,…,xn) = Π P(xi | Parents(xi))

Where parents are the immediate predecessors of xi.

Joint Probability Distribution

Example:

P(John, Mary,Alarm,~Burglary,~Earthquake) :

P(John|Alarm) P(Mary|Alarm)P(Alarm|~Burglary ^ ~Earthquake)P(~Burglary) P(~Earthquake) = 0.00062

Conditional Probabilities

Alarm

Burglary Earthquake

B E P(A)t t 0.95 t f 0.94f t 0.29f f 0.001

Probabilistic Reasoning

• Bayesian Belief Networks• Constructing Bayesian Networks• Representing Conditional Distributions• Summary

Constructing Bayesian Networks

Choose the right order from causes to effects.

P(x1,x2,…,xn) = P(xn|xn-1,..,x1)P(xn-1,…,x1)

= Π P(xi|xi-1,…,x1) -- chain rule

Example: P(x1,x2,x3) = P(x1|x2,x3)P(x2|x3)P(x3)

How to construct BBNP(x1,x2,x3)

x3

x2

x1

root cause

leaf

Correct order: add root causes first, and then “leaves”, with no influence on other nodes.

Compactness

BBN are locally structured systems.They represent joint distributions compactly.

Assume n random variables, each influencedby k nodes. Size BBN: n2k Full size: 2n

Probabilistic Reasoning

• Bayesian Belief Networks• Constructing Bayesian Networks• Representing Conditional Distributions• Summary

Representing Conditional Distributions

Even if k is small O(2k) may be unmanageable.

Solution: use canonical distributions.

Example:

U.S.

CanadaMexico

North America simpledisjunction

Noisy-OR

Cold Flu Malaria

Fever

A link may be inhibited due to uncertainty

Noisy-OR

Inhibitions probabilities:

P(~fever | cold, ~flu, ~malaria) = 0.6 P(~fever | ~cold, flu, ~malaria) = 0.2 P(~fever | ~cold, ~flu, malaria) = 0.1

Noisy-OR

Now the whole probability can be built:

P(~fever | cold, ~flu, malaria) = 0.6 x 0.1 P(~fever | cold, flu, ~malaria) = 0.6 x 0.2P(~fever | ~cold, flu, malaria) = 0.2 x 0.1P(~fever | cold, flu, malaria) = 0.6 x 0.2 x 0.1

P(~fever | ~cold, ~flu, ~malaria) = 1.0

Continuous Variables

Continuous variables can be discretized.

Or define probability density functionsExample: Gaussian distribution.

A network with both variables is called a Hybrid Bayesian Network.

Continuous Variables

Subsidy Harvest

Cost

Buys

Continuous Variables

P(cost | harvest, subsidy)P(cost | harvest, ~subsidy)

Normal distribution

x

P(x)

Probabilistic Reasoning

• Bayesian Belief Networks• Constructing Bayesian Networks• Representing Conditional Distributions• Summary

Summary

• Bayesian networks are directed acyclic graphs that concisely represent conditional independence relations among random variables.• BBN specify the full joint probability distribution of a set of variables.• BBN can by hybrid, combining categorical variables with numeric variables.