Chapter 2: CONDITIONAL PROBABILITY

Notes and figures are based on or taken from materials in the course textbook: Charles Boncelet,

Probability, Statistics, and Random Signals, Oxford University Press, February 2016.

B.J. Bazuin, Spring 2022 1 of 25 ECE 3800

Charles Boncelet, “Probability, Statistics, and Random Signals," Oxford University Press, 2016. ISBN: 978-0-19-020051-0

Chapter 2: CONDITIONAL PROBABILITY

Sections 2.1 Definitions of Conditional Probability 2.2 Law of Total Probability and Bayes Theorem 2.3 Example: Urn Models 2.4 Example: A Binary Channel 2.5 Example: Drug Testing 2.6 Example: A Diamond Network Summary Problems




Joint Probability – Compound Experiments

Defining probability based on multiple events … two classes for considerations.

Independent experiments: The outcome of one experiment is not affected by past or future experiments.

o flipping coins o repeating an experiment after initial conditions have been restored o Note: these problems are typically easier to solve

Dependent experiments: The result of each subsequent experiment is affected by

the results of previous experiments. o drawing cards from a deck of cards o drawing straws o selecting names from a hat o for each subsequent experiment, the previous results change the possible

outcomes for the next event. o Note: these problems can be very difficult to solve (the “next experiment”

changes based on previous outcomes!)




Conditional Probability

Defining the conditional probability of event A given that event B has occurred.

Using a Venn diagram, we know that B has occurred … then the probability that A has occurred given B must relate to the area of the intersection of A and B …

𝑃𝑟 𝐴 ∩ 𝐵 𝑃𝑟 𝐴|𝐵 ⋅ 𝑃𝑟 𝐵 , for 𝑃𝑟 𝐵 0

Therefore

𝑃𝑟 𝐴|𝐵∩

, for 𝑃𝑟 𝐵 0

For elementary events,

𝑃𝑟 𝐴|𝐵∩ ,

, for 𝑃𝑟 𝐵 0

Special cases for BA , AB , and AB .




Special cases for BA , AB , and AB .

If A is a subset of B, then the conditional probability must be

𝑃𝑟 𝐴|𝐵 ∩, for 𝐴 ⊂ 𝐵

Therefore, it can be said that

𝑃𝑟 𝐴|𝐵 ∩ 𝑃𝑟 𝐴 , for 𝐴 ⊂ 𝐵

If B is a subset of A, then the conditional probability becomes

𝑃𝑟 𝐴|𝐵 ∩ 1, for 𝐵 ⊂ 𝐴

If A and B are mutually exclusive,

𝑃𝑟 𝐴|𝐵 ∩ 0, for 𝐵 ∩ 𝐴 ∅

Conditional probabilities are generally not symmetric!

𝑃𝑟 𝐴 ∩ 𝐵 𝑃𝑟 𝐴|𝐵 ⋅ 𝑃𝑟 𝐵 , for 𝑃𝑟 𝐵 0

𝑃𝑟 𝐴 ∩ 𝐵 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴 , for 𝑃𝑟 𝐴 0

and

𝑃𝑟 𝐴|𝐵 ∩, for 𝑃𝑟 𝐵 0

𝑃𝑟 𝐵|𝐴 ∩, for 𝑃𝑟 𝐴 0

Then

𝑃𝑟 𝐵|𝐴𝑃𝑟 𝐴 ∩ 𝐵𝑃𝑟 𝐴

⋅𝑃𝑟 𝐵𝑃𝑟 𝐵

𝑃𝑟 𝐴 ∩ 𝐵𝑃𝑟 𝐵

⋅𝑃𝑟 𝐵𝑃𝑟 𝐴

𝑃𝑟 𝐴|𝐵 ⋅𝑃𝑟 𝐵𝑃𝑟 𝐴

Therefore, we would expect (unless Pr(A) and Pr(B) are equal)

𝑃𝑟 𝐵|𝐴 𝑃𝑟 𝐴|𝐵




Total Probability

For a space, S, that consists of multiple mutually exclusive events, the probability of a random event, B, occurring in space S, can be described based on the conditional probabilities associated with each of the possible events.

Proof:

𝑆 𝐴 ∪ 𝐴 ∪ 𝐴 ⋯∪ 𝐴

and

𝐴 ∩ 𝐴 ∅,𝑓𝑜𝑟𝑖 𝑗

𝐵 𝐵 ∩ 𝑆 𝐵 ∩ 𝐴 ∪ 𝐴 ∪ 𝐴 ⋯∪ 𝐴

𝐵 𝐵 ∩ 𝐴 ∪ 𝐵 ∩ 𝐴 ∪ 𝐵 ∩ 𝐴 ⋯∪ 𝐵 ∩ 𝐴

𝑃𝑟 𝐵 𝑃𝑟 𝐵 ∩ 𝐴 𝑃𝑟 𝐵 ∩ 𝐴 𝑃𝑟 𝐵 ∩ 𝐴 ⋯ 𝑃𝑟 𝐵 ∩ 𝐴

But

𝑃𝑟 𝐵 ∩ 𝐴 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴 , for 𝑃𝑟 𝐴 0

Therefore

𝑃𝑟 𝐵 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴 ⋯ 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴

Remember your math properties: distributive, associative, commutative etc. applied to set theory.




Continuing Concepts

Conditional Probability

When the probability of an event depends upon prior events. If trials are performed without replacement and/or the initial conditions are not restored, you expect trial outcomes to be dependent on prior results or conditions.

𝑃𝑟 𝐴|𝐵 𝑃𝑟 𝐴 when A follows B

The joint probability is.

𝑃𝑟 𝐴,𝐵 𝑃𝑟 𝐵,𝐴 𝑃𝑟 𝐴|𝐵 ⋅ 𝑃𝑟 𝐵 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴

Applicable for objects that have multiple attributes and/or for trials performed without replacement.

Experiment 3: A bag of marbles, draw 2 without replacement

Experiment: Draw two marbles, without replacement

Sample Space: {BB, BR, BY, RR, RB, RY, YB, YR}

Therefore

1st-rows \ 2nd-col

2nd-Blue 2nd-Red 2nd-Yellow Total

1st Marble

1st-Blue 30

6

5

2

6

3

30

6

5

2

6

3

30

3

5

1

6

3

6

3

1st-Red 30

6

5

3

6

2

30

2

5

1

6

2

30

2

5

1

6

2

6

2

1st-Yellow 30

3

5

3

6

1

30

2

5

2

6

1

30

0

5

0

6

1

6

1

Total 2nd Marble 6

3 6

2 6

1 6

6

Note the row and column sums!




Resistor Example: Joint and Conditional Probability

Assume we have a bunch of resistors (150) of various impedances and powers… Similar to old textbook problems (more realistic resistor values)

50 ohms 100 ohms 200 ohms Watt Subtotal

¼ watt 40 20 10 70

½ watt 30 20 5 55

1 watt 10 10 5 25

ohm Subtotal 80 50 20 150

Each object has two attributes: impedance (ohms) and power rating (watts)

Better Be Right Or Your Great Big Plan Goes Wrong. (p=purple for violet)

Bat Brained Resistor Order You Gotta Be Very Good With

Marginal Probabilities: (uses subtotals)

Pr(¼ watt) = 70/150 Pr(½ watt) = 55/150 Pr(1 watt) = 25/150

Pr(50 ohms) = 80/150 Pr(100 ohms) = 50/150 Pr(200 ohms) = 20/150

These are called the marginal probabilities … when fewer than all the attributes are considered (or don’t matter).




Joint Probabilities: divided each member of the table by 150!

50 ohms 100 ohms 200 ohms Subtotal

¼ watt 40/150=0.266 20/150=0.133 10/150=0.066 70/150=0.466

½ watt 30/150=0.20 20/150=0.133 5/150=0.033 55/150=0.366

1 watt 10/150=0.066 10/150=0.066 5/150=0.033 25/150=0.166

Subtotal 80/150=0.533 50/150=0.333 20/150=0.133 150/150=1.0

These are called the joint probabilities … when all unique attributes must be considered.

(Concept of total probability … things that sum to 1.0)

Conditional Probabilities:

When one attributes probability is determined based on the existence (or non-existence) of another attribute. Therefore,

The probability of a ¼ watt resistor given that the impedance is 50 ohm.

Pr(¼ watt given that the impedance is 50 ohms) = Pr(¼ watt | 50 ohms) = 40/80 = 0.50

50 ohms

¼ watt 40/80=0.50

½ watt 30/80=0.375

1 watt 10/80=0.125

Total 80/80=1.0

Simple math that does not work to find the solution: (they are not independent)

Pr(¼ watt) = 70/150 and Pr(50 ohms) = 80/150

Pr(¼ watt) x Pr(50 ohms) = 70/150 x 80/150 = 56/225 = 0.249 NO!!! Not independent!!

Math that does work

50.080

40

15080

15040

Pr

,Pr

Pr

Pr|Pr

B

BA

B

BABA




What about Pr(50 ohms given the power is ¼ watt)

50 ohms 100 ohms 200 ohms Total

¼ watt 40/70=0.571 20/70=0.286 10/70=0.143 70/70=1.0

Pr(50 ohms | ¼ watt) = Pr(50 | ¼) = 40/70 = 0.571

571.070

40

15070

15040

Pr

,Pr

Pr

Pr|Pr

B

BA

B

BABA

Can you determine?

Pr(100, ½) = Pr(100) =

Pr(50, ½) = Pr(½ | 50) =

Pr(50 | ½) = Pr( 1 ) =




Using the “table” it is rather straight forward … 50 ohms 100 ohms 200 ohms Subtotal

¼ watt 40 20 10 70 ½ watt 30 20 5 55 1 watt 10 10 5 25

Subtotal 80 50 20 150

Joint Probabilities 𝑃𝑟 𝐴 ∩ 𝐵 𝑃𝑟 𝐴,𝐵

Pr(100, ½) = Pr(50, ½) =

Conditional Probabilities 𝑃𝑟 𝐴|𝐵 ∩ ,

Pr(½ | 100) = Pr(200 | ½) =

Marginal Probability 𝑃𝑟 𝐵 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴 ⋯ 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴

Pr( 1 ) = Pr(100) =

Are there multiple ways to conceptually define such problems? … Yes Relative Frequency Approach (statistics) Set Theory Approach (formal math) Venn Diagrams (pictures based on set theory)

All ways to derive equations that form desired probabilities …. The Relative Frequency Approach is the slowest and requires the most work!




A Priori and A Posteriori Probability (Sec. 2.2 Bayes Theorem)

The probabilities defined for the expected outcomes, 𝑃𝑟 𝐴 , are referred to as a priori probabilities (before the event). They describe the probability before the actual experiment or experimental results are known.

After an event has occurred, the outcome B is known. The probability of the event belonging to one of the expected outcomes can be defined as

𝑃𝑟 𝐴 |𝐵

or from before

𝑃𝑟 𝐴 ∩ 𝐵 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴 𝑃𝑟 𝐴 |𝐵 ⋅ 𝑃𝑟 𝐵

𝑃𝑟 𝐴 |𝐵 | ⋅, for 𝑃𝑟 𝐵 0

Using the concept of total probability


We also have the following forms

𝑃𝑟 𝐴 |𝐵𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴

𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴 ⋯ 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴

or


∑ 𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴𝑃𝑟 𝐵|𝐴 ⋅ 𝑃𝑟 𝐴

𝑃𝑟 𝐵

This probability is referred to as the a-posteriori probability (after the event).

It is also referred to as Bayes Theorem.




Example

More Resistors

Bin 1 Bin 2 Bin 3 Bin 4 Bin 5 Bin 6 Subtotal

10 ohm 500 0 200 800 1200 1000 3700

100 ohm 300 400 600 200 800 0 2300

1000 ohm 200 600 200 600 0 1000 2600

Subtotal 1000 1000 1000 1600 2000 2000 8600

What is the probability of selecting a 10 ohm resistor from a random bin?

Given Bin marginal probability𝑃𝑟 𝐵𝑖𝑛#

𝑃𝑟 10𝛺|𝐵𝑖𝑛1 1000

02|10Pr Bin

1000

2003|10Pr Bin

1600

8004|10Pr Bin

2000

12005|10Pr Bin

2000

10006|10Pr Bin

nn AABAABAABB Pr|PrPr|PrPr|PrPr 2211

6

1

2000

1000

6

1

2000

1200

6

1

1600

800

6

1

1000

200

6

1

1000

0

6

1

1000

500Pr B

3833.06

1

10

23

6

1

10

5

6

1

10

6

6

1

10

5

6

1

10

2

6

1

10

0

6

1

10

5Pr B

Assuming a 10 ohm resistor is selected, what is the probability it came from bin 3?

nn

iii AABAABAAB

AABBA

Pr|PrPr|PrPr|Pr

Pr|Pr|Pr

2211

𝑃𝑟 𝐵𝑖𝑛3|10𝛺𝑃𝑟 10𝛺|𝐵𝑖𝑛3 ⋅ 𝑃𝑟 𝐵𝑖𝑛3

𝑃𝑟 10𝛺|𝐵𝑖𝑛1 ⋅ 𝑃𝑟 𝐵𝑖𝑛1 ⋯ 𝑃𝑟 10𝛺|𝐵𝑖𝑛6 ⋅ 𝑃𝑟 𝐵𝑖𝑛6

08696.03833.0

61

102

10|3Pr

Bin




DigitalTransmissions

A digital communication system sends a sequence of 0 and 1, each of which are received at the other end of a link. Assume that the probability that 0 is received correctly is 0.90 and that a 1 is received correctly is 0.90. Alternately, the probability that a 0 or 1 is not received correctly is 0.10 (the cross-over probability, ). Within the sequence, the probability that a 0 is sent is 60% and that a one is sent is 40%. [S is Send and R is Receive]. The A-priori probabilities are:

𝑃𝑟 𝑆 0.60 𝑃𝑟 𝑆 0.40

𝑃𝑟 𝑅 |𝑆 0.90 1 𝛽 190.0|Pr 11 SR

10.0|Pr 01 SR 10.0|Pr 10 SR

Figure 1.7-1

a) What is the probability that a zero is received?

Total Probability:


𝑃𝑟 𝑅 𝑃𝑟 𝑅 |𝑆 ⋅ 𝑃𝑟 𝑆 𝑃𝑟 𝑅 |𝑆 ⋅ 𝑃𝑟 𝑆

𝑃𝑟 𝑅 0.90 ⋅ 0.60 0.10 ⋅ 0.40

𝑃𝑟 𝑅 0.54 0.04 0.58

b) What is the probability that a one is received?

𝑃𝑟 𝑅 𝑃𝑟 𝑅 |𝑆 ⋅ 𝑃𝑟 𝑆 𝑃𝑟 𝑅 |𝑆 ⋅ 𝑃𝑟 𝑆

𝑃𝑟 𝑅 0.10 ⋅ 0.60 0.90 ⋅ 0.40

𝑃𝑟 𝑅 0.06 0.36 0.42




DigitalCommunications(continued)

𝑃𝑟 𝑆 0.60 𝑃𝑟 𝑆 0.40

𝑃𝑟 𝑅 |𝑆 0.90 𝑃𝑟 𝑅 |𝑆 0.90

𝑃𝑟 𝑅 |𝑆 0.10 𝑃𝑟 𝑅 |𝑆 0.10

c) A-posteriori: What is the probability that a received zero was transmitted as a 0?

Bayes Theorem



𝑃𝑟 𝑆 |𝑅𝑃𝑟 𝑅 |𝑆 ⋅ 𝑃𝑟 𝑆

𝑃𝑟 𝑅𝑃𝑟 𝑅 |𝑆 ⋅ 𝑃𝑟 𝑆

𝑃𝑟 𝑅 |𝑆 ⋅ 𝑃𝑟 𝑆 𝑃𝑟 𝑅 |𝑆 ⋅ 𝑃𝑟 𝑆


0.580.90 ⋅ 0.60

0.580.540.58

0.931

d) A-posteriori: What is the probability that a received one was transmitted as a 1?

Bayes Theorem







0.420.90 ⋅ 0.40

0.420.360.42

0.857





𝑃𝑟 𝑆 0.60 𝑃𝑟 𝑆 0.40

𝑃𝑟 𝑅 |𝑆 0.90 𝑃𝑟 𝑅 |𝑆 0.90

𝑃𝑟 𝑅 |𝑆 0.10 𝑃𝑟 𝑅 |𝑆 0.10

e) A-posteriori: What is the probability that a received zero was transmitted as a 1?

Bayes Theorem







0.54 0.040.10 ⋅ 0.40

0.580.040.58

0.069

Note: 𝑃𝑟 𝑆 |𝑅 1 𝑃𝑟 𝑆 |𝑅 1 0.931 0.069

f) A-posteriori: What is the probability that a received one was transmitted as a 0?

Bayes Theorem

nn

iii AABAABAAB

AABBA

Pr|PrPr|PrPr|Pr

Pr|Pr|Pr

2211

111001

001

1

00110 Pr|PrPr|Pr

Pr|Pr

Pr

Pr|Pr|Pr

SSRSSR

SSR

R

SSRRS


0.420.10 ⋅ 0.60

0.420.060.42

0.143

Note: 𝑃𝑟 𝑆 |𝑅 1 𝑃𝑟 𝑆 |𝑅 1 0.857 0.143





𝑃𝑟 𝑆 0.60 𝑃𝑟 𝑆 0.40

𝑃𝑟 𝑅 |𝑆 0.90 𝑃𝑟 𝑅 |𝑆 0.90

𝑃𝑟 𝑅 |𝑆 0.10 𝑃𝑟 𝑅 |𝑆 0.10

e) Verification: What is the probability that a symbol is received in error?

𝑃𝑟 𝐸𝑟𝑟𝑜𝑟 𝑃𝑟 𝑅 |𝑆 ⋅ 𝑃𝑟 𝑆 𝑃𝑟 𝑅 |𝑆 ⋅ 𝑃𝑟 𝑆

𝑃𝑟 𝐸𝑟𝑟𝑜𝑟 0.10 ⋅ 0.40 0.10 ⋅ 0.60 0.04 0.06 0.10

Alternately, using a-posteriori probabilities

𝑃𝑟 𝐸𝑟𝑟𝑜𝑟 𝑃𝑟 𝑆 |𝑅 ⋅ 𝑃𝑟 𝑅 𝑃𝑟 𝑆 |𝑅 ⋅ 𝑃𝑟 𝑅

𝑃𝑟 𝐸𝑟𝑟𝑜𝑟 0.143 ⋅ 0.42 0.069 ⋅ 0.58 0.060 0.040 0.100

Which way is easier?

Notice that you were told originally that there was a 0.10 chance of receiving a symbol in error! The computations must all be consistent!

Summary:

A-priori Probabilities

𝑃𝑟 𝑆 0.60 𝑃𝑟 𝑆 0.40

𝑃𝑟 𝑅 |𝑆 0.90 𝑃𝑟 𝑅 |𝑆 0.90

𝑃𝑟 𝑅 |𝑆 0.10 𝑃𝑟 𝑅 |𝑆 0.10

Computed Total Probability

𝑃𝑟 𝑅 0.58 𝑃𝑟 𝑅 0.42

Bayes Theorem (A-posteriori Probabilities)

𝑃𝑟 𝑆 |𝑅 0.931 𝑃𝑟 𝑆 |𝑅 0.857

𝑃𝑟 𝑆 |𝑅 0.069 𝑃𝑟 𝑆 |𝑅 0.143




Example1.7‐2:Amyloidtest:isitagoodtestforAlzheimer’s?(Stark&WoodsExample)

An amyloid test for Alzheimer’s disease had reported results/information for people 65 and older.

Alzheimer’s patients with disease = 90% had amyloid protein Alzheimer’s free patients = 36% had amyloid protein

General population facts for Alzheimer’s Total Alzheimer’s probability = 10% Total non-Alzheimer’s probability = 1-10% = 90%

The setup – a-priori probabilities (given)

𝑃𝑟 𝑎𝑚|𝐴𝑙𝑧 0.90 and 𝑃𝑟 𝑎𝑚|𝑛𝑜𝑛𝐴𝑙𝑧 0.36

𝑃𝑟 𝐴𝑙𝑧 0.10 and 𝑃𝑟 𝑛𝑜𝑛𝐴𝑙𝑧 0.90

What we want to know – if someone had the amyloid protein, what is the probability they have Alzheimer’s?

𝑃𝑟 𝐴𝑙𝑧|𝑎𝑚 ? ? ?

Using Bayes Theorem

𝑃𝑟 𝐴𝑙𝑧|𝑎𝑚𝑃𝑟 𝑎𝑚|𝐴𝑙𝑧 ⋅ 𝑃𝑟 𝐴𝑙𝑧

𝑃𝑟 𝑎𝑚

But we need to know 𝑃𝑟 𝑎𝑚 … determine the total probability of the amyloid protein

𝑃𝑟 𝑎𝑚 𝑃𝑟 𝑎𝑚|𝐴𝑙𝑧 ⋅ 𝑃𝑟 𝐴𝑙𝑧 𝑃𝑟 𝑎𝑚|𝑛𝑜𝑛𝐴𝑙𝑧 ⋅ 𝑃𝑟 𝑛𝑜𝑛𝐴𝑙𝑧

𝑃𝑟 𝑎𝑚 0.90 ⋅ 0.10 0.36 ⋅ 0.90 0.414

Therefore

𝑃𝑟 𝐴𝑙𝑧|𝑎𝑚0.90 ⋅ 0.10

0.4140.2174

The diagnosis is better than 10%, but for completeness … what about the non-Alzheimer’s population …

𝑃𝑟 𝑛𝑜𝑛𝐴𝑙𝑧|𝑎𝑚0.36 ⋅ 0.90

0.4140.7826

… too high a probability for a good test




TextbookUrnModels on p. 34-36

Textbook comment, p. 36: “Mathematicians have studied urn models for three centuries. Although they appear simplistic, urns and marbles can model numerous real experiments. Their study has led to many advances in our knowledge of probability.”

Assume two urns with different distribution of marbles …

Urn 1: 5 red, 5 blue Urn 2: 2 Red, 4 Blue

Select an urn with probability 𝑃𝑟 𝑈 and 𝑃𝑟 𝑈 1 𝑃𝑟 𝑈

What is the probability of selecting a Red marble?

Using total probability

𝑃𝑟 𝑅 𝑃𝑟 𝑅|𝑈 ∙ 𝑃𝑟 𝑈 𝑃𝑟 𝑅|𝑈 ∙ 𝑃𝑟 𝑈

𝑃𝑟 𝑅5

10∙

23

26∙

13

13

19

49

What is the probability of selecting two consecutive Red marbles from the same urn when (1) the urn is selected at random and (2) the first marble is red?

𝑃𝑟 𝑅 |𝑅𝑃𝑟 𝑅 ∩ 𝑅𝑃𝑟 𝑅

𝑃𝑟 𝑅 ∩ 𝑅 |𝑈 ∙ 𝑃𝑟 𝑈 𝑃𝑟 𝑅 ∩ 𝑅 |𝑈 ∙ 𝑃𝑟 𝑈𝑃𝑟 𝑅

To compute the elements we need to determine U1 and U2 based probabilities. First,

𝑃𝑟 𝑅 ∩ 𝑅 |𝑈𝑃𝑟 𝑅 ∩ 𝑅 ∩ 𝑈

𝑃𝑟 𝑈𝑃𝑟 𝑅 ∩ 𝑅 ∩ 𝑈𝑃𝑟 𝑅 ∩ 𝑈

∙𝑃𝑟 𝑅 ∩ 𝑈𝑃𝑟 𝑈

and continuing

𝑃𝑟 𝑅 ∩ 𝑅 |𝑈𝑃𝑟 𝑅 ∩ 𝑅 ∩ 𝑈𝑃𝑟 𝑅 ∩ 𝑈


𝑃𝑟 𝑅 |𝑅 ∩ 𝑈 ∙ 𝑃𝑟 𝑅 |𝑈

but

𝑃𝑟 𝑅 |𝑈5

10

𝑃𝑟 𝑅 |𝑅 ∩ 𝑈49




Therefore

𝑃𝑟 𝑅 ∩ 𝑅 |𝑈49

∙5

1029

Similarly,

𝑃𝑟 𝑅 ∩ 𝑅 |𝑈𝑃𝑟 𝑅 ∩ 𝑅 ∩ 𝑈

𝑃𝑟 𝑈𝑃𝑟 𝑅 ∩ 𝑅 ∩ 𝑈𝑃𝑟 𝑅 ∩ 𝑈


and

𝑃𝑟 𝑅 ∩ 𝑅 |𝑈𝑃𝑟 𝑅 ∩ 𝑅 ∩ 𝑈𝑃𝑟 𝑅 ∩ 𝑈


𝑃𝑟 𝑅 |𝑅 ∩ 𝑈 ∙ 𝑃𝑟 𝑅 |𝑈

With

𝑃𝑟 𝑅 |𝑈26

𝑃𝑟 𝑅 |𝑅 ∩ 𝑈15

𝑃𝑟 𝑅 ∩ 𝑅 |𝑈15

∙26

115

Finally


𝑃𝑟 𝑅 ∩ 𝑅 |𝑈 ∙ 𝑃𝑟 𝑈 𝑃𝑟 𝑅 ∩ 𝑅 |𝑈 ∙ 𝑃𝑟 𝑈𝑃𝑟 𝑅


29 ∙

23

115 ∙

13

49

427

145

49

4 ∙ 5 3135

49

23135

49


23135

49

2360

0.3833

Note that you also computed the probability of two consecutive red marbles from a random urn

𝑃𝑟 𝑅 ∩ 𝑅23

1350.1704

Be careful, the definition of this term changes in the next variation! In the next problem, the urn isn’t necessarily the same when drawing the second marble!




DigitalTransmissionsTextbookversion on p. 36-37

𝑃𝑟 𝑋 1 𝑝 𝑃𝑟 𝑋 𝑝

𝑃𝑟 𝑌 |𝑋 1 𝜀 𝑃𝑟 𝑌 |𝑋 1 𝜈

𝑃𝑟 𝑌 |𝑋 𝜀 𝑃𝑟 𝑌 |𝑋 𝜈

Determine total probability 𝑃𝑟 𝑌 and 𝑃𝑟 𝑌

𝑃𝑟 𝑌 𝑃𝑟 𝑌 |𝑋 ∙ 𝑃𝑟 𝑋 𝑃𝑟 𝑌 |𝑋 ∙ 𝑃𝑟 𝑋

𝑃𝑟 𝑌 1 𝜀 ∙ 1 𝑝 𝜈 ∙ 𝑝

𝑃𝑟 𝑌 𝑃𝑟 𝑌 |𝑋 ∙ 𝑃𝑟 𝑋 𝑃𝑟 𝑌 |𝑋 ∙ 𝑃𝑟 𝑋

𝑃𝑟 𝑌 𝜀 ∙ 1 𝑝 1 𝜈 ∙ 𝑝

Bayes Theorem (A-posteriori Probabilities)

𝑃𝑟 𝑋 |𝑌𝑃𝑟 𝑌 |𝑋 ∙ 𝑃𝑟 𝑋

𝑃𝑟 𝑌1 𝜀 ∙ 1 𝑝

1 𝜀 ∙ 1 𝑝 𝜈 ∙ 𝑝


𝑃𝑟 𝑌𝜈 ∙ 𝑝

1 𝜀 ∙ 1 𝑝 𝜈 ∙ 𝑝


𝑃𝑟 𝑌𝜀 ∙ 1 𝑝

𝜀 ∙ 1 𝑝 1 𝜈 ∙ 𝑝


𝑃𝑟 𝑌1 𝜈 ∙ 𝑝

𝜀 ∙ 1 𝑝 1 𝜈 ∙ 𝑝




Maximum a-posteriori estimation

If a Y=1 was received, decide that a X=1 was sent if

𝑃𝑟 𝑋 |𝑌 𝑃𝑟 𝑋 |𝑌

𝑃𝑟 𝑌 |𝑋 ∙ 𝑃𝑟 𝑋𝑃𝑟 𝑌


or

𝑃𝑟 𝑌 |𝑋 ∙ 𝑃𝑟 𝑋 𝑃𝑟 𝑌 |𝑋 ∙ 𝑃𝑟 𝑋

which lead to

1 𝜈 ∙ 𝑝𝜀 ∙ 1 𝑝 1 𝜈 ∙ 𝑝

𝜀 ∙ 1 𝑝𝜀 ∙ 1 𝑝 1 𝜈 ∙ 𝑝

1 𝜈 ∙ 𝑝 𝜀 ∙ 1 𝑝

or

𝑝1 𝑝

𝜀1 𝜈

Using the bit error probability from before, let 𝜀 𝜈 0.1

𝑝1 𝑝

0.11 0.1

19

0.111

So … you should believe what you see is what you get as long as the expected probability of a 1 is

𝑝 0.1

The probability that a one was sent is greater than 0.1 …




If a Y=0 was received, decide that a X=0 was sent if (Note that my derivation differs from a homework solution set answer for the same concept)

The maximum a-posteriori estimation condition that a 0 was set can be stated as

𝑃𝑟 𝑋 |𝑌 𝑃𝑟 𝑋 |𝑌



or

𝑃𝑟 𝑌 |𝑋 ∙ 𝑃𝑟 𝑋 𝑃𝑟 𝑌 |𝑋 ∙ 𝑃𝑟 𝑋

and

1 𝜀 ∙ 1 𝑝1 𝜀 ∙ 1 𝑝 𝜈 ∙ 𝑝

𝜈 ∙ 𝑝1 𝜀 ∙ 1 𝑝 𝜈 ∙ 𝑝

or

1 𝜀𝜈

𝑝1 𝑝

Using the bit error probability from before, let 𝜀 𝜈 0.1

91

1 𝜀𝜈

𝑝1 𝑝

So … you should believe what you see is what you get as long as the expected probability of a 1 is

0.9 𝑝 or 1 𝑝 0.1

Therefore, combining the two inequalities … we would hope that …

0.9 𝑝 0.1

Or if the bit error probabilities were not given

1 𝜀𝜈

𝑝1 𝑝

𝜀1 𝜈




2.6 Diamond Network

Figure 2.4 A Diamond Network

More Law of total Probability work (LTP)

The vertical link is a new consideration in our paths! By conditioning on the vertical link, we can derive two separate problems and then find a general solution.

If we consider Link3 to be broken and connected, we can establish the total probability

Total Probability: (based on link 3 connected or link 3 not connected)


𝑃𝑟 𝑆 → 𝐷 𝑃𝑟 𝑆 → 𝐷|𝐿 1 ∙ 𝑃𝑟 𝐿 1 𝑃𝑟 𝑆 → 𝐷|𝐿 0 ∙ 𝑃𝑟 𝐿 0

When Link 3 =1, the problem reduces to

Figure 2.5: A diamond network with Link 3 connected.

And 𝑃𝑟 𝑆 → 𝐷|𝐿 1 𝑃𝑟 𝐿 ∪ 𝐿 ∩ 𝐿 ∪ 𝐿

For equal probability, independent link closer (p)

𝑃𝑟 𝐿 ∪ 𝐿 ∩ 𝐿 ∪ 𝐿 𝑃𝑟 𝐿 ∪ 𝐿 ∙ 𝑃𝑟 𝐿 ∪ 𝐿 with

𝑃𝑟 𝐿 ∪ 𝐿 𝑃𝑟 𝐿 𝑃𝑟 𝐿 𝑃𝑟 𝐿 ∩ 𝐿 𝑝 𝑝 𝑝

𝑃𝑟 𝐿 ∪ 𝐿 𝑃𝑟 𝐿 𝑃𝑟 𝐿 𝑃𝑟 𝐿 ∩ 𝐿 𝑝 𝑝 𝑝 then

𝑃𝑟 𝐿 ∪ 𝐿 ∩ 𝐿 ∪ 𝐿 2 ∙ 𝑝 𝑝 ∙ 2 ∙ 𝑝 𝑝

𝑃𝑟 𝐿 ∪ 𝐿 ∩ 𝐿 ∪ 𝐿 4 ∙ 𝑝 4 ∙ 𝑝 𝑝

We next need the link open/failed which can be diagramed as




Figure 2.6: A diamond network with Link 3 not connected.

𝑃𝑟 𝑆 → 𝐷|𝐿 0 𝑃𝑟 𝐿 ∩ 𝐿 ∪ 𝐿 ∩ 𝐿

𝑃𝑟 𝑆 → 𝐷|𝐿 0 𝑃𝑟 𝐿 ∩ 𝐿 𝑃𝑟 𝐿 ∩ 𝐿 𝑃𝑟 𝐿 ∩ 𝐿 ∩ 𝐿 ∩ 𝐿

𝑃𝑟 𝑆 → 𝐷|𝐿 0 𝑝 𝑝 𝑝

The diamond network solution then becomes (total probability)

𝑃𝑟 𝑆 → 𝐷 𝑃𝑟 𝑆 → 𝐷|𝐿 1 ∙ 𝑃𝑟 𝐿 1 𝑃𝑟 𝑆 → 𝐷|𝐿 0 ∙ 𝑃𝑟 𝐿 0

𝑃𝑟 𝑆 → 𝐷 4 ∙ 𝑝 4 ∙ 𝑝 𝑝 ∙ 𝑝 2 ∙ 𝑝 𝑝 ∙ 1 𝑝

𝑃𝑟 𝑆 → 𝐷 4 ∙ 𝑝 4 ∙ 𝑝 𝑝 2 ∙ 𝑝 𝑝 2 ∙ 𝑝 𝑝

𝑃𝑟 𝑆 → 𝐷 2 ∙ 𝑝 2 ∙ 𝑝 5 ∙ 𝑝 2 ∙ 𝑝

To use some real numbers …

For p=0.9

𝑃𝑟 𝑆 → 𝐷 2 ∙ 0.81 2 ∙ 0.729 5 ∙ 0.6561 2 ∙ 0.59049

𝑃𝑟 𝑆 → 𝐷 0.97848

For p=0.5

𝑃𝑟 𝑆 → 𝐷 2 ∙ 0.25 2 ∙ 0.125 5 ∙ 0.0625 2 ∙ 0.03125

𝑃𝑟 𝑆 → 𝐷 0.5

Summary

Conditional probability captures the notion of partial information. Bayes Theorem is a widely discussed and used methodology in probability.




The diamond network solution Matlab plot

𝑃𝑟 𝑆 → 𝐷 2 ∙ 𝑝 2 ∙ 𝑝 5 ∙ 𝑝 2 ∙ 𝑝 %% % Diamond Network Probability % equally probable switch closure, p clear; close all; p=(0:0.01:1)'; Pr = 2*p.^2+2*p.^3-5*p.^4+2*p.^5; figure plot(p,Pr); ylabel('Probability') xlabel('Switch Prob. (p)') title('Diamond Network Connection') grid

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Pro

babi

lity

Switch Prob. (p)

Diamond Network Connection

Documents

Chapter 2: CONDITIONAL PROBABILITY