Conditional Probability Total ProbabilityConditional ...berlin.csie.ntnu.edu.tw/Courses/Probability/2009Lectures/PROB2009… · Conditional Probability (1/2) • Conditional probability

Conditional Probability Total ProbabilityConditional Probability, Total Probability Theorem and Bayes’ Rule

Berlin ChenBerlin ChenDepartment of Computer Science & Information Engineering

National Taiwan Normal University

Reference:- D. P. Bertsekas, J. N. Tsitsiklis, Introduction to Probability , Sections 1.3-1.4

Conditional Probability (1/2)

• Conditional probability provides us with a way to reason about the outcome of an experiment based on partialabout the outcome of an experiment, based on partial information– Suppose that the outcome is within some given event , weBSuppose that the outcome is within some given event , we

wish to quantify the likelihood that the outcome also belongs some other given event

B

A

– Using a new probability law, we have the conditional probability of given , denoted by , which is defined as:BA ( )BAP( )

( ) ( )( )BBABA

PPP I=

A B

• If has zero probability, is undefined• We can think of as out of the total probability of the

( )BP( )BP ( )BAP

( )BAP

Probability-Berlin Chen 2

We can think of as out of the total probability of the elements of , the fraction that is assigned to possible outcomes that also belong to

( )BAPB

A

Conditional Probability (2/2)

• When all outcomes of the experiment are equally likely, p q y y,the conditional probability also can be defined as

( ) BAofelementsofnumber I( )BBABA

of elements ofnumber ofelementsofnumber I

=P

• Some examples having to do with conditional probability1. In an experiment involving two successive rolls of a die, you are told that

th f th t ll i 9 H lik l i it th t th fi t ll 6?the sum of the two rolls is 9. How likely is it that the first roll was a 6?2. In a word guessing game, the first letter of the word is a “t”. What is the

likelihood that the second letter is an “h”?3. How likely is it that a person has a disease given that a medical test was

negative?4. A spot shows up on a radar screen. How likely is it that it corresponds to


yan aircraft?

Conditional Probabilities Satisfy the Three Axioms

• Nonnegative:g

• Normalization:

( ) 0≥BAP• Normalization:

( ) ( )( )( )( ) 1==

Ω=Ω

BB

BBB

PP

PPP I

• Additivity: If and are two disjoint events( ) ( )BB PP

1A 2A

( ) ( )( )BAABAA 21PP IUU( ) ( )( )( )( ) ( )( )BABA

BBAA

21

2121

PP

P

=

=

IUI

IUU

distributive

A ( )( ) ( )

( )BABA

B

21

PPP+

=

=

II disjoint sets1A 2A


( )( ) ( )BABA

B

21

PPP+=

B

Conditional Probabilities Satisfy General Probability LawsProbability Laws

• Properties probability lawsp p y–

–

( ) ( ) ( )BABABAA 2121 PPP +≤U( ) ( ) ( ) ( )BAABABABAA 212121 IU PPPP −+=

– … ( ) ( ) ( ) ( )BAABABABAA 212121 IU PPPP +=

Conditional probabilities can also be viewed as a probabilityConditional probabilities can also be viewed as a probability law on a new universe , because all of the conditional probability is concentrated on .

BB


Simple Examples using Conditional Probabilities (1/3)


Simple Examples using Conditional Probabilities (2/3)g ( )


Simple Examples using Conditional Probabilities (3/3)

SF FFF

SS FSN

S F

S

C


Using Conditional Probability for Modeling (1/2)

• It is often natural and convenient to first specify p yconditional probabilities and then use them to determine unconditional probabilities

• An alternative way to represent the definition of conditional probability

( ) ( ) ( )BABBA PPP =I( ) ( ) ( )I


Using Conditional Probability for Modeling (2/2)

( )BAIP

( )( )cBAIP

( )BAc IP( )

( )cc BA IP


Multiplication (Chain) Rule

• Assuming that all of the conditioning events have g gpositive probability, we have

( ) ( ) ( ) ( ) ( )III 1−nn AAAAAAAAA PPPPPTh b f l b ifi d b iti

( ) ( ) ( ) ( ) ( )ILII 112131211 == = ni inni i AAAAAAAAA PPPPP– The above formula can be verified by writing

( ) ( ) ( )( )( )( )

( )( )I

LIII

I 13212111==ni in

i iAAAAAAAA PPPPP( ) ( ) ( ) ( ) ( )III 1121111 −== = ni ii i AAAAAA PPPPP

– For the case of just two events, the multiplication rule is simply the definition of conditional probability


( ) ( ) ( )12121 AAAAA PPP =I

Multiplication (Chain) Rule: Examples (1/2)

• Example 1.10. Three cards are drawn from an ordinary 52-card deck without replacement (drawn cards are not placed back in the deck). We wish to find the probability that none of the three cards is a “heart”.

( ) { } 1,2,3 ,heart anot is cardth the == iiAiP

( ) ( ) ( ) ( )213121321 = AAAAAAAAA III PPPP( ) ( ) ( ) ( )

5037

5138

5239 ⋅⋅=

?393C


505152 ?523C

Multiplication (Chain) Rule: Examples (2/2)

• Example 1.11. A class consisting of 4 graduate and 12 undergraduate students is randomly divided into 4 groups of 4 Whatundergraduate students is randomly divided into 4 groups of 4. What is the probability that each group includes a graduate student?

{ }diff tt2d1t d td tA { }{ }{ }diff4d321dd

groupsdifferent at are 3 and ,2, 1 students graduategroupsdifferent at are2and1studentsgraduate

2

1==

AAA

{ }groupsdifferent at are4 and3,,2,1studentsgraduate3 =A( ) ( ) ( ) ( ) ( )2131213213 == AAAAAAAAAA III PPPPP( )

( ) 81512

1 =

AA

A

P

P12

( )

( )134

14

213

12

=

=

AAA

AA

IP

P8

4


13

( )134

148

1512

3 ⋅⋅=∴ AP

4

Total Probability Theorem (1/2)

• Let be disjoint events that form a partition of the nAA ,,1 L j psample space and assume that , for all . Then, for any event , we have

n1i

B( ) 0>iAP

( ) ( ) ( )( ) ( )

n BABAB PPP ++= ILI1( ) ( ) ( ) ( )nn ABAABA PPPP ++= L11

– Note that each possible outcome of the experiment (sample space) is included in one and only one of the events nAA ,,1 L


Total Probability Theorem (2/2)

Figure 1.13:


Some Examples Using Total Probability Theorem (1/3)

Example 1.13.



Example 1.14.

(1,3),(1,4) (2,2),(2,3),(2,4) (4)



• Example 1.15. Alice is taking a probability class and at the end of each week she can be either up-to-date or she may have fallen p ybehind. If she is up-to-date in a given week, the probability that she will be up-to-date (or behind) in the next week is 0.8 (or 0.2, respectively) If she is behind in a given week the probability thatrespectively). If she is behind in a given week, the probability that she will be up-to-date (or behind) in the next week is 0.4 (or 0.6, respectively). Alice is (by default) up-to-date when she starts the class What is the probability that she is up to date after three weeks?class. What is the probability that she is up-to-date after three weeks?

( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( ) 4080

4080

111211212

222322323

BUBUPBUUPUU

.B.UBUPBUUPUU

⋅+⋅=+=

⋅+⋅=+=

PPPPP

PPPPPbehind:

date-to-up:

i

i

BU

( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( )( )0120806020

4080

011

111211212

111211212

UBU.B.UBBPBUBPBB

.B.UBUPBUUPUU

===

⋅+⋅=+=

⋅+⋅=+=

PPPPPPPP

PPPPP

know thatweAs Q

i

( ) ( ) ( )( )( )( ) 28060202080

72040208080012080

2

2

011

.....B.....U

.U.B,.U

=⋅+⋅==⋅+⋅==>

===

PP

PPP

know that we As Q

( ) ( ) ( )( ) ( ) ( ) 6020

4.08.0formuleaRecursion

1 ⋅+⋅=+BUBBUU iii

PPPPPP


( )( ) 688040280807203

2.....U =⋅+⋅=∴ P

( ) ( ) ( )( ) ( ) 2.0 ,8.0

6.02.0

11

1

==⋅+⋅=+

BUBUB iii

PPPPP

Bayes’ Rule

• Let be disjoint events that form a partition of nAAA ,,, 21 K j pthe sample space, and assume that , for all . Then, for any event such that we have

n21( ) 0≥iAP i

B ( ) 0>BP

( ) ( )( )i

i BBA

BAP

PP =

IMultiplication rule( )

( ) ( )( )

ii

BABA

PPP

= T t l b bilit th( )

( ) ( )( ) ( )

ii ABABPP

P

=

Total probability theorem

( ) ( )( ) ( )ii

nk kk

ABA

ABA

PP

PP∑ =1


( )( ) ( ) ( ) ( )nn ABAABA PPPP ++

=L11

Inference Using Bayes’ Rule (1/2)

惡性腫瘤

Figure 1.14:

良性腫瘤


Inference Using Bayes’ Rule (2/2)

• Example 1.18. The False-Positive Puzzle. – A test for a certain disease is assumed to be correct 95% of the time: if

a person has the disease, the test with are positive with probability 0.95 ( ), and if the person does not have the disease, the test ( ) 95.0=ABP

⎞⎛results are negative with probability 0.95 ( ). A random person drawn from a certain population has probability 0.001 ( ) of having the disease. Given that the person just tested

( )

( ) 001.0=AP

95.0=⎟⎠⎞⎜

⎝⎛ cc ABP

( )positive, what is the probability of having the disease ( ) ?

• : the event that the person has a diseaseh h h l i i

( )BAP

A• : the event that the test results are positiveB

( ) ( ) ( )( )= BABA

BAP

PPP

⎞⎛⎞⎛( )( ) ( )

( ) ( ) ( ) ( ) += cc ABAABAABA

B

PPPP

PPP

05.01 =⎟⎠⎞⎜

⎝⎛−=⎟

⎠⎞⎜

⎝⎛ ccc ABAB PP


0187.005.0999.095.0001.0

95.0001.0 =⋅+⋅

⋅=

Recitation

• SECTION 1.3 Conditional Probabilityy– Problems 11, 14, 15

• SECTION 1.4 Probability Theorem, Bayes’ Rule– Problems 17, 23, 24, 25


Homework -1

• Chapter 1 : Additional Problemsp(http://www.athenasc.com/CH1-prob-supp.pdf)

– Problems 2, 8, 13, 18, 24


Documents

Conditional Probability Total ProbabilityConditional ...berlin.csie.ntnu.edu.tw/Courses/Probability/2009Lectures/PROB2009… · Conditional Probability (1/2) • Conditional probability