Transcript
Page 1: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Conditional Probability

Page 2: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Recall: S = all possible outcomes. E = the event.

Axioms of Probability

• Axiom 1: 0 £ P(E) £ 1

• Axiom 2: P(S) = 1

• Axiom 3: P(Ec) = 1 – P(E)

Aside: axiom 3 is often stated as the probability of mutually exclusive events. We’ll come back to that later in the lecture…

Page 3: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Recall: S = all possible outcomes. E = the event.

Axioms of Probability

• Axiom 1: 0 £ P(E) £ 1

• Axiom 2: P(S) = 1

• Axiom 3: If events E and F are mutually exclusive:

P (E [ F ) = P (E) + P (F )

Page 4: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Mutually Exclusive Events

P (E [ F ) = P (E) + P (F )

If events are mutually exclusive, probability of OR is simple:

7/50 4/50

Page 5: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

P (E [ F ) = P (E) + P (F )

If events are mutually exclusive, probability of OR is simple:

7/50 4/50

P (E [ F ) =7

50+

4

5=

11

50

7/50 4/50

Mutually Exclusive Events

Page 6: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

X1

P (X1 [X2 [ · · · [Xn) =nX

i=1

P (Xi)

X2

X3

OR with Many Mutually Exclusive EventsWahoo! All my

events are mutually

exclusive

Page 7: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

If events are mutually exclusive probability of

OR is easy!

Page 8: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

P(Ec) = 1 – P(E)?

P(E ∪ Ec) = P(E) + P(Ec) Since E and Ec are mutually exclusive

P(S) = P(E) + P(Ec) Since everything must either be in Eor Ec

1 = P(E) + P(Ec)

P(Ec) = 1 – P(E)

Axiom 2

Rearrange

Page 9: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Why study probability?

Page 10: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• Roll two 6-sided dice, yielding values D1 and D2

• Let E be event: D1 + D2 = 4• What is P(E)?

§ |S| = 36, E = {(1, 3), (2, 2), (3, 1)}§ P(E) = 3/36 = 1/12

• Let F be event: D1 = 2• P(E, given F already observed)?

§ S = {(2, 1), (2, 2), (2, 3), (2, 4), (2, 5), (2, 6)}§ E = {(2, 2)}§ P(E, given F already observed) = 1/6

Dice – Our Misunderstood Friends

Page 11: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Dice – Our Misunderstood Friends

• Two people each roll a die, yielding D1 and D2. You win if D1 + D2 = 4

• Q: What do you think is the best outcome for D1 ?

• Your Choices:§ A. 1 and 3 tie for best§ B. 1, 2 and 3 tie for best§ C. 2 is the best§ D. Other/none/more than one

Page 12: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• Conditional probability is probability that E occurs given that F has already occurred “Conditioning on F”

• Written as § Means “P(E, given F already observed)”§ Sample space, S, reduced to those elements

consistent with F (i.e. S Ç F)§ Event space, E, reduced to those elements

consistent with F (i.e. E Ç F)

Conditional Probability

P (E|F )

Page 13: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

With equally likely outcomes:

P(E | F) = =

# of outcomes in E consistent with F

# of outcomes in S consistent with F

||||

||||

FEF

SFEF

=

Conditional Probability

P (E|F ) =3

14⇡ 0.21

P (E) =8

50⇡ 0.16

Page 14: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

With equally likely outcomes:

P(E | F) = =

# of outcomes in E consistent with F

# of outcomes in S consistent with F

||||

||||

FEF

SFEF

=

Conditional Probability

P (E|F ) =3

14⇡ 0.21

P (E) =8

50⇡ 0.16

Shorthand notation for set intersection

(aka set “and”)

Page 15: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• General definition:

• Holds even when outcomes are not equally likely• Implies: P(EF) = P(E | F) P(F) (chain rule)

• What if P(F) = 0?§ P(E | F) undefined§ Congratulations! You observed the impossible!

)()()|(

FPEFPFEP =

Conditional Probability

Page 16: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• General definition of Chain Rule:)...( 321 nEEEEP

)...|()...|()|()( 121213121 -= nn EEEEPEEEPEEPEP

Generalized Chain Rule

Page 17: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Conditional Paradigm

Page 18: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

+ Learn

Page 19: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Netflix and Learn

P(E)

S = {Watch, Not Watch}

E = {Watch}

P(E) = ½ ?

What is the probability that a user will watch

Life is Beautiful?

Page 20: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

What is the probability that a user will watch

Life is Beautiful?

P(E)

Netflix and Learn

Page 21: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

What is the probability that a user will watch

Life is Beautiful?

P(E)

Netflix and Learn

P(E) = 10,234,231 / 50,923,123 = 0.20

P (E) = limn!1

n(E)

n⇡ #people who watched movie

#people on Netflix

Page 22: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Netflix and Learn

Let E be the event that a user watched the given movie:

P(E) =0.19

P(E) =0.32

P(E) =0.20

P(E) =0.09

P(E) =0.23

* These are the actual estimates

Page 23: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Netflix and Learn

What is the probability that a user will watchLife is Beautiful, given they watched Amelie?

P(E|F)

P (E|F ) =P (EF )

P (F )=

#people who watched both#people on Netflix

#people who watched F#people on Netflix

P (E|F ) =P (EF )

P (F )=

#people who watched both#people on Netflix

#people who watched F#people on Netflix

Page 24: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Netflix and Learn

P(E|F)

What is the probability that a user will watchLife is Beautiful, given they watched Amelie?

P (E|F ) =P (EF )

P (F )=

#people who watched both#people on Netflix

#people who watched F#people on Netflix

Page 25: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Netflix and Learn

P(E|F)

P(E|F) = 0.42

What is the probability that a user will watchLife is Beautiful, given they watched Amelie?

P (E|F ) =P (EF )

P (F )=

#people who watched both#people on Netflix

#people who watched F#people on Netflix

#people who watched both

#people who watched F

Page 26: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Netflix and Learn

Let E be the event that a user watched the given movie,Let F be the event that the same user watched Amelie:

P(E|F) =0.14

P(E|F) =0.35

P(E|F) =0.20

P(E|F) =0.72

P(E|F) =0.49

Page 27: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Machine Learning

Machine Learning is:Probability + Data + Computers

Page 28: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• There are 400 students in CS109:– Probability that a random student in CS109 is a Sophomore

is 0.43– We can observe the probability that a student is both a

Sophomore and is in class– What is the conditional probability of a student coming to

class given that they are a Sophomore?• Solution:

– S is the event that a student is a sophomore– A is the event that a student is in class

Sophomores

P (A|S) = P (SA)

P (S)

Page 29: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Page 30: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• Rev. Thomas Bayes (1702 –1761) was a British mathematician and Presbyterian minister

• He looked remarkably similar to Charlie Sheen§ But that’s not important right now...

Thomas Bayes

Bi-Winning!

Page 31: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

But First!

Page 32: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• Say E and F are events in S

E F

S

E = EF È EFc

Note: EF Ç EFc = Æ

So, P(E) = P(EF) + P(EFc)

Background Observation

Page 33: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Law of Total Probability

P (E) = P (EF ) + P (EFC)

= P (E|F )P (F ) + P (E|FC)P (FC)

F FC

Sample Space

E

Page 34: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Law of Total Probability

B1B2

Sample Space

EB3 B4

P (E) =X

i

P (Bi \ E)

=X

i

P (E|Bi)P (Bi)

Page 35: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Moment of Silence…

Page 36: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

I want to calculate P(State of the world F | Observation E)

It seems so tricky!…

The other way around is easyP(Observation E | State of the world F)

What options to I have, chief?

P( F | E )

P( E | F )

Bayes Theorem

Page 37: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Bayes Theorem Want P( F | E ). Know P( E | F )

P (F |E) =P (EF )

P (E)Def. of Conditional Prob.

=P (E|F )P (F )

P (E)Chain Rule

<latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit>

P (F |E) =P (EF )

P (E)Def. of Conditional Prob.

=P (E|F )P (F )

P (E)Chain Rule

<latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit>

P (F |E) =P (EF )

P (E)Def. of Conditional Prob.

=P (E|F )P (F )

P (E)Chain Rule

<latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit>

P (F |E) =P (EF )

P (E)Def. of Conditional Prob.

=P (E|F )P (F )

P (E)Chain Rule

<latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit><latexit sha1_base64="IvilxdU97DewTmP2DDnG60i4sKo=">AAACXXicbVFBT9swFHayMaBjrMCBAxdr1ar2EiXTpHFBQhQqjtm0AlJTVY77Qi0cO7JfplUhf3I3uOyvzCmV2OieZPnT9973PflzWkhhMQwfPP/V6403m1vbrbc773bft/f2r6wuDYcR11Kbm5RZkELBCAVKuCkMsDyVcJ3eDZr+9Q8wVmj1HRcFTHJ2q0QmOENHTdsY94b3F33aPaFJZhiv4t7FsF83V7+m3S5NEH5idQ5ZQHVGB1rNRKNkksZGp0GdJC2nfRbfD/vOct1hMGdC0W+lhLo1bXfCIFwWXQfRCnTIquJp+1cy07zMQSGXzNpxFBY4qZhBwRvDpLRQMH7HbmHsoGI52Em1TKemHx0zo5k27iikS/ZvRcVyaxd56iZzhnP7steQ/+uNS8yOJ5VQRYmg+NOirJQUNW2ipjNhgKNcOMC4cbFxyufM5YTuQ5oQopdPXgdXn4IoDKKvnzunZ6s4tsgR+UB6JCJfyCm5JDEZEU4ePeJtey3vt7/h7/i7T6O+t9IckH/KP/wDIT6u1w==</latexit>

A little while later…

Page 38: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• Most common form:

• Expanded form:

Bayes Theorem

P (F |E) =P (E|F )P (F )

P (E|F )P (F ) + P (E|FC)P (FC)

P (F |E) =P (E|F )P (F )

P (E)<latexit sha1_base64="cvILkzdW4PJWAhOKDO2FSpw3y+Q=">AAACDnicbZDLSsNAFIYnXmu9RV26GSyVZlMSEXQjFKXFZQR7gTaUyXTSDp1cmJkIJc0TuPFV3LhQxK1rd76NkzYLbf1h4OM/53Dm/G7EqJCm+a2trK6tb2wWtorbO7t7+/rBYUuEMcekiUMW8o6LBGE0IE1JJSOdiBPku4y03fFNVm8/EC5oGNzLSUQcHw0D6lGMpLL6etmuNKZ1A55eQQh7Hkc4sSv1acNQvpFmbKSw2NdLZtWcCS6DlUMJ5LL7+ldvEOLYJ4HEDAnRtcxIOgnikmJG0mIvFiRCeIyGpKswQD4RTjI7J4Vl5QygF3L1Agln7u+JBPlCTHxXdfpIjsRiLTP/q3Vj6V06CQ2iWJIAzxd5MYMyhFk2cEA5wZJNFCDMqforxCOkMpEqwSwEa/HkZWidVS2zat2dl2rXeRwFcAxOQAVY4ALUwC2wQRNg8AiewSt40560F+1d+5i3rmj5zBH4I+3zB9iwmC8=</latexit><latexit sha1_base64="cvILkzdW4PJWAhOKDO2FSpw3y+Q=">AAACDnicbZDLSsNAFIYnXmu9RV26GSyVZlMSEXQjFKXFZQR7gTaUyXTSDp1cmJkIJc0TuPFV3LhQxK1rd76NkzYLbf1h4OM/53Dm/G7EqJCm+a2trK6tb2wWtorbO7t7+/rBYUuEMcekiUMW8o6LBGE0IE1JJSOdiBPku4y03fFNVm8/EC5oGNzLSUQcHw0D6lGMpLL6etmuNKZ1A55eQQh7Hkc4sSv1acNQvpFmbKSw2NdLZtWcCS6DlUMJ5LL7+ldvEOLYJ4HEDAnRtcxIOgnikmJG0mIvFiRCeIyGpKswQD4RTjI7J4Vl5QygF3L1Agln7u+JBPlCTHxXdfpIjsRiLTP/q3Vj6V06CQ2iWJIAzxd5MYMyhFk2cEA5wZJNFCDMqforxCOkMpEqwSwEa/HkZWidVS2zat2dl2rXeRwFcAxOQAVY4ALUwC2wQRNg8AiewSt40560F+1d+5i3rmj5zBH4I+3zB9iwmC8=</latexit><latexit sha1_base64="cvILkzdW4PJWAhOKDO2FSpw3y+Q=">AAACDnicbZDLSsNAFIYnXmu9RV26GSyVZlMSEXQjFKXFZQR7gTaUyXTSDp1cmJkIJc0TuPFV3LhQxK1rd76NkzYLbf1h4OM/53Dm/G7EqJCm+a2trK6tb2wWtorbO7t7+/rBYUuEMcekiUMW8o6LBGE0IE1JJSOdiBPku4y03fFNVm8/EC5oGNzLSUQcHw0D6lGMpLL6etmuNKZ1A55eQQh7Hkc4sSv1acNQvpFmbKSw2NdLZtWcCS6DlUMJ5LL7+ldvEOLYJ4HEDAnRtcxIOgnikmJG0mIvFiRCeIyGpKswQD4RTjI7J4Vl5QygF3L1Agln7u+JBPlCTHxXdfpIjsRiLTP/q3Vj6V06CQ2iWJIAzxd5MYMyhFk2cEA5wZJNFCDMqforxCOkMpEqwSwEa/HkZWidVS2zat2dl2rXeRwFcAxOQAVY4ALUwC2wQRNg8AiewSt40560F+1d+5i3rmj5zBH4I+3zB9iwmC8=</latexit><latexit sha1_base64="cvILkzdW4PJWAhOKDO2FSpw3y+Q=">AAACDnicbZDLSsNAFIYnXmu9RV26GSyVZlMSEXQjFKXFZQR7gTaUyXTSDp1cmJkIJc0TuPFV3LhQxK1rd76NkzYLbf1h4OM/53Dm/G7EqJCm+a2trK6tb2wWtorbO7t7+/rBYUuEMcekiUMW8o6LBGE0IE1JJSOdiBPku4y03fFNVm8/EC5oGNzLSUQcHw0D6lGMpLL6etmuNKZ1A55eQQh7Hkc4sSv1acNQvpFmbKSw2NdLZtWcCS6DlUMJ5LL7+ldvEOLYJ4HEDAnRtcxIOgnikmJG0mIvFiRCeIyGpKswQD4RTjI7J4Vl5QygF3L1Agln7u+JBPlCTHxXdfpIjsRiLTP/q3Vj6V06CQ2iWJIAzxd5MYMyhFk2cEA5wZJNFCDMqforxCOkMpEqwSwEa/HkZWidVS2zat2dl2rXeRwFcAxOQAVY4ALUwC2wQRNg8AiewSt40560F+1d+5i3rmj5zBH4I+3zB9iwmC8=</latexit>

Page 39: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• A test is 98% effective at detecting H1N1§ However, test has a “false positive” rate of 1%§ 0.5% of US population has H1N1§ Let E = you test positive for H1N1 with this test§ Let F = you actually have H1N1§ What is P(F | E)?

• Solution:

P(E | F) P(F) + P(E | Fc) P(Fc)P(F | E) =

P(E | F) P(F)

(0.98)(0.005) + (0.01)(1 - 0.005)P(F | E) =

(0.98)(0.005)» 0.330

H1N1 Testing

Page 40: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Intuition Time

Page 41: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Bayes Theorem Intuition

All People

Page 42: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Bayes Theorem Intuition

All People

People with H1N1

Page 43: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Bayes Theorem Intuition

All People

People who test positive

Page 44: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Bayes Theorem Intuition

All People

People with H1N1

People who test positive

Page 45: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Bayes Theorem Intuition

Conditioning on a positive result changes the sample space to this:

» 0.330

People who test positive

People who test positive and have H1N1

Page 46: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Bayes Theorem Intuition

Conditioning on a positive result changes the sample space to this:

» 0.330

People who test positive

P(F)P(E|F)

P(F)P(E|F) + P(Fc)P(E|Fc)

People who test positive and have H1N1

Page 47: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Bayes Theorem Intuition

All People

People with positive test

People with H1N1

Page 48: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Bayes Theorem IntuitionSay we have 1000 people:

5 have H1N1 and test positive, 985 do not have H1N1 and test negative.10 do not have H1N1 and test positive. » 0.333

Page 49: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

§ Let Ec = you test negative for H1N1 with this test§ Let F = you actually have H1N1§ What is P(F | Ec)?

P(Ec | F) P(F) + P(Ec | Fc) P(Fc)P(F | Ec) = P(Ec | F) P(F)

(0.02)(0.005) + (0.99)(1 - 0.005)P(F | Ec) = (0.02)(0.005)

» 0.0001

H1N1 + H1N1 –Test + 0.98 = P(E | F) 0.01 = P(E | Fc)Test – 0.02 = P(Ec | F) 0.99 = P(Ec | Fc)

Why It’s Still Good to get Tested

Page 50: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Slicing Up Spam

In 2010 88% of email was spam

Page 51: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• Say 60% of all email is spam§ 90% of spam has a forged header§ 20% of non-spam has a forged header§ Let E = message contains a forged header§ Let F = message is spam§ What is P(F | E)?

• Solution:P(E | F) P(F) + P(E | Fc) P(Fc)

P(F | E) =P(E | F) P(F)

(0.9)(0.6) + (0.2)(0.4)P(F | E) =

(0.9)(0.6)» 0.871

Simple Spam Detection

Page 52: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Update Belief

Before Observation

P (L1) P (L2)

P (L5)

Page 53: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Update Belief

Before Observation After Observation

P (L5)

Know:

Page 54: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Update Belief

Before Observation After Observation

P (L5)

P (L5|O) =P (O|L5)P (L5)

P (O)

P (L5|O)

Know:

Page 55: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

P (L5|O) =P (O|L5)P (L5)Pi P (O|Li)P (Li)

<latexit sha1_base64="xt2+x1cuyZ83goqxUqyrBW8XG3A=">AAACJXicbVDLSgMxFL1TX7W+qi7dBIvQ2ZQZQdqFQsGNi0Ir2Ae0ZcikmTY28yDJCGXan3Hjr3TjwiKCK3/F9LHQ6oHA4ZxzubnHjTiTyrI+jdTG5tb2Tno3s7d/cHiUPT5pyDAWhNZJyEPRcrGknAW0rpjitBUJin2X06Y7vJ37zScqJAuDBzWKaNfH/YB5jGClJSd7XctXnCs0RlUT3aCOJzBJavnqWIvmwjInSUfGvsOQlnWu4jATzR1mTpxszipYC6C/xF6RXLk4fSwDQM3Jzjq9kMQ+DRThWMq2bUWqm2ChGOF0kunEkkaYDHGftjUNsE9lN1lcOUEXWukhLxT6BQot1J8TCfalHPmuTvpYDeS6Nxf/89qx8krdhAVRrGhAlou8mCMVonllqMcEJYqPNMFEMP1XRAZYN6V0sRldgr1+8l/SuCzYVsG+122UYIk0nME55MGGIpThDmpQBwLPMIU3mBkvxqvxbnwsoyljNXMKv2B8fQP+pKLK</latexit><latexit sha1_base64="Kcpj61YCglpnx2qAeijVh+OqHV8=">AAACJXicbVDLSgMxFM3UV62vqks3wSJ0NmVGkBZUKLhxUegI9gFtGTJppo3NPEgyQpnOv4gbf6UbFxYRXIg/4sK0daGtBwKHc87l5h4nZFRIw3jXUiura+sb6c3M1vbO7l52/6AugohjUsMBC3jTQYIw6pOapJKRZsgJ8hxGGs7gauo37gkXNPBv5TAkHQ/1fOpSjKSS7OyFla/YZ3AEqzq8hG2XIxxb+epIifrM0pO4LSLPplDJKlexqQ6nDtUTO5szCsYMcJmYPyRXLo7vzj++Hiw7O2l3Axx5xJeYISFaphHKToy4pJiRJNOOBAkRHqAeaSnqI4+ITjy7MoEnSulCN+Dq+RLO1N8TMfKEGHqOSnpI9sWiNxX/81qRdEudmPphJImP54vciEEZwGllsEs5wZINFUGYU/VXiPtINSVVsRlVgrl48jKpnxZMo2DeqDZKYI40OALHIA9MUARlcA0sUAMYPIIxeAET7Ul71l61t3k0pf3MHII/0D6/AUYLpT4=</latexit><latexit sha1_base64="Kcpj61YCglpnx2qAeijVh+OqHV8=">AAACJXicbVDLSgMxFM3UV62vqks3wSJ0NmVGkBZUKLhxUegI9gFtGTJppo3NPEgyQpnOv4gbf6UbFxYRXIg/4sK0daGtBwKHc87l5h4nZFRIw3jXUiura+sb6c3M1vbO7l52/6AugohjUsMBC3jTQYIw6pOapJKRZsgJ8hxGGs7gauo37gkXNPBv5TAkHQ/1fOpSjKSS7OyFla/YZ3AEqzq8hG2XIxxb+epIifrM0pO4LSLPplDJKlexqQ6nDtUTO5szCsYMcJmYPyRXLo7vzj++Hiw7O2l3Axx5xJeYISFaphHKToy4pJiRJNOOBAkRHqAeaSnqI4+ITjy7MoEnSulCN+Dq+RLO1N8TMfKEGHqOSnpI9sWiNxX/81qRdEudmPphJImP54vciEEZwGllsEs5wZINFUGYU/VXiPtINSVVsRlVgrl48jKpnxZMo2DeqDZKYI40OALHIA9MUARlcA0sUAMYPIIxeAET7Ul71l61t3k0pf3MHII/0D6/AUYLpT4=</latexit><latexit sha1_base64="/KIkVMEnebPw97R9WEJxCLAdwHw=">AAACJXicbVDLSsNAFJ3UV62vqEs3g0VoNiURxC4UCm5cFBrBPqApYTKdtEMnD2YmQkn7M278FTcuLCK48lectFlo64GBwznncuceL2ZUSNP80gobm1vbO8Xd0t7+weGRfnzSFlHCMWnhiEW86yFBGA1JS1LJSDfmBAUeIx1vfJf5nSfCBY3CRzmJST9Aw5D6FCOpJFe/sSsN9wpOYdOAt9DxOcKpXWlOlWgsLGOWOiIJXAqVrHINlxowc6gxc/WyWTUXgOvEykkZ5LBdfe4MIpwEJJSYISF6lhnLfoq4pJiRWclJBIkRHqMh6SkaooCIfrq4cgYvlDKAfsTVCyVcqL8nUhQIMQk8lQyQHIlVLxP/83qJ9Gv9lIZxIkmIl4v8hEEZwawyOKCcYMkmiiDMqforxCOkmpKq2JIqwVo9eZ20L6uWWbUezHK9ltdRBGfgHFSABa5BHdwDG7QABs/gFbyDufaivWkf2ucyWtDymVPwB9r3D2nPoOE=</latexit>

Update Belief

Before Observation After Observation

P (L5) P (L5|O)

Page 56: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Update Belief

Before Observation After Observation

P (L5) P (L5|O)

P (L5|O) =P (O|L5)P (L5)Pi P (O|Li)P (Li)

<latexit sha1_base64="xt2+x1cuyZ83goqxUqyrBW8XG3A=">AAACJXicbVDLSgMxFL1TX7W+qi7dBIvQ2ZQZQdqFQsGNi0Ir2Ae0ZcikmTY28yDJCGXan3Hjr3TjwiKCK3/F9LHQ6oHA4ZxzubnHjTiTyrI+jdTG5tb2Tno3s7d/cHiUPT5pyDAWhNZJyEPRcrGknAW0rpjitBUJin2X06Y7vJ37zScqJAuDBzWKaNfH/YB5jGClJSd7XctXnCs0RlUT3aCOJzBJavnqWIvmwjInSUfGvsOQlnWu4jATzR1mTpxszipYC6C/xF6RXLk4fSwDQM3Jzjq9kMQ+DRThWMq2bUWqm2ChGOF0kunEkkaYDHGftjUNsE9lN1lcOUEXWukhLxT6BQot1J8TCfalHPmuTvpYDeS6Nxf/89qx8krdhAVRrGhAlou8mCMVonllqMcEJYqPNMFEMP1XRAZYN6V0sRldgr1+8l/SuCzYVsG+122UYIk0nME55MGGIpThDmpQBwLPMIU3mBkvxqvxbnwsoyljNXMKv2B8fQP+pKLK</latexit><latexit sha1_base64="Kcpj61YCglpnx2qAeijVh+OqHV8=">AAACJXicbVDLSgMxFM3UV62vqks3wSJ0NmVGkBZUKLhxUegI9gFtGTJppo3NPEgyQpnOv4gbf6UbFxYRXIg/4sK0daGtBwKHc87l5h4nZFRIw3jXUiura+sb6c3M1vbO7l52/6AugohjUsMBC3jTQYIw6pOapJKRZsgJ8hxGGs7gauo37gkXNPBv5TAkHQ/1fOpSjKSS7OyFla/YZ3AEqzq8hG2XIxxb+epIifrM0pO4LSLPplDJKlexqQ6nDtUTO5szCsYMcJmYPyRXLo7vzj++Hiw7O2l3Axx5xJeYISFaphHKToy4pJiRJNOOBAkRHqAeaSnqI4+ITjy7MoEnSulCN+Dq+RLO1N8TMfKEGHqOSnpI9sWiNxX/81qRdEudmPphJImP54vciEEZwGllsEs5wZINFUGYU/VXiPtINSVVsRlVgrl48jKpnxZMo2DeqDZKYI40OALHIA9MUARlcA0sUAMYPIIxeAET7Ul71l61t3k0pf3MHII/0D6/AUYLpT4=</latexit><latexit sha1_base64="Kcpj61YCglpnx2qAeijVh+OqHV8=">AAACJXicbVDLSgMxFM3UV62vqks3wSJ0NmVGkBZUKLhxUegI9gFtGTJppo3NPEgyQpnOv4gbf6UbFxYRXIg/4sK0daGtBwKHc87l5h4nZFRIw3jXUiura+sb6c3M1vbO7l52/6AugohjUsMBC3jTQYIw6pOapJKRZsgJ8hxGGs7gauo37gkXNPBv5TAkHQ/1fOpSjKSS7OyFla/YZ3AEqzq8hG2XIxxb+epIifrM0pO4LSLPplDJKlexqQ6nDtUTO5szCsYMcJmYPyRXLo7vzj++Hiw7O2l3Axx5xJeYISFaphHKToy4pJiRJNOOBAkRHqAeaSnqI4+ITjy7MoEnSulCN+Dq+RLO1N8TMfKEGHqOSnpI9sWiNxX/81qRdEudmPphJImP54vciEEZwGllsEs5wZINFUGYU/VXiPtINSVVsRlVgrl48jKpnxZMo2DeqDZKYI40OALHIA9MUARlcA0sUAMYPIIxeAET7Ul71l61t3k0pf3MHII/0D6/AUYLpT4=</latexit><latexit sha1_base64="/KIkVMEnebPw97R9WEJxCLAdwHw=">AAACJXicbVDLSsNAFJ3UV62vqEs3g0VoNiURxC4UCm5cFBrBPqApYTKdtEMnD2YmQkn7M278FTcuLCK48lectFlo64GBwznncuceL2ZUSNP80gobm1vbO8Xd0t7+weGRfnzSFlHCMWnhiEW86yFBGA1JS1LJSDfmBAUeIx1vfJf5nSfCBY3CRzmJST9Aw5D6FCOpJFe/sSsN9wpOYdOAt9DxOcKpXWlOlWgsLGOWOiIJXAqVrHINlxowc6gxc/WyWTUXgOvEykkZ5LBdfe4MIpwEJJSYISF6lhnLfoq4pJiRWclJBIkRHqMh6SkaooCIfrq4cgYvlDKAfsTVCyVcqL8nUhQIMQk8lQyQHIlVLxP/83qJ9Gv9lIZxIkmIl4v8hEEZwawyOKCcYMkmiiDMqforxCOkmpKq2JIqwVo9eZ20L6uWWbUezHK9ltdRBGfgHFSABa5BHdwDG7QABs/gFbyDufaivWkf2ucyWtDymVPwB9r3D2nPoOE=</latexit>

Page 57: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Update Belief

Before Observation After Observation

P (L5) P (L5|O)

P (L5|O) =P (O|L5)P (L5)Pi P (O|Li)P (Li)

<latexit sha1_base64="xt2+x1cuyZ83goqxUqyrBW8XG3A=">AAACJXicbVDLSgMxFL1TX7W+qi7dBIvQ2ZQZQdqFQsGNi0Ir2Ae0ZcikmTY28yDJCGXan3Hjr3TjwiKCK3/F9LHQ6oHA4ZxzubnHjTiTyrI+jdTG5tb2Tno3s7d/cHiUPT5pyDAWhNZJyEPRcrGknAW0rpjitBUJin2X06Y7vJ37zScqJAuDBzWKaNfH/YB5jGClJSd7XctXnCs0RlUT3aCOJzBJavnqWIvmwjInSUfGvsOQlnWu4jATzR1mTpxszipYC6C/xF6RXLk4fSwDQM3Jzjq9kMQ+DRThWMq2bUWqm2ChGOF0kunEkkaYDHGftjUNsE9lN1lcOUEXWukhLxT6BQot1J8TCfalHPmuTvpYDeS6Nxf/89qx8krdhAVRrGhAlou8mCMVonllqMcEJYqPNMFEMP1XRAZYN6V0sRldgr1+8l/SuCzYVsG+122UYIk0nME55MGGIpThDmpQBwLPMIU3mBkvxqvxbnwsoyljNXMKv2B8fQP+pKLK</latexit><latexit sha1_base64="Kcpj61YCglpnx2qAeijVh+OqHV8=">AAACJXicbVDLSgMxFM3UV62vqks3wSJ0NmVGkBZUKLhxUegI9gFtGTJppo3NPEgyQpnOv4gbf6UbFxYRXIg/4sK0daGtBwKHc87l5h4nZFRIw3jXUiura+sb6c3M1vbO7l52/6AugohjUsMBC3jTQYIw6pOapJKRZsgJ8hxGGs7gauo37gkXNPBv5TAkHQ/1fOpSjKSS7OyFla/YZ3AEqzq8hG2XIxxb+epIifrM0pO4LSLPplDJKlexqQ6nDtUTO5szCsYMcJmYPyRXLo7vzj++Hiw7O2l3Axx5xJeYISFaphHKToy4pJiRJNOOBAkRHqAeaSnqI4+ITjy7MoEnSulCN+Dq+RLO1N8TMfKEGHqOSnpI9sWiNxX/81qRdEudmPphJImP54vciEEZwGllsEs5wZINFUGYU/VXiPtINSVVsRlVgrl48jKpnxZMo2DeqDZKYI40OALHIA9MUARlcA0sUAMYPIIxeAET7Ul71l61t3k0pf3MHII/0D6/AUYLpT4=</latexit><latexit sha1_base64="Kcpj61YCglpnx2qAeijVh+OqHV8=">AAACJXicbVDLSgMxFM3UV62vqks3wSJ0NmVGkBZUKLhxUegI9gFtGTJppo3NPEgyQpnOv4gbf6UbFxYRXIg/4sK0daGtBwKHc87l5h4nZFRIw3jXUiura+sb6c3M1vbO7l52/6AugohjUsMBC3jTQYIw6pOapJKRZsgJ8hxGGs7gauo37gkXNPBv5TAkHQ/1fOpSjKSS7OyFla/YZ3AEqzq8hG2XIxxb+epIifrM0pO4LSLPplDJKlexqQ6nDtUTO5szCsYMcJmYPyRXLo7vzj++Hiw7O2l3Axx5xJeYISFaphHKToy4pJiRJNOOBAkRHqAeaSnqI4+ITjy7MoEnSulCN+Dq+RLO1N8TMfKEGHqOSnpI9sWiNxX/81qRdEudmPphJImP54vciEEZwGllsEs5wZINFUGYU/VXiPtINSVVsRlVgrl48jKpnxZMo2DeqDZKYI40OALHIA9MUARlcA0sUAMYPIIxeAET7Ul71l61t3k0pf3MHII/0D6/AUYLpT4=</latexit><latexit sha1_base64="/KIkVMEnebPw97R9WEJxCLAdwHw=">AAACJXicbVDLSsNAFJ3UV62vqEs3g0VoNiURxC4UCm5cFBrBPqApYTKdtEMnD2YmQkn7M278FTcuLCK48lectFlo64GBwznncuceL2ZUSNP80gobm1vbO8Xd0t7+weGRfnzSFlHCMWnhiEW86yFBGA1JS1LJSDfmBAUeIx1vfJf5nSfCBY3CRzmJST9Aw5D6FCOpJFe/sSsN9wpOYdOAt9DxOcKpXWlOlWgsLGOWOiIJXAqVrHINlxowc6gxc/WyWTUXgOvEykkZ5LBdfe4MIpwEJJSYISF6lhnLfoq4pJiRWclJBIkRHqMh6SkaooCIfrq4cgYvlDKAfsTVCyVcqL8nUhQIMQk8lQyQHIlVLxP/83qJ9Gv9lIZxIkmIl4v8hEEZwawyOKCcYMkmiiDMqforxCOkmpKq2JIqwVo9eZ20L6uWWbUezHK9ltdRBGfgHFSABa5BHdwDG7QABs/gFbyDufaivWkf2ucyWtDymVPwB9r3D2nPoOE=</latexit>

Page 58: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

Monty Hall

Page 59: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• Game show with 3 doors: A, B, and C

§ Behind one door is prize (equally likely to be any door)§ Behind other two doors is nothing§ We choose a door§ Then host opens 1 of other 2 doors, revealing nothing§ We are given option to change to other door

• Should we?§ Note: If we don’t switch, P(win) = 1/3 (random)

Let’s Make a Deal

Page 60: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• Without loss of generality, say we pick A§ P(A is winner) = 1/3

o Host opens either B or C, we always lose by switchingo P(win | A is winner, picked A, switched) = 0

§ P(B is winner) = 1/3o Host must open C (can’t open A and can’t reveal prize in B)o So, by switching, we switch to B and always wino P(win | B is winner, picked A, switched) = 1

§ P(C is winner) = 1/3o Host must open B (can’t open A and can’t reveal prize in C)o So, by switching, we switch to C and always wino P(win | C is winner, picked A, switched) = 1

§ Should always switch!o P(win | picked A, switched) = (1/3*0) + (1/3*1) + (1/3*1) = 2/3

Let’s Make a Deal

Page 61: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University

• Start with 1,000 envelopes, of which 1 is winner§ You get to choose 1 envelope

o Probability of choosing winner = 1/1000

§ Consider remaining 999 envelopeso Probability one of them is the winner = 999/1000

§ I open 998 of remaining 999 (showing they are empty)o Probability the last remaining envelope being winner =

999/1000

§ Should you switch?o Probability winning without switch =

o Probability winning with switch =

1

original # envelopes

original # envelopes - 1

original # envelopes

Slight Variant to Clarify

Page 62: Conditional Probability - Stanford University€¦ · •Conditional probabilityis probability that E occurs giventhat F has already occurred “Conditioning on F” •Written as

Piech, CS106A, Stanford University


Recommended