View
218
Download
1
Embed Size (px)
Citation preview
Agenda
• Project• WebCT• Late HW• Math
– Independence– Conditional Probability– Bayes Formula & Theorem
• Steyvers, et al 2003
Independence
• Two events A and B are independent if the occurrence of A has no influence on the probability of the occurrence of B.– Independent: “It doesn’t matter who is elected
president, the world will still be a mess.”– Not independent: “If candidate B is elected
president, the probability that the world will be a mess is 99%. If candidate A is elected, the probability that the world will be a mess will be lowered to 98%.”
Independence• A and B are independent if P(AB) = P(A) x
P(B).– Independent:
• Pick a card from a deck. • A = “The card is an ace”, P(A) = 1/13.• B = “The card is a spade”, P(B) = 1/4• P(AB) = 1/13 x 1/4 = 1/52.
– Not independent: • Draw two cards from a deck without replacement. • A = “The first card is a space”, P(A) = 1/4• B = “The second card is a spade”, P(B) = 1/4• P(AB) = (13 x 12) / (52 x 51) < 1/4 x 1/4.
Conditional Probability
• Example:– What is the probability that a husband will
vote Democrat given that his wife does?– P(HusbandDemocrat|WifeDemocrat)– This is different from:
• What is the probability that a husband will vote democrat?
• What is the probability that a hustband and wife will vote democrat?
All possible events
Conditional Probability
• P(B|A) is the conditional probability that B will occur given that A has occurred: P(B|A) = P(BA) / P(A).
P(A) = A occurs
P(B) = B occurs
P(BA) = A and B occur
Conditional Probability
• Suppose we roll two dice– A = “The sum is 8”
• A = {(2,6), (3,5), (4,4), (5,3), (6,2)}• P(A) = 5/36
– B = “The first die is 3”• B = {(3,1), (3,2), (3,3), (3,4), (3,5), (3,6)}• P(B) = 6/36
– AB = {(3,5)}– P(B|A) = (1/36)/(5/36) = 1/5.
Conditional Probability36
5 61
P(A) = 5/36P(B) = 6/36P(BA) = 1/36P(B|A) = (1/36)/(5/36) = 1/5
P(B|A) = P(BA)/P(A)
Bayes FormulaSuppose families have 1, 2, or 3 children with 1/3probability each. Bobby has no brothers. What is the probability he is an only child?
Bayes FormulaSuppose families have 1, 2, or 3 children with 1/3probability each. Bobby has no brothers. What is the probability he is an only child?
Let child1, child2, and child3 be the events thatA family has 1, 2, or 3 children, respectively.
Let boy1 be the event that a family has only 1 boy.
Want to compute P(child1|boy1) = P(child1boy1)/P(boy1)
Bayes FormulaSuppose families have 1, 2, or 3 children with 1/3probability each. Bobby has no brothers. What is the probability he is an only child?
Want to compute P(child1|boy1) = P(child1boy1)/P(boy1)
We need to compute P(child1boy1) and P(boy1)
Bayes Formula
We need to compute P(child1boy1) and P(boy1)
Because P(B|C) = P(CB) / P(C), we can write: P(CB) = P(C) P(B|C).
So, P(child1boy1) = P(child1)P(boy1| child1)= 1/3 x 1/2 = 1/6.
Bayes Formula
We need to compute P(child1boy1) and P(boy1)
P(boy1) = P(child1boy1) + P(child2boy1) + P(child3boy1)
We know P(child1boy1) = 1/6.Likewise,
P(child2boy1) = 1/6P(child3boy1) = 1/8
P(boy1) = 1/6 + 1/6 + 1/8
Bayes FormulaSuppose families have 1, 2, or 3 children with 1/3probability each. Bobby has no brothers. What is the probability he is an only child?
P(child1|boy1) = P(child1boy1)/P(boy1) = (1/6) / (1/6 + 1/6 + 1/8) = 4/11
Bayes Formula
1 Child120
2 Children120
3 Children120
Suppose families have 1, 2, or 3 children with 1/3probability each. Bobby has no brothers. What is the probability he is an only child?
1 boy60
1 boy60
1 boy45
Bayes Formula
1 Child120
2 Children120
3 Children120
1 boy60
1 boy60
1 boy45
P(child1|boy1) = P(child1boy1)/P(boy1) = (60/360) / ((60+60+45)/360) = 60/165 = 4/11
Bayes Formula
Event1 Event2 Eventn
Sub-event
Sub-event
Sub-event
…
Known: P(Eventi), P(Eventi) = 1, and P(Subevent|Eventi)
Compute: P(Event1|Subevent)
Bayes Formula
Event1 Event2 Eventn
Sub-event
Sub-event
Sub-event
…
P(Event1|Subevent) = P(Event1Subevent) / P(Subevent)
P(EventiSubevent) = P(Eventi)P(Subevent|Eventi)P(Subevent) = P(EventiSubevent) = P(Eventi)P(Subevent|Eventi)
Bayes Formula
Event1 Event2 Eventn
Sub-event
Sub-event
Sub-event
…
P(Event1)P(Subevent|Event1)P(Event1|Subevent) = -----------------------------------------
P(Eventi)P(Subevent|Eventi)
Bayes Formula
1% of the population has a disease.99% of the people who have the disease have the symptoms.10% who don’t have the disease have the symptoms.
Let D = “A person has the disease”.Let S = “A person has the symptoms”.
P(D) = .01 and so P(~D) = .99P(S|D) = .99 and P(S|~D) = .10
What is P(D|S)?
Bayes Formula P(D) P(S|D)
P(D|S) = -------------------------------------------- P(D) P(S|D) + P(~D) P(S|~D)
= (.01 x .99) / (.01 x .99 + .99 x .10) = .091
Bayes Formula P(Event1)P(Subevent|Event1)
P(Event1|Subevent) = ----------------------------------------- P(Eventi)P(Subevent|Eventi)
If there are a very large portion of events, the denominator may be very hard to calculate.
If, however, you are only interested in relative probabilities…
Bayes Formula P(Event1)P(Subevent|Event1)
P(Event1|Subevent) = ----------------------------------------- P(Eventi)P(Subevent|Eventi)
P(Event1|Subevent) P(Event1)P(Subevent|Event1)--------------------------- = ----------------------------------------P(Event2|Subevent) P(Event2)P(Subevent|Event2)
This is called the “odds”.
Bayes Formula• Let’s say you have 2 hypotheses (or
models), H1 and H2, under consideration.
• The log odds of these two hypotheses given experimental data, D, is:
€
logP(H1 |D)
P(H2 |D)= log
P(H1)
P(H2)
P(D |H1)
P(D |H2)
⎛
⎝ ⎜
⎞
⎠ ⎟
= logP(H1)
P(H2)
⎛
⎝ ⎜
⎞
⎠ ⎟+ log
P(D |H1)
P(D |H2)
⎛
⎝ ⎜
⎞
⎠ ⎟
Bayes Formula
€
logP(H1 |D)
P(H2 |D)= log
P(H1)
P(H2)
P(D |H1)
P(D |H2)
⎛
⎝ ⎜
⎞
⎠ ⎟
= logP(H1)
P(H2)
⎛
⎝ ⎜
⎞
⎠ ⎟+ log
P(D |H1)
P(D |H2)
⎛
⎝ ⎜
⎞
⎠ ⎟
Posterior odds =Relative belief in 2 hypotheses given the data. Quantity of interest.
Bayes Formula
€
logP(H1 |D)
P(H2 |D)= log
P(H1)
P(H2)
P(D |H1)
P(D |H2)
⎛
⎝ ⎜
⎞
⎠ ⎟
= logP(H1)
P(H2)
⎛
⎝ ⎜
⎞
⎠ ⎟+ log
P(D |H1)
P(D |H2)
⎛
⎝ ⎜
⎞
⎠ ⎟
Prior odds =Relative belief in 2 hypotheses Before observing any data. Often assumed to be 1 (0 in log odds).