Embed Size (px)
Citation preview
Rules of Probability
Recall:
Axioms of Probability
1. P[E] ≥ 0.
2. P[S] = 1
3. 1 2 3 1 2 3P E E E P E P E P E
Property 3 is called the additive rule for probability
if Ei ∩ Ej = f
Note: for any event E
1 P S P E E P E P E
and S E E E E f
hence
or 1P E P E
Note:
Hence 1 1 1 0P P S P Sf
S f
Example –Probabilities for various events in Lotto 6/49
n Prize Prob
0 winning 6,096,454 nil 0.4359649755
1 winning 5,775,588 nil 0.4130194505
2 winning 1,678,950 nil 0.1200637937
2 + bonus 172,200 5.00$ 0.0123142353
3 winning 246,820 10.00$ 0.0176504039
4 winning 13,545 80.00$ 0.0009686197
5 winning 252 2,500.00$ 0.0000180208
5 + bonus 6 100,000.00$ 0.0000004291
6 winning 1 4,000,000.00$ 0.0000000715
Total 13,983,816
Let E = Event that no money is won
{0 winning #} {1 winning #} {2 winning #}
= 0.435965 + 0.413019 + 0.120064 = 0.969048
0 winning # 1 winning # 2 winning #P E P P P
money is won 1P E P P E
= 1 - 0.969048 = 0.030952
Note: This also could have been calculated by
adding up the probability of events where money
is won.
1P E P E
If it is easier to compute the probability of the
compliment of an event, than use the above equation
to calculate the probability of the event E.
Example: The birthday problem
In a room of n unrelated individuals, what is the
probability that at least 2 have the same birthday.
1P E P E
E
n(S) = total number of ways of assigning the n
individuals birthdays = 365n
= the event no pair of individuals have the
same birthday.
Let E = the event that at least 2 have the same
birthday.
365
!n E nn
= total number of ways of assigning
the n distinct birthdays
Hence
365!
1 1365n
nn
P E P E
365!
365n
nn E n
P En S
365!
!! 365 !
1365n
nn n
365!1
365 !365nn
365 364 363 365 11
365 365 365 365
n
Thus Two people have same birthday
in a room of people
Pn
365 364 363 365 11
365 365 365 365
n
n P [E ]
10 0.1169
20 0.4114
22 0.4757
23 0.5073
30 0.7063
40 0.8912
Table:
50%
Rule The additive rule
P[A B] = P[A] + P[B] – P[A B]
and
if A B = f P[A B] = P[A] + P[B]
Proof
and Ei Ej = f
A B A B A B A B
1 2 3E E E
B
A
A B A BA B
Als0 1 2A A B A B E E
2 3B A B A B E E
Now P A B P A B P A B P A B
1 2 3P E P E P E
BA
A B A BA B
1 2P A P A B P A B P E P E
2 3P B P A B P A B P E P E
1 2 32P A P B P E P E P E
P A B P A B
Hence P A B P A P B P A B
Example:
Saskatoon and Moncton are two of the cities competing
for the World university games. (There are also many
others). The organizers are narrowing the competition to
the final 5 cities.
There is a 20% chance that Saskatoon will be amongst
the final 5. There is a 35% chance that Moncton will be
amongst the final 5 and an 8% chance that both
Saskatoon and Moncton will be amongst the final 5.
What is the probability that Saskatoon or Moncton will
be amongst the final 5.
Hence P A B P A P B P A B
Solution:
Let A = the event that Saskatoon is amongst the final 5.
Let B = the event that Moncton is amongst the final 5.
Given P[A] = 0.20, P[B] = 0.35, and P[A B] = 0.08
What is P[A B]?
Note: “and” ≡ , “or” ≡ .
P A B P A P B P A B
0.20 0.35 0.08 0.47
Another Example – Bridge Hands
A Bridge hand is 13 cards chosen from a deck of
52 cards.
Total number of Bridge Hands
52 52 51 50 49 42 41 40
13 13 12 11 10 3 2 1N n S
635,013,559,600
1. What is the probability that the hand
contains exactly 5 spades (8 non-spades)?
2. What is the probability that the hand
contains exactly 5 hearts?
3. What is the probability that the hand
contains exactly 5 spades and 5 hearts?
4. What is the probability that the hand
contains exactly 5 spades or 5 hearts?
(i. e. a five card major)
Some Questions
1. What is the probability that the hand contains
exactly 5 spades (8 non-spades)?
Solutions
Let A = the event that the hand contains exactly 5
spades (8 non-spades)?
n(A) = the number of hands that contain exactly 5
spades (8 non-spades)?
13 39
1,287 61,523,7485 8
79,181,063,676
the number of ways of choosing the 5 spades
the number of ways of choosing the 8 non-spades
79,181,063,676
= 0.124691926635,013,559,600
n AP A
n S
2. What is the probability that the hand
contains exactly 5 hearts?
Let B = the event that the hand contains exactly
5 hearts?
13 39
5 8 =
52
13
n BP B
n S
79,181,063,676= 0.124691926
635,013,559,600
3. What is the probability that the hand
contains exactly 5 spades and 5 hearts?
A B = the event that the hand contains exactly 5
spades and 5 hearts?
13 13 26
5 5 3n A B
4,306,559,400
the number of ways of
choosing the 5 spades
the number of ways of
choosing the 3 non-
spades,hearts
1,287 1,287 61,523,748
the number of ways of
choosing the 5 hearts
4,306,559,400
and 0.006781838635,013,559,600
P A B
4. What is the probability that the hand
contains exactly 5 spades or 5 hearts?
(i. e. a five card major)
P A B P A P B P A B
0.124691926 0.124691926 0.006781838
0.2426
Thus there is a 24.26% chance that a bridge hand
will contain at least one five card major.
Rule The additive for more than two events
P[A B C] = P[A] + P[B] + P[C] – P[A B]
then
if A B = f, A C = f and B C = f
P[A B C] = P[A] + P[B] + P[C]
For three events
– P[A C] – P[B C] + P[A B C]
A
B
C
E1
E3 E2
E6
E5
E4 E7
A B C = E1 E2 E3 E4 E5 E5 E7
A B = E1 E2
A C = E1 E4
B C = E1 E3
A = E1 E2 E4 E5
B = E1 E2 E3 E6
C = E1 E3 E4 E7
A B C = E1
When these three are added
E2, E3 and E4 are counted
twice and E1is counted three
times
When these three are subtracted off
the extra contributions of E2, E3 and
E4 are removed and the contribution
of E1is completely removed
To correct we now have to add back
in the contribution of E1.
An Example
Three friends A, B and C live together in the same apartment.
For an upcoming “Stanley cup playoff game” there is
1. An 80% chance that A will watch.
2. A 60% chance that B will watch.
3. A 45% chance that C will watch.
4. A 50% chance that A and B will watch. (and possibly C)
5. A 30% chance that A and C will watch. (and possibly B)
6. A 25% chance that B and C will watch. (and possibly A)
7. A 15% chance that all three (A , B and C) will watch.
What is the probability that either A, B or C watch the upcoming
“Stanley cup playoff game?”
Thus 1. P[A] = 0.80.
2. P[B] = 0.60.
3. P[C] = 0.45.
4. P[A B] = 0.50.
5. P[A C] = 0.30.
6. P[B C] = 0.25.
7. P[A B C] = 0.15.
P[A B C] = P[A] + P[B] + P[C] – P[A B]
– P[A C] – P[B C] + P[A B C]
= 0.80 + 0.60 + 0.45 – 0.50 – 0.30 – 0.25 + 0.15
= 0.95
Rule The additive rule for more than two events
then
Finally if Ai Aj = f for all i ≠ j.
For n events
11
n n
i i i j
i i ji
P A P A P A A
i j k
i j k
P A A A
1
1 21n
nP A A A
11
n n
i i
ii
P A P A
Example: This is a Classic problem
Suppose that we have a family of n persons.
• At Christmas they decide to put their names in a hat.
• Each person will randomly pick a name.
• This will be the only person for which they will buy a present.
Questions
1. What is the probability that at least one person picks his (or her) own name?
2. How does this probability change as the size of the family, n, increase to infinity?
Solution:
Let Ai = the event that person i picks his own name.
We want to calculate:
11
n n
i i i j
i i ji
P A P A P A A
i j k
i j k
P A A A
1
1 21n
nP A A A
1
n
i
i
P A
Now because of the additive rule
11
n n
i i i j
i i ji
P A P A P A A
i j k
i j k
P A A A
1
1 21n
nP A A A
Note: N = n(S) = the total number of ways you can
assign the names to the n people = n!
To calculate:
we need to calculate:
1 2, , , ,i i j i j k nP A P A A P A A A P A A A
Now
n(Ai) = the number of ways we can assign person i his
own name (1) and arbitrarily assign names to the
remaining n – 1 persons ((n – 1)!)
= 1 × (n – 1)!
= (n – 1)! :
Thus:
1 ! 1
!
i
i
n A nP A
n S n n
1 1
11
n n
i
i i
P An
and
Now
n(Ai ∩ Aj) = the number of ways we can assign person i and
person j their own name (1) and arbitrarily assign
names to the remaining n – 2 persons ((n – 2)!)
= 1 × (n – 2)!
= (n – 2)! :
Thus:
2 ! 1
! 1
i j
i j
n A A nP A A
n S n n n
1 1
2 1 2!i i
i j
nP A A
n n
and
Now
n(Ai ∩ Aj ∩ Ak) = the number of ways we can assign person
i, person j and person k their own name (1) and
arbitrarily assign names to the remaining n – 3
persons ((n – 3)!)
= 1 × (n – 3)!
= (n – 3)! :
Thus:
3 ! 1
! 1 2
i j k
i j k
n A A A nP A A A
n S n n n n
1 1
3 1 2 3!i i k
i j k
nP A A A
n n n
and
Continuing
n(A1 ∩ A2 ∩ A3 ∩ … ∩ An) = the number of ways we can
assign all persons their own name
= 1
Thus:
1 2 3
1 2 3
1
!
n
n
n A A A AP A A A A
n S n
11
n n
i i i j
i i ji
P A P A P A A
i j k
i j k
P A A A
1
1 21n
nP A A A
Finally:
11 1 1 1 11 ( 1)
2! 3! 4! 5! !
n
n
As the family size increases to infinity ( the number of
terms become infinite)
1
1
1 1 1 1 11 1 1
2! 3! 4! 5!i
i
P A ee
2 3 4 5
since 12! 3! 4! 5!
x x x x xe x
Summary of Rules to date
1. 1P E P E
2. P A B P A P B P A B
P A P B
Additive Rules
if A and B disjoint
11
3. n n
i i i j
i i ji
P A P A P A A
i j k
i j k
P A A A
1
1 21n
nP A A A
1
n
i
i
P A
if Ai and Aj are all disjoint.
Conditional Probability
Conditional Probability
• Frequently before observing the outcome of a random experiment you are given information regarding the outcome
• How should this information be used in prediction of the outcome.
• Namely, how should probabilities be adjusted to take into account this information
• Usually the information is given in the following form: You are told that the outcome belongs to a given event. (i.e. you are told that a certain event has occurred)
Definition
Suppose that we are interested in computing the
probability of event A and we have been told
event B has occurred.
Then the conditional probability of A given B is
defined to be:
P A BP A B
P B
if 0P B
Rationale:
If we’re told that event B has occurred then the sample space is restricted to B.
The probability within B has to be normalized, This is achieved by dividing by P[B]
The event A can now only occur if the outcome is in of A ∩ B. Hence the new probability of A is:
P A BP A B
P B
B A
A ∩ B
An Example
The academy awards is soon to be shown.
For a specific married couple the probability that
the husband watches the show is 80%, the
probability that his wife watches the show is
65%, while the probability that they both watch
the show is 60%.
If the husband is watching the show, what is the
probability that his wife is also watching the
show
Solution:
The academy awards is soon to be shown.
Let B = the event that the husband watches the show
P[B]= 0.80
Let A = the event that his wife watches the show
P[A]= 0.65 and P[A ∩ B]= 0.60
P A BP A B
P B
0.600.75
0.80
Another Example
Suppose a bridge hand (13 cards) is selected from a
deck of 52 cards
Suppose that the hand contains 5 spades. What is the
probability that it also contains 5 hearts.
Solution
N = n(S) = the total # of bridge hands = 52
13
Let A = the event that the hand contains 5 hearts
Let B = the event that the hand contains 5 spades
13 39
13 39 5 8,
525 8
13
n B P B
13 13 26 52 13 26
5 5 3 13 5 3
13 39 52 39
5 8 13 8
P A BP A B
P B
13 13 26
13 13 26 5 5 3,
525 5 3
13
n A B P A B
Another Example
In the dice game – craps, if on the first roll you roll
i. a 7 or 11 you win
ii. a 2 or 12 you lose,
iii. If you roll any other number {3,4,5,6,8,9,10}
that number is your point. You continue to roll
until you roll your point (win) or a 7 (lose)
Suppose that your point is 5 what is the probability that
you win.
Repeat the calculation for other values of your point.
The Sample Space S
N = n(S) = 36
(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)
(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)
(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)
(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)
n(B) = 4 + 6 = 10 10
36P B
Let B = the event {5} or {7}
(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)
(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)
(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)
(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)
n(A) = n(A ∩ B) = 4 4
36P A P A B
Let A = the event {5} = A ∩ B
(1,1) (1,2) (1,3) (1,4) (1,5) (1,6)
(2,1) (2,2) (2,3) (2,4) (2,5) (2,6)
(3,1) (3,2) (3,3) (3,4) (3,5) (3,6)
(4,1) (4,2) (4,3) (4,4) (4,5) (4,6)
(5,1) (5,2) (5,3) (5,4) (5,5) (5,6)
(6,1) (6,2) (6,3) (6,4) (6,5) (6,6)
4 36 40.4
10 36 10
P A BP A B
P B
Probability of winning Point = {3, 4, 5, 6, 8, 9, 10}
Point P [win]
3 0.250
4 0.333
5 0.400
6 0.455
8 0.455
9 0.400
10 0.333
Independence
Definition
Two events A and B are called independent if
P A B P A P B
P A B P A P B
P A B P AP B P B
Note if 0 and 0 thenP B P A
and P A B P A P B
P B A P BP A P A
Thus in the case of independence the conditional probability of
an event is not affected by the knowledge of the other event
Difference between independence
and mutually exclusive
0 and 0. (also 0P A P B P A B
Two mutually exclusive events are independent only in
the special case where
mutually exclusive
A B
Mutually exclusive events are
highly dependent otherwise. A
and B cannot occur
simultaneously. If one event
occurs the other event does not
occur.
P A B P A P B
or
Independent events
A B
The ratio of the probability of the
set A within B is the same as the
ratio of the probability of the set
A within the entire sample S.
P A B P AP A
P B P S
A B
S
P A B P A P B
To check if A and B are independent
compute , and P A P B P A B
and verify that
Example: A coin is tossed three times
S = sample space =
{HHH, HHT, HTH, THH, HTT, THT, TTH, TTT}
Each outcome is equally likely (prob = 1/8}
Let A = the event that the first Head occurs on toss 2.
= {THH, THT}
Let B = the event that the more Tails than Heads
= {HTT, THT, TTH, TTT}
Are A and B independent?
1 1 1
since 8 4 2
P A B P A P B
2 1 4 1 1
, = = and 8 4 8 2 8
P A P B P A B
Solution:
A and B are independent.
if 0
if 0
P A P B A P AP A B
P B P A B P B
The multiplicative rule of probability
and
P A B P A P B
if A and B are independent.
Example: Polya’s Urn model
An urn contains r red balls
and b black balls.
A ball is selected at random,
its colour is noted, it is
replaced together with c balls
of the same colour.
A second ball is selected.
What is the probability that you get first a black ball
then a red ball?
What is the probability that a red ball is selected on the
second draw?
Solution:
Let B1 = the event that the first ball is black.
Let R2 = the event that the second ball is red.
What is the probability that you get first a black ball
then a red ball?
1
bP B
r b
2 1
rP R B
r b c
1 2 1 2 1
b rP B R P B P R B
r b r b c
What is the probability, that a red ball is selected
on the second draw?
2P R
2 2 1 2 1: R R B R B Note
2 1 2 1 2P R P B R P B R 2 1R B
2R 1B
2 1R B
1 2 1 1 2 1P B P R B P B P R B
b r r r c
r b r b c r b r b c
r b r c r
r b c r b r b r b
Bayes Rule
• Due to the reverend T. Bayes
• Picture found on website: Portraits of Statisticians
• http://www.york.ac.uk/depts/maths/histstat/people/welcome.htm#h
P A P B AP A B
P A P B A P A P B A
Proof:
P A BP A B
P B
P A P B A
P A P B A P A P B A
P A B
P A B P A B
Example:
We have two urns. Urn 1 contains 14 red balls and 12 black balls. Urn 2 contains 6 red balls and 20 black balls.
An Urn is selected at random and a ball is selected from that urn.
If the ball turns out to be red what is the probability
that it came from the first urn?
Urn 1 Urn 2
Solution:
Note: the desired conditional probability is in the
reverse direction of the given conditional probabilities.
This is the case when Bayes rule should be used
14 6,
26 26P B A P B A
A
Let A = the event that we select urn 1
= the event that we select urn 2
1
2P A P A
Let B = the event that we select a red ball
We want .P A B
Bayes rule states
P A P B AP A B
P A P B A P A P B A
1 142 26
61 14 12 26 2 26
14 140.70
14 6 20
Example: Testing for a disease
Suppose that 0.1% of the population have a certain
genetic disease.
A test is available the detect the disease.
If a person has the disease, the test concludes that he
has the disease 96% of the time. It the person doesn’t
have the disease the test states that he has the disease
2% of the time.
Two properties of a medical test
Sensitivity = P[ test is positive | disease] = 0.96
Specificity = P[ test is negative | no disease] = 1 – 0.02 = 0.98
A person takes the test and the test is positive, what is the
probability that he (or she) has the disease?
Solution:
Note: Again the desired conditional probability is in
the reverse direction of the given conditional
probabilities.
0.96, 0.02P B A P B A
A
Let A = the event that the person has the disease
= the event that the person doesn’t have the
disease
0.001, 1 0.001 0.999P A P A
Let B = the event that the test is positive.
We want .P A B
Bayes rule states
P A P B AP A B
P A P B A P A P B A
0.001 0.96
0.001 0.96 0.999 0.02
0.000960.0458
0.00096 .01998
Thus if the test turns out to be positive the chance of
having the disease is still small (4.58%).
Compare this to (.1%), the chance of having the disease
without the positive test result.
