www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
1
CS1512Foundations of
Computing Science 2
Lecture 23
Probability and statistics (4)
© J R W Hunter, 2006
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
2
a priori and Experimental ProbabilitiesRanges and Odds
• In both cases the minimum probability is 0• a priori – event cannot occur
a blue ball is drawn throwing 7 on an ordinary dice
• experimental – event has not occurred
• In both cases the maximum probability is 1• a priori – event must occur
sun will rise tomorrow (knowledge of physics)• experimental – the event always has occurred
the sun has risen every day in my experience
• For two events, express as odds:• odds on a red ball are 3/10 to 7/10 i.e. 3 to 7
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
3
Question
I think there’s a better than evens chance that English Jim will win the 2:50 at Exeter today.
How likely is it to snow tomorrow?
What kind of probability are we talking about here?• a priori?• experimental?
Neither• future event so no experimental evidence (and only happens once)• reasoning from first principles – what principles?
Subjective• best guess• most valuable when the guesser is an expert
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
4
Probability of two independent events
Experiment
• trial 1: pick a ball – event E1 = the ball is red
• replace the ball in the bag
• trial 2: pick a ball – event E2 = the ball is red
What is the probability of the event E: E = picking two red balls on successive trials (with replacement).
P( E ) = P( E1 and E2 ) or P( E1∩ E2 )
If the two trials are independent (the outcome of the first trial does not affect the outcome of the second) then:
P( E1 and E2 ) = P( E1 )* P( E2 ) - the multiplication law
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
5
Probability of two red balls
R R R W W W W W W WR Y Y Y N N N N N N NR Y Y Y N N N N N N NR Y Y Y N N N N N N NW N N N N N N N N N NW N N N N N N N N N NW N N N N N N N N N NW N N N N N N N N N NW N N N N N N N N N NW N N N N N N N N N NW N N N N N N N N N N
• 9 favourable outcomes out of 100 possibilities• all equally likely• probability of two reds = 9/100 ( = 3/10 * 3/10)
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
6
Probability of two non-independent events
Experiment
• trial 1: pick a ball – event E1 = the ball is red
• do not replace the ball in the bag
• trial 2: pick a ball – event E2 = the ball is red
What is the probability of the event E:
E = picking two red balls on successive trials (without replacement).
P( E1 and E2 ) = P( E1 )* P( E2 | E1 )
• the general form of the multiplication law
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
7
Probability of two red balls
R R W W W W W W WR Y Y N N N N N N NR Y Y N N N N N N NR Y Y N N N N N N NW N N N N N N N N NW N N N N N N N N NW N N N N N N N N NW N N N N N N N N NW N N N N N N N N NW N N N N N N N N NW N N N N N N N N N
firstdraw
second draw – assuming red chosen first time
• 6 favourable outcomes out of 90 possibilities• all equally likely• probability of two reds = 6/90 ( = 3/10 * 2/9)
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
8
Conditional probabilities
P( E2 | E1 ) = • the probability that E2 will happen given that E1 has
already happened.
If E2 and E1 are (conditionally, statistically) independent then
P( E2 | E1 ) = P( E2 )
because the occurrence of E1 has no effect on the occurrence of E2
so P( E1 and E2 ) = P( E1 )* P( E2 | E1 ) = P( E1 )* P( E2 )
Be careful about assuming independence.
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
9
(Revd. Thomas) Bayes Theorem
P( E1 and E2 ) = P( E1 )* P( E2 | E1 )
order of E1 and E2 is not important (probability of two red balls without replacement)
P( E2 and E1 ) = P( E2 )* P( E1 | E2 )
P( E2 )* P( E1 | E2 ) = P( E1 )* P( E2 | E1 )
P( E1 )* P( E2 | E1 )
P( E1 | E2 ) =
P( E2 )
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
10
Bayes Theorem
Why bother?
• Suppose E1 is a particular disease (say measles)
• Suppose E2 is a particular disease (say spots)
• P( E1 ) is the probability of a given patient having measles (before you see them)
• P( E2 ) is the probability of a given patient having spots (for whatever reason)
• P( E2 | E1 ) is the probability that someone who has measles has spots
Bayes theorem allows us to calculate the probability:
P( E1 | E2 ) = P (patient has measles given that the patient has spots)
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
11
Probability of two mutually exclusive events
Experiment• bag contains 3 red balls, 7 white balls and 5 blue balls• trial: pick a ball• event E1 = the ball is red• event E2 = the ball is blue
What is the probability of the event E = picking either a red ball or a blue ball
P( E ) = P( E1 or E2 ) or P( E1 ⋃ E2 )
If the two events are mutually exclusive (they can’t both happen in the same trial) then:
P( E1 or E2 ) = P( E1 ) + P( E2 ) - the addition law
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
12
Additive probabilities
1 2 3 4 5 6 1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12
• 9 outcomes correspond to event E = E1 or E2 • 36 possible outcomes• probability = 9/36 (= ¼ ) = 5/36 + 4/36 = P(8) + P(9)
Probability of throwing 8 (E1) or 9 (E2) with two dice:
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
13
Probability of two non-mutually exclusive events
1 2 3 4 5 6
1 1,1 1,2 1,3 1,4 1,5 1,6
2 2,1 2,2 2,3 2,4 2,5 2,6
3 3,1 3,2 3,3 3,4 3,5 3,6
4 4,1 4,2 4,3 4,4 4,5 4,6
5 5,1 5,2 5,3 5,4 5,5 5,6
6 6,1 6,2 6,3 6,4 6,5 6,6
• 11 outcomes correspond to event E E = E1 or E2 or [implicitly] (E1 and E2)
• 36 possible outcomes• probability = 11/36 = 6/36 + 6/36 – 1/36
E = Probability of throwing at least one 5 with two diceE1 = 5 on first dice; E2 = 5 on second dice
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
14
Discrete random variables
Variable
• an attribute or property of an element which has a value
Discrete variable
• a variable where the possible values are countable
Discrete random variable
• a discrete variable where the values taken are associated with a probability
Discrete probability distribution
• formed by the possible values and their probabilities
• P(X = <some value>) = ...
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
15
Example – fair coin
Let variable (X) be number of heads in one trial (toss).
Values are 0 and 1
Probability distribution:
Value of X ( i ) 0 1
P(X = i) 0.5 0.5
0
0.1
0.2
0.3
0.4
0.5
0.6
0 1
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
16
Example – total of two dice
Let variable (X) be total when two dice are tossed.Values are 2, 3, 4, ... 11, 12
Probability distribution:
P(X = 2) = 1/36 P(X = 3) = 2/36
...
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0.18
2 3 4 5 6 7 8 9 10 11 12
1 2 3 4 5 6
1 2 3 4 5 6 7
2 3 4 5 6 7 8
3 4 5 6 7 8 9
4 5 6 7 8 9 10
5 6 7 8 9 10 11
6 7 8 9 10 11 12
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
17
Bernouilli probability distribution
Bernoulli trial:
• one in which there are only two possible and exclusive outcomes;
• call the outcomes ‘success’ and ‘failure’;
• consider the variable X which is the number of successes in one trial
Probability distribution:
Value of X ( i ) 0 1
P(X = i) (1-p) p
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0 1
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
18
Binomial distribution
Carry out n Bernoulli trials; in each trial the probability of success is p
X is the number of successes in n trials?
Possible values are: 0, 1, 2, ... i, ... n-1, n
We state (without proving) that the distribution of X is:
P(X=r) = nCr pr (1-p)n-r
n!nCr = n! = n (n-1) (n-2) ... 2 1 r! (n-r)!
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
19
Example of binomial distribution
0.00E+00
2.00E-02
4.00E-02
6.00E-02
8.00E-02
1.00E-01
1.20E-01
1.40E-01
1.60E-01
1.80E-01
2.00E-01
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Probability of r heads in n trials (n = 20)
P(X = r)
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
20
Example of binomial distribution
0.00E+00
1.00E-02
2.00E-02
3.00E-02
4.00E-02
5.00E-02
6.00E-02
7.00E-02
8.00E-02
9.00E-02
Probability of r heads in n trials (n = 100)
P(X = r)
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
21
Probability of being in a given range
Probability of r heads in n trials (n = 100)
P(0 ≥ X < 5)P(5 ≥ X < 10)...
Moving towards the notionof a continuous distribution
0.00E+00
5.00E-02
1.00E-01
1.50E-01
2.00E-01
2.50E-01
3.00E-01
3.50E-01
4.00E-01
www.csd.abdn.ac.uk/~jhunter/teaching/CS1512/lectures/
CS1512
CS1512
22
Probability of being in a given range
P(55 ≥ X < 65) = P(55 ≥ X < 60) + P(60 ≥ X < 65) (mutually exclusive events)
Add the heights of the columns
0.00E+00
5.00E-02
1.00E-01
1.50E-01
2.00E-01
2.50E-01
3.00E-01
3.50E-01
4.00E-01