Probability

Dr Azmi Mohd TamilDept of Community Health

Universiti Kebangsaan Malaysia

Sample Spaces

4 A sample space is the set of all possible outcomes. However, some sample spaces are better than others.

4 Consider the experiment of flipping two coins. It is possible to get 0 heads, 1 head, or 2 heads. Thus, the sample space could be {0, 1, 2}. Another way to look at it is flip { HH, HT, TH, TT }. The second way is better because each event is as equally likely to occur as any other.

4 When writing the sample space, it is highly desirable to have events which are equally likely.

Sample Spaces

4Another example is rolling two dice. The sums are { 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 }. However, each of these aren't equally likely. The only way to get a sum 2 is to roll a 1 on both dice, but you can get a sum of 4 by rolling a 3-1, 2-2, or 3-1. The following table illustrates a better sample space for the sum obtain when rolling two dice.

Example

1 2 3 4 5 61 2 3 4 5 6 72 3 4 5 6 7 83 4 5 6 7 8 94 5 6 7 8 9 105 6 7 8 9 10 116 7 8 9 10 11 12

First DieSecond Die

Classical Probability

4 The relative frequency of a frequency distribution is the probability of the event occurring. This is only true, however, if the events are equally likely.

4 This gives us the formula for classical probability. The probability of an event occurring is the number in the event divided by the number in the sample space.

4 P(E) = n(E) / n(S)

Sum Freq Relative Freq

2 1 1/363 2 2/364 3 3/365 4 4/366 5 5/367 6 6/368 5 5/369 4 4/3610 3 3/3611 2 2/3612 1 1/36

Empirical Probability

4 Empirical probability is based on observation. The empirical probability of an event is the relative frequency of a frequency distribution based upon observation.

4 P(E) = f / n

Probability Rules

4 All probabilities are between 0 and 1 inclusive0 <= P(E) <= 1

4 The sum of all the probabilities in the sample space is 1

4 The probability of an event which cannot occur is 0.

4 The probability of an event which must occur is 1.

4 The probability of an event not occurring is one minus the probability of it occurring.

P(E') = 1 - P(E)

Mutually Exclusive Events

4Two events are mutually exclusive if they cannot occur at the same time.

4 If two events are mutually exclusive , then the probability of them both occurring at the same time is 0. Mutually Exclusive : P(A and B) = 0

4 If two events are mutually exclusive, then the probability of either occurring is the sum of the probabilities of each occurring.

Specific Addition Rule

4Only valid when the events are mutually exclusive.

P(A or B) = P(A) + P(B)

Example 1

4Given: P(A) = 0.20, P(B) = 0.70, A and B are mutually exclusive

B B' TotalA 0 0.2 0.2A' 0.7 0.1 0.8

Total 0.7 0.3 1

Non-Mutually Exclusive Events

4 In events which aren't mutually exclusive, there is some overlap. When P(A) and P(B) are added, the probability of the intersection (and) is added twice. To compensate for that double addition, the intersection needs to be subtracted.

4General Addition RuleP(A or B) = P(A) + P(B) - P(A and B)

Example 2

4Given P(A) = 0.20, P(B) = 0.70, P(A and B) = 0.15

B B' TotalA 0.15 0.05 0.2A' 0.55 0.25 0.8

Total 0.7 0.3 1

Independent Events

4 Two events are independent if the occurrence of one does not change the probability of the other occurring.

4 An example would be rolling a 2 on a die and flipping a head on a coin. Rolling the 2 does not affect the probability of flipping the head.

4 If events are independent, then the probability of them both occurring is the product of the probabilities of each occurring.

Specific Multiplication Rule

4Only valid for independent events

P(A and B) = P(A) * P(B)

Example 3

4P(A) = 0.20, P(B) = 0.70, A and B are independent.

B B' TotalA 0.14 0.06 0.2A' 0.56 0.24 0.8

Total 0.7 0.3 1

Dependent Events

4If the occurrence of one event does affect the probability of the other occurring, then the events are dependent.

Conditional Probability

4The probability of event B occurring that event A has already occurred is read "the probability of B given A" and is written: P(B|A)

4General Multiplication RuleP(A and B) = P(A) * P(B|A)

Example 4

4P(A) = 0.20, P(B) = 0.70, P(B|A) = 0.40

B B' TotalA 0.08 0.12 0.2A' 0.62 0.18 0.8

Total 0.7 0.3 1

Independence Revisited

4The following four statements are equivalent

1.A and B are independent events 2.P(A and B) = P(A) * P(B) 3.P(A|B) = P(A) 4.P(B|A) = P(B)

The question, "Do you smoke?" was asked of 100 people. Results are shown in the table.

1006931Total402812Female

604119MaleTotalNoYes.

•What is the probability of a randomly selected individual being a male who smokes? This is just a joint probability. The number of "Male and Smoke" divided by the total = 19/100 = 0.19 •What is the probability of a randomly selected individual being a male? This is the total for male divided by the total = 60/100 = 0.60. Since no mention is made of smoking or not smoking, it includes all the cases. •What is the probability of a randomly selected individual smoking? Again, since no mention is made of gender, this is a marginal probability, the total who smoke divided by the total = 31/100 = 0.31. •What is the probability of a randomly selected male smoking? This time, you're told that you have a male - think of stratified sampling. What is the probability that the male smokes? Well, 19 males smoke out of 60 males, so 19/60 = 0.31666... •What is the probability that a randomly selected smoker is male? This time, you're told that you have a smoker and asked to find the probability that the smoker is also male. There are 19 male smokers out of 31 total smokers, so 19/31 = 0.6129 (approx)

There are three major manufacturing companies that make a product: Aberations, Brochmailians, and Chompielians. Aberations has a 50% market share, and Brochmailians has a 30% market share. 5% of Aberations' product is defective, 7% of Brochmailians' product is defective, and 10% of Chompieliens' product is defective. This information can be placed into a joint probability distribution

1.000.0660.934Total0.200.10(0.20) = 0.0200.20-0.020 = 0.180Chompieliens0.300.07(0.30) = 0.0210.30-0.021 = 0.279Brochmailians0.500.05(0.50) = 0.0250.50-0.025 = 0.475Aberations

TotalDefectiveGoodCompany

•What is the probability a randomly selected product is defective? •What is the probability that a defective product came from Brochmailians?

The percent of the market share for Chompieliens wasn't given, but since the marginals must add to be 1.00, they have a 20% market share. Notice that the 5%, 7%, and 10% defective rates don't go into the table directly. This is because they are conditional probabilities and the table is a joint probability table. These defective probabilities are conditional upon which company was given. That is, the 7% is not P(Defective), but P(Defective|Brochmailians). The joint probability P(Defective and Brochmailians) = P(Defective|Brochmailians) * P(Brochmailians). The "good" probabilities can be found by subtraction as shown above, or by multiplication using conditional probabilities. If 7% of Brochmailians' product is defective, then 93% is good. 0.93(0.30)=0.279. •What is the probability a randomly selected product is defective? P(Defective) = 0.066 •What is the probability that a defective product came from Brochmailians? P(Brochmailian|Defective) = P(Brochmailian and Defective) / P(Defective) = 0.021/0.066 = 7/22 = 0.318 (approx). •Are these events independent? No. If they were, then P(Brochmailians|Defective)=0.318 would have to equal the P(Brochmailians)=0.30, but it doesn't. Also, the P(Aberations and Defective)=0.025 would have to be P(Aberations)*P(Defective) = 0.50*0.066=0.033, and it doesn't.

Bayes' Theorem

Let's use the same example, but shorten each event to its one letter initial, ie: A, B, C, and D instead of Aberations, Brochmailians, Chompieliens, and Defective.

P(D|B) is not a Bayes problem.

This is given in the problem. Bayes' formula finds the reverse conditional probability P(B|D).

It is based that the Given (D) is made of three parts, the part of D in A, the part of D in B, and the part of D in C.

P(B and D) P(B|D) = -----------------------------------------------------

P(A and D) + P(B and D) + P(C and D)

Inserting the multiplication rule for each of these joint probabilities gives

P(D|B)*P(B) P(B|D) = ------------------------------------------------------------

P(D|A)*P(A) + P(D|B)*P(B) + P(D|C)*P(C)