Chapter 1 Basics of Probability

Chapter 1 – Basics of Probability

1. Introduction

2. Basic Concepts and Definitions

3. Counting Problems

4. Axioms of Probability and the Addition Rule

5. Conditional Probability and the Multiplication Rule

6. Bayes’ Theorem

7. Independent Events

1.1 - Introduction

• Probability deals with describing random experiments– An activity in which the result is not known until it

is performed–Most everything in life is a random experiment

• Informally– Probability is a measure of how likely something

is to occur

• Probabilities take values between 0 and 1

1.2 – Basic Concepts and Definitions

• Definition 1.2.1 – An outcome is a result of a random experiment– The sample space of a random experiment is the

set of all possible outcomes– An event is a subset of the sample space

• Notation– Events are typically denoted by capital letters– P(A) = probability of event A occurring

Calculating Probabilities

Relative Frequency Approximation: To estimate the probability of an event A, repeat the random experiment several times (each repetition is called a trial) and count the number of times the event occurred. Then

The fraction on the right is called a relative frequency.

number of times occurred

number of trials

AP A

Calculating Probabilities

Theoretical Approach: If all outcomes of a random experiment are equally likely, S is the finite sample space, and A is an event, then

where n (A) is the number of elements in the set A and n (S) is the number of elements in the set S.

n AP A

n S

Relation

Law of Large Numbers: As the number of trials gets larger, the relative frequency approximation of P (A) gets closer to the theoretical value.

A probability is an average in the long run.

Example

• Bag with 4 blue, 3 red, 2 green, and 1 yellow cube

• Question: “How likely is it that we get a green cube?”– Let G = event we get a green cube–We want to know P(G)

Relative Frequency

• Choose a cube, record its color, replace, and repeat 50 times

Student 1 Student 2

Theoretical Approach

• Let

then

1 2 3 4 1 2 3 1 2 1

1 2

, , , , , , , , , , and

, ,

S B B B B R R R G G Y

G G G

( ) 10, ( ) 2

2( ) 0.2

10

n S n G

P G

Law of Large Numbers

• Combine students’ results– 50 + 50 = 100 trials– 12 + 7 = 19 green cubes

19Rel Freq 0.19

1.

000 2

Example 1.2.3

• Roll two “fair” six-sided dice and add the results– Let A = event the sum is greater than 7– Find P(A)

Sample Space

15 50.417

36 12P A

Random Variable

• Let X = the sum of the dice– Called a random variable

– Table is called the distribution of X

Subjective Probabilities

• Subjective Probabilities: P(A) is estimated by using knowledge of pertinent aspects of the situation

Example 1.2.4

• Will it rain tomorrow?– Let

then

since the outcomes are not equally likely

rain, not rain and rainS A

2(

1)P A

Example 1.2.4

• What is P(A)?–Weather forecasters use knowledge of weather and

information from radar, satellites, etc. to estimate a likelihood that it will rain tomorrow

Example 1.2.5

• Suppose we randomly choose an integer between 1 and 100

• Let

then

1, ,100 and 75, ,100S A

260.26

100

n AP A

n S

Example 1.2.5

• Suppose we randomly choose any number between 1 and 100

• Let

then a reasonable assignment of probability is

1,100 and 75,100S B

length of interval 250.253

length of interval 99

BP B

S

1.3 – Counting Problems

• Fundamental Counting Principle: Suppose a choice has to be made which consists of a sequence of two sub-choices. If the first choice has n1 options and the second choice has n2 options, then the total number of options for the overall choice is n1 × n2.

• Definition 1.3.1 Let n > 0 be an integer. The symbol n! (read “n factorial”) is

For convenience, we define 0! = 1.

! 1 2 2 1n n n n

Permutations

Definition 1.3.2 Suppose we are choosing r objects from a set of n objects and these requirements are met:

1. The n objects are all different.

2. We are choosing the r objects without replacement.

3. The order in which the choices are made is important.

Then the number of ways the overall choice can be made is called the number of permutations of n objects chosen r at a time.

!

!n r

nP

n r

Combinations

Definition 1.3.3 Suppose we are choosing r objects from a set of n objects and these requirements are met:

1. The n objects are all different.

2. We are choosing the r objects without replacement.

3. The order in which the choices are made is not important.

Then the number of ways the overall choice can be made is called the number of combinations of n objects chosen r at a time.

!

! !n r

n nC

r r n r

Arrangements

Definition 1.3.4 Suppose we are arranging n objects, n1 are identical, n2 are identical, … , nr are identical. Then the number of unique arrangements of the n objects is

1 2

!

! ! !r

n

n n n

Example 1.3.7

A local college is investigating ways to improve the scheduling of student activities. A fifteen-person committee consisting of five administrators, five faculty members, and five students is being formed. A five-person subcommittee is to be formed from this larger committee. The chair and co-chair of the subcommittee must be administrators, and the remainder will consist of faculty and students. How many different subcommittees could be formed?

Example 1.3.7

• Two sub-choices: 1. Choose two administrators.

2. Choose three faculty and students.

• Number of choices:

5 2

1020 120 2400

3P

Binomial Theorem

• Example 1.3.120

( )n

n n k k

k

na b a b

k

44 4

0

4 0 0 4 1 1 4 2 2 4 3 3 4 4 4

4 3 2 2 3 4

4( )

4 4 4 4 4

0 1 2 3 4

4 6 4

k k

k

x y x yk

x y x y x y x y x y

x x y x y xy y

1.4 – Axioms of Probability and the Addition Rule

Axioms of Events Let S be the sample space of a random experiment. An event is a subset of S. Let E be the set of all events, and assume that E satisfies the following three properties:

1 2 1 2

2. If , then (where is the complement of )

3. If

1.

, , , then

S E

A E A E A A

A A E A A E

1.4 – Axioms of Probability and the Addition Rule

Axioms of Probability Assume that for each event , a number P(A) is defined

in such a way that the following three properties hold:

1 2

0 ( ) 1

2. 1

3. If , , for which for , the

.

n

1

i j

P A

P S

A A E A A i j

1 2 1 2

1 2 1 2

for each integer 0, andn nP A A A P A P A P A

n

P A A P A P A

Theorem 1.4.2

1

Note that and that . So,

using axioms 2 and 3, we have

1

P A P A

S A A A A

P S P A A P A P A

Theorem 1.4.2 :

Proof :

Theorem 1.4.3

Let and be events. If ,

then ( ) ( ).

By elementary set theory,

and

so by axiom 3,

( ) ( ) ( )

since 0 by axiom 1.

A B A B

P A P B

B A B A A B A

P B P A B A P A P B A P A

P B A

Theorem 1.4.3 :

Proof :

The Addition Rule

Example 1.4.4: Randomly choose a studentE = event student did not complete the problems

F = event student got a C or below

( ) ( ) ( ) ( )P A B P A P B P A B Theorem 1.4.4 :

Example 1.4.4

38 34 26( ) 0.475 ( ) 0.425 ( ) 0.325

80 80 80( ) 0.475 0.425 0.325 0.575

P E P F P E F

P E F

1.5 – Conditional Probability and the Multiplication Rule

• Definition 1.5.1 The conditional probability of event A given that event B has occurred is

( )( | )

( )

provided that ( ) 0

P A BP A B

P B

P B

Example 1.5.2

If we randomly choose a family with two children and find out that at least one child is a boy, find the probability that both children are boys.

– A = event that both children are boys– B = event that at least one is a boy

boy boy, boy girl, girl boy, girl girlS

Example 1.5.2

boy boy, boy girl, girl boy, girl girl

3 1( ) and ( )

4 4( ) 1/ 4 1

( | )( ) 3 / 4 3

S

P B P A B

P A BP A B

P B

– A = event that both children are boys– B = event that at least one is a boy

Multiplication Rule

Definition 1.5.2 The probability that events A and B both occur is one trial of a random experiment is

( ) ( ) ( | )P A B P A P B A

Example 1.5.4

Randomly choose two different students. Find the probability they both completed the problems

A = event first student completed the problems

B = event second student completed the problems

Example 1.5.4

42 21 41( ) ( | )

80 40 7921 41

( ) ( ) ( | ) 0.27240 79

P A P B A

P A B P A P B A

1.6 – Bayes’ Theorem

Definition 1.6.1 Let S be the sample space of a random experiment and let A1 and A2 be two events such that

The collection of sets {A1, A2} is called a partition of S.

1 2 1 2andA A S A A

1.6 – Bayes’ Theorem

Theorem 1.6.1 If A1 and A2 form a partition of S, and B ⊂ S is any event, then for i = 1, 2

1 1 2 2

||

| |i i

i

P A P B AP A B

P A P B A P A P B A

Example 1.6.1

Medical tests for the presence of drugs are not perfect. They often give false positives where the test indicates the presence of the drug in a person who has not used the drug, and false negatives where the test does not indicate the presence of the drug in a person who has actually used the drug.

Suppose a certain marijuana drug test gives 13.5% false positives and 2.5% false negatives and that in the general population 0.5% of people actually use marijuana. If a randomly selected person from the population tests positive, find the conditional probability that the person actually used marijuana.

Example 1.6.1

Define the events

Note that {U, NU} forms a partition.

used marijuana, not used marijuana,

tested positive, and tested negative

U NU

T T

| 0.135, | 0.865,

| 0.025, | 0.975

P T NU P T NU

P T U P T U

Example 1.6.1

||

| |

(0.005)(0.975)

(0.005)(0.975) (0.995)(0.135)

0.035

P U P T UP U T

P U P T U P NU P T NU

Example 1.6.1

• “Tree diagram”

(0.005)(0.975)|

(0.005)(0.975) (0.995)(0.135)P U T

1.7 – Independent Events

Definition 1.7.1 Two events A and B are said to be independent if

If they are not independent, they are said to be dependent.

Informally: Independence means the occurrence of one event does not affect the probability of the other event occurring

( ) ( )P A B P A P B

Example 1.7.2

Suppose a student has an 8:30 AM statistics test and is worried that her alarm clock will fail and not ring, so she decides to set two different battery-powered alarm clocks. If the probability that each clock will fail is 0.005. Find the probability that at least one clock will ring.– Let F1 and F2 be the events that the first and

second clocks fail– Assume independence

Example 1.7.2

1 2

1 2

(at least one rings) 1 (both fail

1

1

1 0.005 0.005

0.999975

)

P F F

P F P F

P P

Mutual Independence

Definition 1.7.2 Three events A, B, and C are said to be pairwise independent if

They are said to be mutually independent (or simply independent) if they are pairwise independent and

( ) ( ) ( ), ( ) ( ) ( ),

and ( ) ( ) ( )

P A B P A P B P A C P A P C

P B C P B P C

( ) ( ) ( ) ( )P A B C P A P B P C

Example 1.7.7

Suppose that on a college campus of 1,000 students (referred to as the population), 600 support the idea of building a new gym and 400 are opposed. The president of the college randomly selects five different students and talks with each about their opinion. Find the probability that they all oppose the idea.

Example 1.7.7

Let – A1 = event that the first student opposes the idea,

– A2 = event that the second student opposes it, etc.

These events are dependent since the selections are made without replacement

1 2 3 4 5

400 399 398 397 396

1000 999 998 997 9960.010087

P A A A A A

Example 1.7.7

Now suppose the selections are made with replacement– The events are independent

1 2 3 4 5

5

400 400 400 400 400

1000 1000 1000 1000 1000

(0.4) 0.01024.

P A A A A A

Example 1.7.7

• Without replacement: Prob ≈ 0.010087• With replacement: Prob = 0.01024– Practically the same

• 5% Guideline: If no more than 5% of the population is being selected, then the selections may be treated as independent, even though they are technically dependent