MAS113 Introduction to Probability and Statistics · Introduction Set theory and probability Measure MAS113 Introduction to Probability and Statistics Dr Jonathan Jordan School of

IntroductionSet theory and probability

Measure

MAS113 Introduction to Probability and

Statistics

Dr Jonathan Jordan

School of Mathematics and Statistics, University of Sheffield

2017–18

Dr Jonathan Jordan MAS113 Introduction to Probability and Statistics


Measure

Studying probability theory

There are (at least) two ways to think about the study ofprobability theory.



Measure

Pure mathematical approach

Probability theory is a branch of Pure Mathematics.

We start with some axioms and investigate the implications ofthose axioms; what results are we able to prove?

Probability theory is a subject in its own right, that we canstudy (and appreciate!) without concerning ourselves with anypossible applications.

We can also observe links to other areas of mathematics.



Measure








Measure








Measure








Measure

Modelling approach

Probability theory gives us a mathematical model forexplaining chance phenomena that we observe in the realworld.

As with any mathematical theory, it may not describe realityperfectly, but it can still be extremely useful.

We must think carefully about when and how the theory canbe applied in practice.



Measure

Modelling approach






Measure

Modelling approach






Measure

This course

We will be studying probability from both perspectives.

Applications of probability theory are important andinteresting, but as a mathematician, you should also developan understanding and an appreciation of the theory in its ownright.



Measure

This course

We will be studying probability from both perspectives.

Applications of probability theory are important andinteresting, but as a mathematician, you should also developan understanding and an appreciation of the theory in its ownright.



Measure

Motivation

We are often faced with making decisions in the presence ofuncertainty.

A doctor is treating a patient, and is considering whetherto prescribe a particular drug. How likely is it that thedrug will cure the patient?

A probation board is considering whether to release aprisoner early on parole. How likely is the prisoner tore-offend?A bank is considering whether to approve a loan. Howlikely is it that the loan will be repaid?A government is considering a CO2 emissions target.What would the effect of a 20% cut in emissions be onglobal mean temperatures in 20 years’ time?



Measure

Motivation


A doctor is treating a patient, and is considering whetherto prescribe a particular drug. How likely is it that thedrug will cure the patient?A probation board is considering whether to release aprisoner early on parole. How likely is the prisoner tore-offend?

A bank is considering whether to approve a loan. Howlikely is it that the loan will be repaid?A government is considering a CO2 emissions target.What would the effect of a 20% cut in emissions be onglobal mean temperatures in 20 years’ time?



Measure

Motivation


A doctor is treating a patient, and is considering whetherto prescribe a particular drug. How likely is it that thedrug will cure the patient?A probation board is considering whether to release aprisoner early on parole. How likely is the prisoner tore-offend?A bank is considering whether to approve a loan. Howlikely is it that the loan will be repaid?

A government is considering a CO2 emissions target.What would the effect of a 20% cut in emissions be onglobal mean temperatures in 20 years’ time?



Measure

Motivation


A doctor is treating a patient, and is considering whetherto prescribe a particular drug. How likely is it that thedrug will cure the patient?A probation board is considering whether to release aprisoner early on parole. How likely is the prisoner tore-offend?A bank is considering whether to approve a loan. Howlikely is it that the loan will be repaid?A government is considering a CO2 emissions target.What would the effect of a 20% cut in emissions be onglobal mean temperatures in 20 years’ time?



Measure

Describing uncertainty

We often use verbal expressions of uncertainty (“possible”,“quite unlikely”, “very likely”), but these are often inadequateif we want to communicate with each other about uncertainty.

Consider the following example.



Measure

Describing uncertainty

We often use verbal expressions of uncertainty (“possible”,“quite unlikely”, “very likely”), but these are often inadequateif we want to communicate with each other about uncertainty.




Measure

Disease test

You are being screened for a particular disease. You are toldthat the disease is “quite rare”, and that the screening test isaccurate but not perfect.

If you have the disease, the test will “almost certainly” detectit, but if you don’t have the disease, there is “a small chance”the test will mistakenly report that you have it anyway.

The test result is positive. How certain are you that you reallyhave the disease?



Measure

Disease test






Measure

Disease test






Measure

Quantifying uncertainty

Clearly, in this example and in many others, it would be usefulif we could quantify our uncertainty.

In other words, we would like to measure how likely it is thatsomething will happen, or how likely it is that some statementabout the world turns out to be true.



Measure

Quantifying uncertainty

Clearly, in this example and in many others, it would be usefulif we could quantify our uncertainty.

In other words, we would like to measure how likely it is thatsomething will happen, or how likely it is that some statementabout the world turns out to be true.



Measure

Quantifying uncertainty continued

We will introduce a theory for measuring uncertainty:probability theory.

We have two ingredients: set theory, which we will use todescribe what sort of things we want to measure ouruncertainty about, and, appropriately, measure theory, whichsets out some basic rules for how to measure things in asensible way.



Measure

Quantifying uncertainty continued

We will introduce a theory for measuring uncertainty:probability theory.

We have two ingredients: set theory, which we will use todescribe what sort of things we want to measure ouruncertainty about, and, appropriately, measure theory, whichsets out some basic rules for how to measure things in asensible way.



Measure

Motivation - the need for set theory and measures

If you have studied probability at GCSE or A-level, you mayhave seen a definition of probability like this:

Suppose all the outcomes in an experiment are equally likely.

The probability of an event A is defined to be

P(A) :=number of outcomes in which A occurs

total number of possible outcomes. (1)



Measure









Measure









Measure

Example

Example

A deck of 52 playing cards is shuffled thoroughly. What is theprobability that the top card is an ace?

The ‘definition’ can work for examples such as this (with afinite number of equally likely outcomes), but we soon run intodifficulties:



Measure

Example

Example

A deck of 52 playing cards is shuffled thoroughly. What is theprobability that the top card is an ace?

The ‘definition’ can work for examples such as this (with afinite number of equally likely outcomes), but we soon run intodifficulties:



Measure

Harder example

Example

Imagine generating a random angle between 0 and 2π radians,where any angle is equally likely.

(For example, consider spinning a roulette wheel, andmeasuring the angle between the horizontal axis (viewing thewheel from above) and a line from the centre of the wheelthat bisects the zero on the wheel).What is the probability that the angle is between 0 and

√2

radians?



Measure

Harder example

Example

Imagine generating a random angle between 0 and 2π radians,where any angle is equally likely.(For example, consider spinning a roulette wheel, andmeasuring the angle between the horizontal axis (viewing thewheel from above) and a line from the centre of the wheelthat bisects the zero on the wheel).

What is the probability that the angle is between 0 and√

2radians?



Measure

Harder example

Example

Imagine generating a random angle between 0 and 2π radians,where any angle is equally likely.(For example, consider spinning a roulette wheel, andmeasuring the angle between the horizontal axis (viewing thewheel from above) and a line from the centre of the wheelthat bisects the zero on the wheel).What is the probability that the angle is between 0 and

√2

radians?



Measure

Harder example continued

You might guess that the answer should be√

22π

, but thedefinition in (1) causes us problems!

There are infinitely many possible angles that could begenerated, and there are infinitely many angles between 0 and√

2 radians. Clearly, we can’t divide infinity by infinity andcome up with

√2

2π.



Measure

Harder example continued

You might guess that the answer should be√

22π

, but thedefinition in (1) causes us problems!

There are infinitely many possible angles that could begenerated, and there are infinitely many angles between 0 and√

2 radians. Clearly, we can’t divide infinity by infinity andcome up with

√2

2π.



Measure

Are outcomes equally likely

It may not make much sense to think of outcomes as equallylikely:

Example

What is the probability that a team from Manchester will winthe Premier League this season?

We will resolve this by constructing a more general definitionof probability, using the tools of set theory and a special typeof function called a measure, and we’ll show how choosingdifferent types of measure will give us sensible answers in boththe above examples.



Measure



Example





Measure



Example





Measure

Set theory and probability

Set theory is covered in more detail in MAS110; in this modulewe consider set theory in the context of probability.

We consider uncertainty in the context of an experiment.

(Here we use the word experiment in a loose sense to meanobserving something in the future, or discovering the truestatus of something that we are currently uncertain about.)



Measure







Measure







Measure

Sample space

In an experiment, suppose we want to consider how likelysome particular outcome is.

We might start by considering what all the possible outcomesof the experiment are. We can use a set to list all the possibleoutcomes.

Definition

A sample space is a set which lists all the possible outcomesof an ‘experiment’.



Measure

Sample space



Definition




Measure

Sample space



Definition




Measure

Universal set

In set theory, we sometimes work with a universal set S ,which lists all the elements we wish to consider for thesituation at hand.

(Note that, in spite of the word “universal”, the choice of Sdepends on context.)

In the context of probability, the sample space will play therole of the universal set S .



Measure

Universal set






Measure

Universal set






Measure

Examples

Example

Examples of sample spaces



Measure

Events

Definition

An event is a subset of a sample space.

If the outcome of the experiment is a member of the event, wesay that the event has occurred.

Example

Examples of events



Measure

Events

Definition

An event is a subset of a sample space.If the outcome of the experiment is a member of the event, wesay that the event has occurred.

Example

Examples of events



Measure

Events

Definition

An event is a subset of a sample space.If the outcome of the experiment is a member of the event, wesay that the event has occurred.

Example

Examples of events



Measure

Set operations and events

Given a sample space S and two subsets/events A and B , theset operations union, intersection, complement and differenceall define further events.



Measure

Union

The union A ∪ B corresponds to either A occurring or to Boccuring (or to both occurring).

We can visualise this using a Venn diagram. The rectanglerepresents the universal set, the two circles represent thesubsets A and B , and the shaded area represents the set A∪B .



Measure

Union

The union A ∪ B corresponds to either A occurring or to Boccuring (or to both occurring).

We can visualise this using a Venn diagram. The rectanglerepresents the universal set, the two circles represent thesubsets A and B , and the shaded area represents the set A∪B .



Measure

Intersection

The intersection A ∩ B corresponds to both A and Boccurring.



Measure

Empty set

If there are no elements in S that are both in A and in B , thenA ∩ B = ∅.

In this case we say that A and B are mutually exclusive.



Measure

Empty set

If there are no elements in S that are both in A and in B , thenA ∩ B = ∅.

In this case we say that A and B are mutually exclusive.



Measure

Complement

The complement of A, which we will write A, corresponds toA not occurring.

(Alternative notation: A is also written as AC and A′.)

Note that ∅ = S , and that the event ∅ is one which cannothappen, as it contains no elements.



Measure

Complement






Measure

Complement






Measure

Set difference

The set difference A \ B corresponds to A occurring but Bnot.

Note thatA \ B = A ∩ B .



Measure

Set difference

The set difference A \ B corresponds to A occurring but Bnot.

Note thatA \ B = A ∩ B .



Measure

De Morgan’s Laws

Theorem

(De Morgan’s Laws) For any two sets A and B,

(A ∪ B) = A ∩ B ,

(A ∩ B) = A ∪ B .

Proof: an exercise in MAS110.



Measure

Examples

Example

Combinations of events



Measure

The difference between an outcome and an event

Let’s clarify the difference between an “outcome” and an“event”.

An outcome is an element of the sample space S , whereas anevent is a subset of S .




Measure







Measure







Measure

Spain v. Germany

Suppose Spain play Germany at football this evening.

The possible outcomes here can be thought of as ordered pairsof non-negative integers, e.g. (2, 1) representing the outcomethat Spain win by 2 goals to 1.

So the sample space is the set of such pairs, N0 × N0.



Measure

Spain v. Germany






Measure

Spain v. Germany






Measure

Spain v. Germany continued

Before the match, you can bet on an event, eg “EitherSpain win or draw”.

Tomorrow, a news reporter will tell you the outcome, eg“Spain won 2-1”. The news reporter would not report theevent “either Spain won or the match was drawn lastnight”.

If the observed outcome belongs to the event “eitherSpain win or draw”, the event has occurred, and you havewon your bet.

An event could be a single outcome (you could bet on“Spain win 2-1”).



Measure








Measure








Measure








Measure

Notation

For a sample space S , outcome a and event A, we would usethe notation a ∈ S and A ⊂ S . Make sure you understandwhen to use the symbol ∈ and when to use the symbol ⊂ or ⊆.



Measure

Set functions

We can define set functions that take sets as their inputs andproduce real numbers as outputs.

Not all set functions make sense for all possible sets, so whendefining a set function, we’ll first say what the sample spaceis, and then define set functions that operate on subsets of asample space.



Measure

Set functions

We can define set functions that take sets as their inputs andproduce real numbers as outputs.

Not all set functions make sense for all possible sets, so whendefining a set function, we’ll first say what the sample spaceis, and then define set functions that operate on subsets of asample space.



Measure

Set functions continued

Example

Examples of set functions

Formally, we can think of a set function as a function whosedomain is a set whose elements are subsets of S .



Measure

Set functions continued

Example

Examples of set functions

Formally, we can think of a set function as a function whosedomain is a set whose elements are subsets of S .



Measure

Measures

We’re close to being able to give a formal definition ofprobability: we’re going to define probability as a special typeof set function that we apply to a set.

We first define a measure as a special type of set function.



Measure

Measures

We’re close to being able to give a formal definition ofprobability: we’re going to define probability as a special typeof set function that we apply to a set.

We first define a measure as a special type of set function.



Measure

Definition

Definition

Given a sample space (or universal set) S , a (finite) measureis a set function that assigns a non-negative real number m(A)to a set A ⊆ S , which satisfies the following condition:

For any two disjoint subsets A and B of S (ie A ∩ B = ∅),

m(A ∪ B) = m(A) + m(B). (2)



Measure

Definition

Definition

Given a sample space (or universal set) S , a (finite) measureis a set function that assigns a non-negative real number m(A)to a set A ⊆ S , which satisfies the following condition:For any two disjoint subsets A and B of S (ie A ∩ B = ∅),

m(A ∪ B) = m(A) + m(B). (2)



Measure

Intuition

Informally, the above definition can be put into words asfollows.

A measure is something which assigns a number to a set,produces a non-negative number as its output, and ‘behavesadditively’:

If we take two non-overlapping (disjoint) sets, we can eithercombine the sets and then apply the measure, or apply themeasure to each set and then sum the results, and we will getthe same value either way.



Measure

Intuition






Measure

Intuition






Measure

Finite unions

We can extend the condition in (2) to finite unions of disjointsets.

If three sets A, B and C are all mutually disjoint(A ∩ B = A ∩ C = B ∩ C = ∅), if we define D = B ∪ C , then

m(A ∪ B ∪ C ) = m(A ∪ D)

= m(A) + m(D)

= m(A) + m(B ∪ C )

= m(A) + m(B) + m(C ).



Measure

Finite unions



m(A ∪ B ∪ C ) = m(A ∪ D)

= m(A) + m(D)

= m(A) + m(B ∪ C )

= m(A) + m(B) + m(C ).



Measure

Finite unions



m(A ∪ B ∪ C ) = m(A ∪ D)

= m(A) + m(D)

= m(A) + m(B ∪ C )

= m(A) + m(B) + m(C ).



Measure

Finite unions



m(A ∪ B ∪ C ) = m(A ∪ D)

= m(A) + m(D)

= m(A) + m(B ∪ C )

= m(A) + m(B) + m(C ).



Measure

Finite unions



m(A ∪ B ∪ C ) = m(A ∪ D)

= m(A) + m(D)

= m(A) + m(B ∪ C )

= m(A) + m(B) + m(C ).



Measure

Examples

Example

Examples of measures



Measure

Properties of measuresTheorem

(Properties of measures)Let m be a measure on some set S, with A,B ⊆ S.

(M1) If B ⊆ A, then

m(A \ B) = m(A)−m(B).

More generally,m(A \ B) = m(A \ (A ∩ B)) = m(A)−m(A ∩ B).

(M2) If B ⊆ A, thenm(B) ≤ m(A).

(M3) m(∅) = 0



Measure




m(A \ B) = m(A)−m(B).



(M3) m(∅) = 0



Measure




m(A \ B) = m(A)−m(B).



(M3) m(∅) = 0



Measure




m(A \ B) = m(A)−m(B).



(M3) m(∅) = 0



Measure




m(A \ B) = m(A)−m(B).



(M3) m(∅) = 0



Measure

Properties of measures continued

Theorem

(Properties of measures)

(M4) m(A ∪ B) = m(A) + m(B)−m(A ∩ B).

(M5) For any constant c > 0, the function g(A) = c ×m(A) isalso a measure.

(M6) If m and n are both measures, the functionh(A) = m(A) + n(A) is also a measure.



Measure


Theorem


(M4) m(A ∪ B) = m(A) + m(B)−m(A ∩ B).





Measure


Theorem


(M4) m(A ∪ B) = m(A) + m(B)−m(A ∩ B).





Measure

Measures on partitions of sets

Definition

A partition of a set S is a collection of setsE = {E1,E2, . . . ,En}

such that

1 Ei ∩ Ej = ∅, for all 1 ≤ i , j ≤ n with i 6= j ;

2 E1 ∪ E2 ∪ . . . ∪ En = S .

We say that {E1,E2, . . . ,En} are mutually exclusive andexhaustive.If S is a sample space, one of the events in E must occur, butno two events can occur simultaneously.



Measure


Definition

A partition of a set S is a collection of setsE = {E1,E2, . . . ,En} such that


2 E1 ∪ E2 ∪ . . . ∪ En = S .




Measure


Definition



2 E1 ∪ E2 ∪ . . . ∪ En = S .




Measure


Definition



2 E1 ∪ E2 ∪ . . . ∪ En = S .

We say that {E1,E2, . . . ,En} are mutually exclusive andexhaustive.

If S is a sample space, one of the events in E must occur, butno two events can occur simultaneously.



Measure


Definition



2 E1 ∪ E2 ∪ . . . ∪ En = S .




Measure

Partitions continued

From (2), we have

m(S) =n∑

i=1

m(Ei).


Documents

MAS113 Introduction to Probability and Statistics · Introduction Set theory and probability Measure MAS113 Introduction to Probability and Statistics Dr Jonathan Jordan School of