View
227
Download
0
Category
Preview:
Citation preview
8/18/2019 Probability 2013 Ch01 Axioms of Probability
1/28
Probability
Chapter Outline
Chapter one
xioms of probability
1.1 Introduction
1.2 Sample space and events1.3 Axioms of probability
1.4 Basic Theorems
1.5 Continuity of probability function
1.6 Probabilities 0 and 1
1.7
Random
selection
of
points
from
intervals
8/18/2019 Probability 2013 Ch01 Axioms of Probability
2/28
Probability
1.1 Introduction
Event: a set of outcomes (to which a probability is assigned)
– Event may or may not occur Random event
– Occurrence of event is inevitable Certain event
– Event never occur Impossible event Probability theory aims at determining the exact value, or an
estimate of the chance of occurrence of a random event
Humans have been interested in games of chance and gambling
– 4‐sided die: Ancient Egypt; 3500 B.C.
– 6‐sided die: 1600 B.C.
– Dice: China; 7th‐10th centuries
– Playing cards: Much more recent
Advent of probability as a math discipline is relatively recent
Chien-Chao Tseng 2
8/18/2019 Probability 2013 Ch01 Axioms of Probability
3/28
Probability
Early Era of Probability Theory
Studies of Chances of Events (begins at fifteen century)
Italian scholars tried to construct a math foundation for Prob.
– Luca Paccioli (1445‐1514)
– Niccolo Tartaglia (1499‐1557) – Girolamo Cardano (1501‐1576)
– Galileo Galielei (1564‐1642)
Real progress (started in France) (in 1654) General methods for the calculation of probabilities
– Blaise Pascal (1623‐1662)
– Pierre de Fermat (1601‐1665)
First book on probability (in 1657 )
– Christian Huygens (1629‐1695) (Dutch)
– “On Calculations in Games of Chance"
Chien-Chao Tseng 3
8/18/2019 Probability 2013 Ch01 Axioms of Probability
4/28
Probability
Growth of Probability Theory
Major breakthrough – James Bernoulli (1654‐1705) (book published in 1917) – Abraham de Moivre (1667‐1754) (book published in 1930)
Growth of probability and applications in various directions – Pierre‐Simon Laplace (1749‐1827) – Simeon Denis Poisson (1781‐1840) – Karl Friedrich Gauss (1777‐1855)
Advance of probability theory (19th century)
Russian mathematicians advanced works of Laplace, DeMoivre, and Bernoulli
– Pafnuty Chebyshev (1821‐1894) – Andrei Markov (1856‐1922) – Aleksandr Lyapunov (1857‐1918)
Chien-Chao Tseng 4
8/18/2019 Probability 2013 Ch01 Axioms of Probability
5/28
Probability
Relative Frequency Interpretation of Probability
By the early 20th century, prob. is already a developed theory – But, foundation was not firm
Relative Frequency Interpretation (most satisfactory that time)
Relative frequency of occurrences as the probability p of the occurrence of an event A of an experiment
To define p, we – study a series of sequential or simultaneous performances
of the experiment and – observe the proportion of times that A occurs approaches a
constant.
• : number of times A occurs during performances of the experiment
This definition is mathematically problematic
Cannot be the basis of a rigid probability theoryChien-Chao Tseng 5
lim→
/
8/18/2019 Probability 2013 Ch01 Axioms of Probability
6/28
Probability
Problem with Relative Frequency Interpretation
1) In practice, lim→
/ cannot be computed and no
way to analyze the error of the approximation
2) No reason to believe the limit of ⁄ , as → ∞ exists.
Furthermore, if we accept it exists many dilemmas arise
E.g., uniqueness of the probability is not guaranteed
Cannot justify ⁄ approaches the same limit in a
different series of experiments.
3) Probabilities that based on our personal belief and knowledge is not justifiable
Statements such as the following would be meaningless.• “The probability that the price of oil will be raised in the
next six months is 60%.”
• “The probability that it will snow next Christmas is 30%.”
Chien-Chao Tseng 6
8/18/2019 Probability 2013 Ch01 Axioms of Probability
7/28
Probability
Firm Mathematical Grounds
Axiomatic treatment of the theory of probability (in 1900)
Pointed out by David Hilbert (1862‐1943) Crucial to the advance of probability
Firm mathematical foundation of probability Some work toward this goal done by
– Émile Borel (1871‐1956), Serge Bernstein (1880‐1968),
Richard von Mises (1883‐1953) Successful creation of the axioms of probability (in 1933) by
– Russian mathematician Andrei Kolmogorov (1903‐1087) – Took 3 self ‐evident and indisputable properties as axiomsEntire theory of probability is developed and rigorously
based on axioms
Chien-Chao Tseng 7
• “Existence of a constant p as → ∞" is shown
•Subjective probabilities may be modeled and studied
8/18/2019 Probability 2013 Ch01 Axioms of Probability
8/28
Probability
1.2 Sample space and events
Experiment: “Tossing a die,” …
– Outcome of an experiment is not certain, but – All of the possible outcomes are predictable
Sample Space: set of all possible outcomes (denoted as S) Sample points (or points) of the sample space:
– Possible outcomes of the experiment
E.g., 1.1 Tossing a coin once: sample space S = {H, T} E.g., 1.2 Flipping a coin and
– tossing a die if T or flipping a coin again if H
S={T1, T2, T3, T4, T5, T6, HT, HH}
Events: certain subsets of sample space S
– Set of points of the sample space
an event E occurs if outcome of experiment belongs to E
Chien-Chao Tseng 8
8/18/2019 Probability 2013 Ch01 Axioms of Probability
9/28
Probability
Relations of Events (of an Experiment) (1/3)
Subset
event E is said to be a subset of event F if, whenever E occurs, F also occurs.
– all sample points of E are contained in F . – in set‐theory, E ⊆ F .
Equality
Events E and F are said to be equal if occurrence of E implies
occurrence of F , and vice versa;
– that is, if E ⊆ F and F ⊆ E , hence E = F .
Intersection
An event is called the intersection of two events E and F if it occurs only whenever E and F occur simultaneously.
– In set‐theory, this event denoted by EF or E ∩ F
Chien-Chao Tseng 9
8/18/2019 Probability 2013 Ch01 Axioms of Probability
10/28
8/18/2019 Probability 2013 Ch01 Axioms of Probability
11/28
8/18/2019 Probability 2013 Ch01 Axioms of Probability
12/28
Probability
Useful Relations
= , ∪ = , and = ∅
Commutative laws: E UF = F UE, EF = FE
Associative laws: EU(FUG)=(EUF)UG
Distributive laws: (EF)UH=(EUH)(FUH), (EUF)H=(EH)U(FH) De Morgan’s 1st law: (EUF)c = EcFc
⋃
= ⋂
, ⋃
= ⋂
De Morgan’s 2nd law: (EF)c = EcUFc
⋂
= ⋃
, ⋂
= ⋃
∪ : ∪ = ∪
Elementwise method for identity proof : – Events on both sides formed by the same sample points – Prove set inclusion in both directions
• E.g.,
=
if
and
Chien-Chao Tseng 12
Distributive Law
E E C
E
F C
F EF
EF C
8/18/2019 Probability 2013 Ch01 Axioms of Probability
13/28
Probability
1.3 Axioms of Probability
If a result is false, a counterexample exists to refuse it If a result is valid, a proof must be found.
Venn diagrams (sec. 1.2): to give intuitive justification, or to
create counterexample and show invalidity of relations Proofs in probability theory: by axiomatic method
1) Adopt certain simple, indisputable, and consistent statements without justifications (axioms or postulates).
2) Agree on how and when one statement is a logical consequence of another one
3) Obtain new results (theorems) using the terms already
clearly understood, axioms and definitions Theorems are statements that can be proved.
– Can be used for discovery of new theorems,
– Process
continues
and
a
theory
evolves.Chien-Chao Tseng 13
8/18/2019 Probability 2013 Ch01 Axioms of Probability
14/28
Probability
Axioms of probability
Definition (Probability Axioms)
– S: a sample space of a random phenomenon.
– A: an event of S
– P : a function for each event A, i.e., P : 2S R (or (S) R) P ( A): a number associated with A
If P satisfies the following axioms,
– P is
called
a
probability
and
– P ( A) is said to be the probability of A
Axiom 1: P ( A) 0
Axiom 2: P (S) = 1
Axiom 3: If { A1, A2, A3, …} is a sequence of mutually exclusive events ( = ∅ when ) then
Chien-Chao Tseng 14
)()(11
i iii AP AP
8/18/2019 Probability 2013 Ch01 Axioms of Probability
15/28
Probability
Equally Likely, Empty Set
Equally likely
– Let S be the sample space of an experiment.
– Let A and B be events of S.
– A and B are equally likely if P(A) = P(B).
Theorem 1.1 Probability of empty set ∅ is 0. (P (∅) = 0.)
Proof:
– Let A1 = S and Ai = ∅ for i ≥ 2; , , , . . . is a sequence of mutually exclusive events.
– By Axiom 3,
⋃
∑ ∑ ∅
Implying that ∑ ∅ 0 ∅ 0
Chien-Chao Tseng 15
Axiom 3
8/18/2019 Probability 2013 Ch01 Axioms of Probability
16/28
Probability
Finite Additivity (Theorem 1.2)
Axiom 3 is stated for a countably infinite
collection of mutually exclusive events.
Axiom 3 is also called the axiom of countable additivity .
Theorem 1.2 (Finite
Additivity .) – Let { , , . . . , } be a mutually exclusive set of events.
– Then ⋃ ∑
Proof : – For i , let Ai = ∅.
, , , . . .: a sequence of mutually exclusive events.
By Axiom 3 and Theorem 1.1, ⋃
= ⋃
= ∑
= ∑ ∑
∑
Chien-Chao Tseng 16
Axiom 3
∅ 0
8/18/2019 Probability 2013 Ch01 Axioms of Probability
17/28
Probability
Implication of Theorem 1.2
By theorem 1.2,
if and are two mutually exclusive events
∪ = + (1.2)
Implication of equation 1.2: for any event A, 1
Proof :
– ∪
=
+
– By Axiom 2,
∪ = = 1
+ = 1
For any event A, 0 1
Chien-Chao Tseng 17
Axiom 2
8/18/2019 Probability 2013 Ch01 Axioms of Probability
18/28
Probability
Probability of Equal Likely Outcomes
– Let S be the sample space of an experiment.
– If S has N points that are all
equally
likely
to occur
Probability of each outcome (sample point) is 1 /N .
Inference: – Let S = {s1 , s2 , . . . , sN} the sample space of an experiment;
– All sample points are equally likely to occur,
P({s1}) = P({s2})= ∙ ∙ ∙ = P({sN}) – P(S) = 1, and {s1}, {s2}, …, {sN} are mutually exclusive
1 = P(S) = P({s1, s2, ∙ ∙ ∙, sN})
= P({s1}) + P({s2}) + ∙ ∙ ∙ +P({sN})
= NP({sN})
P({sN}) = 1/N
Chien-Chao Tseng 18
8/18/2019 Probability 2013 Ch01 Axioms of Probability
19/28
Probability
Theorem 1.3 Classical Definition of Probability
– Let S be the sample space of an experiment.
– If S has N points that are all
equally
likely
to occur
For any event A of S ,
P( A) = N( A)/N, where N( A) is the number of points of A. Proof:
– Let S = {s1 , s2 , . . . , sN}, is an outcome of the experiment
– Equiprobable
P({sN})
=
1/N for
all
i ,
1 ≤
i ≤
N. – Let A = { , , . . . , }, where , ∈ S for all .
– { , , . . . , } are mutually exclusive
P( A) = P({
,
, . . . ,
})
= P({}) + P({}) + ∙ ∙ ∙ +P({})
= 1/N+1/N + .. +1/N = N( A)/N
Chien-Chao Tseng 19 N ( A) terms
8/18/2019 Probability 2013 Ch01 Axioms of Probability
20/28
Probability
1.4 Basic Theorems
Theorem 1.4 = 1 – A, mutually exclusive
1 ∪
Theorem
1.5 If
A
B,
then
P(B A) = P(B ) = P(B) P( A)
Proof :
– A BB = (B A) ∪ A
– Events (B A) and A are mutually exclusive (∵ (B A) A = ∅)
P(B) = P((B A) ∪ A) = P(B A) + P( A)
P(B A) = P(B) P( A)
Corollary If A B , then P( A) P(B)
Theorem 1.6 P( A ∪ B) = P( A) + P(B) P( AB)
Theorem
1.7
P( A)
=
P( AB)
+
P( A
)Chien-Chao Tseng 20
8/18/2019 Probability 2013 Ch01 Axioms of Probability
21/28
Probability
Proof of Theorem 1.6
Theorem 1.6 P( A ∪ B) = P( A) + P(B) P( AB) Proof :
̶ A ∪ B = A ∪ (B AB)
̶ A and (B AB) are mutually exclusive P( A ∪ B ) = P( A ∪ (B AB))
= P( A ) + P(B AB)
– By Theorem 1.5,
AB ⊆ B P(B AB) = P(B) P( AB)
P( A ∪ B )
=
P( A
) + P(B)
P( AB)
Chien-Chao Tseng 21
8/18/2019 Probability 2013 Ch01 Axioms of Probability
22/28
Probability
Inclusive‐Exclusive Principle (1/2)
Three Events:
P( ∪∪) = P( )+P( )+P( )
P( )P( )P( )
+P( )
Proof :
̶ By associative laws: P(
∪
∪
) = P(
∪
∪
)
̶ By Theorem 1.6 and distributive Laws:
̶ P( ∪∪) = P(( ∪)∪)
= P( )+P( )P( )+P( )P( ∪ )= P( )+P( )P( )+P( )
P( )P( )+P( )
Chien-Chao Tseng 22
8/18/2019 Probability 2013 Ch01 Axioms of Probability
23/28
Probability
Inclusive‐Exclusive Principle (2/2)
Four Events:
P( ∪∪∪)
= P( )+P( )+P( )+P( )
P( )P( )P( )P( )P( )P( )
+P( )+P( )+P( ) +P( )
P( )
Inclusive‐Exclusive Principle ( events)
⋃ ∑
∑ ∑
+ ∑ ∑ ∑
⋯
+ 1 ⋯
Chien-Chao Tseng 23
8/18/2019 Probability 2013 Ch01 Axioms of Probability
24/28
Probability
Theorem 1.7
Theorem 1.7 P( A) = P( AB) + P( A) Proof :
̶ A = AS = A ∩ (B ∪ ) = AB ∪ A
̶ AB and A are mutually exclusive P( A) = P( AB ∪ A) = P( AB) + P( A)
Chien-Chao Tseng 24
8/18/2019 Probability 2013 Ch01 Axioms of Probability
25/28
Probability
1.5 Continuity of probability function
Recall: probability is a set function from (S) to [0 , 1].– (S), power set (set of all possible events) of S.
A sequence events { , ≥ 1} of a sample space is called
– increasing if ⊆ ⊆ ⊆ ∙ ∙ ∙ ⊆ ⊆ ∙ ∙ ∙ ; – decreasing if ⊇ ⊇ ⊇ ∙ ∙ ∙ ⊇ ⊇ ⊇ ∙ ∙ ∙ .
For an increasing sequence of events { , ≥ 1}, – lim
→
: event that at least one , 1 ≤
8/18/2019 Probability 2013 Ch01 Axioms of Probability
26/28
Probability
Theorem 1.8 Continuity of Probability Function
– For any increasing or decreasing seq. of events, { , ≥ 1}
lim→
) = lim→
roof:
– ’s: circular disks
– ’s: shaded circular annuli, except .
• ⋃ = ⋃
= , 1, 2, 3, ⋯,
⋃ = ⋃
= lim
→
– lim→
= ⋃ = ⋃
= ∑
= lim→ ∑ = lim→ ⋃
= lim→ ⋃
= lim→
Chien-Chao Tseng 26
Referred to textbook for the proof for decreasing sequence
Mutually exclusive
Mutually exclusive
Mutually exclusive
8/18/2019 Probability 2013 Ch01 Axioms of Probability
27/28
Probability
1.6 Probabilities 0 and 1
Misinterpretation of events with probabilities 1 and 0.
– If event E with probabilities 1 then it is the sample space S.
– If event F with probabilities 0 then it is the empty set ∅.
In some experiments, there exist infinitely many events – each with probability 1, and
– each with probability 0.
Ex. (Experiment: Random selection of points from (0,1)) – Probability of the occurrence of any particular point is 0.
(Can be shown by Continuity of Probability, page 29, 30)
– For a point t ∈ (0 , 1), let = (0 , 1) − {t }.
• P({t }) = 0, but {t } ∅
• P() = P() 1 1, but S (0 , 1).
– Infinitely many events {t }’s and ’s exist
Chien-Chao Tseng 27
8/18/2019 Probability 2013 Ch01 Axioms of Probability
28/28
Probability
1.7 Random selection of points from intervals
Definition: A point is said to be randomly selected from an interval (a, b):
– if any two subintervals of (a, b) that have the same length
are
equally likely to
include
the
point.
Probability that the subinterval (, ) contains the point is defined to be ( )/(ba).
(Sketch of Proof: divide interval into (ba)/ ( )
subintervals, apply Axiom 2 and Thm 1.2)
In a random selection of points from intervals,
– Probability of the occurrence of any particular point is 0.
If [α , β] ⊆ (a, b) , events that the point falls in [α , β] , (α , β) , [α , β), and (α , β] are all equiprobable.
– Probability of the occurrence of a finite or countablyinfinite set is 0.
Chien-Chao Tseng 28
Recommended