Upload
brick
View
41
Download
0
Embed Size (px)
DESCRIPTION
C HAPTER 2 Basic Concepts of Probability Theory. Prof. Sang-Jo Yoo [email protected] http://multinet.inha.ac.kr. Random Experiments. Random Experiments An experiment in which the outcome varies in an unpredictable fashion when the experiment is repeated under the same conditions - PowerPoint PPT Presentation
Citation preview
INHA UniversityINHA Universityhttp://multinet.inha.ac.krhttp://multinet.inha.ac.kr
Multimedia Network Lab.Multimedia Network Lab.
CHAPTER 2
Basic Concepts of Probability Theory
Prof. Sang-Jo [email protected]
http://multinet.inha.ac.kr
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Random Experiments Random Experiments
An experiment in which the outcome varies in an unpredictable fashion when the experiment is repeated under the same conditions
The specification of a random experiment must include an unambiguous statement of exactly what is measured or observed.
Example 2.1 E1: Select a ball from an urn containing balls numbered 1 to 50 E2: Select a ball from an urn containing balls numbered 1 to 4. Balls 1 & 2 are black and balls 3 & 4 are white. Note number
and color. E3: Toss a coin three times and note the sequence of heads and tails E4: Toss a coin three times and note the number of heads E7: Pick a number at random between zero and one E12: Pick two numbers at random between zero and one E13: Pick a number X at random between zero and one, then pick a
number Y at random between zero and X
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Sample Space Sample space of a random experiment
The set of all possible outcomes Outcomes(or sample points) are mutually exclusive in the sense that they cannot
occur simultaneously. When we perform a random experiment, one and only one outcome occurs.
A sample space can be finite, countably infinite or uncountably infinite. Example 2.2
E1: Select a ball from an urn containing balls numbered 1 to 50 E2: Select a ball from an urn containing balls numbered 1 to 4. Balls 1 & 2 are black and balls 3 & 4 are white. Note the number and color. E3: Toss a coin three times and note the sequence of heads and tails E4: Toss a coin three times and note the number of heads E7: Pick a number at random between zero and one E12: Pick two numbers at random between zero and one E13: Pick a number X at random between zero and one, then pick a number Y at
random between zero and X
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Sample Space Toss a dice until a ‘six’ appears and count the number of times
the dice was tossed. S = {1, 2, 3, …}; S is discrete and countably infinite (one-to-one correspondence
with positive integers)
Pick a number X at random between zero and one, then pick a number Y at random between zero and X.
S = {(x, y): 0 ≤ y ≤ x ≤ 1}; S is a continuous sample space.
4
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Events Events
Outcomes that satisfies certain conditions A subset of S Certain event consists of all outcomes, always occurs Null event contains no outcomes, never occurs Event class is the events of interests: set of sets
Example 2.3 E1: “An even-numbered ball is selected,” A1= ? E2: “The ball is white and even-numbered,” A2= ? E3: “The three tosses give the same outcome,” A3= ? E4: “The number of heads equals the number of tails,” A4= ? E7: “The number selected is non-negative,” A5= ? E12: “The two numbers differ by less than 1/10,” A6= ?
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Review of Set Theory Terminologies
U: universal set consists of all possible objects Objects are called the elements or points of the set Subset of B : A B Empty set: Equal: A=B Union: AB={x: xA or x B} Intersection: AB={x: xA and x B} Disjoint (mutually exclusive): if AB= Complement: Ac={x: xA} Relative complement (difference): A-B={x: xA and xB}
Notations
6
n
n
kkn
n
kk AAAAandAAAA
121
121
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Set OperationsUnion
A B
A B(a)
A B
A B(b)
Intersection
Complement
Ac(c)
A B
A B = (d)
Mutuallyexclusive
A
Ac
A
B
A B (e)
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Properties of Set Operations Commutative properties
Associative properties
Distributive properties
DeMorgan’s Rules
A B B A and A B B A
and ( ) ( )A B C A B C
( ) ( ) ( )A B C A B A C
( )C C CA B A B ( )C C CA B A B
( ) ( )A B C A B C
and
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Properties of Set Operations De Morgan’s rules
(A ∩ B)c = Ac ∪ Bc and (A ∪ B)c = Ac ∩ Bc
Proof of the second rule: Suppose x ∈ (A ∪ B)c
⇔ x is not contained in any ofthe events A and B⇔ x is contained in Ac and Bc
⇔ x ∈ Ac ∩ Bc.
Proof of the first rule: Based on the second rule, take A → Ac and B → Bc, we then have
(Ac ∪ Bc)c = A ∩ B. Taking complement on both sides, we obtain the first rule.
9
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
The Axioms of Probability Axioms 1 :
Axioms 2 :
Axioms 3 :
Axioms 3’ :
0 [ ]P A
[ ] 1P S
If , then [ ] [ ] [ ]A B P A B P A P B
1 2If , , is a sequenceof eventssuch thatA A
for all , theni jA A i j
11
[ ]k kkk
P A P A
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Properties of Probability Corollary 1 : Corollary 2 : Corollary 3 :
Corollary 4 :
Corollary 5 :
Corollary 6 :
Corollary 7 :
[ ] 1 [ ]cP A P A
[ ] 1P A
1 2If , , , are pairwise mutuallyexclusive,nA A A
11
then [ ]for 2n n
k kkk
P A P A n
[ ] [ ]P A B P A P B P A B
If A B, then [ ] [ ]P A P B
[ ] 0P
kjn
nkj
n
jj
n
kk AAPAAPAPAP ].[)1(][][ 1
1
10
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Discrete Sample Space Example 2.9
An urn contains identical balls numbered 0.1,…,9. Randomly select a ball from the urn
A = “number of ball selected is odd,”
B = “number of ball selected is a multiple of 2,”
C = “number of ball selected is less than 5,”
Find [ ], [ ], [ ], [ ], and [ ].P A P B P C P A B P A B C
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Continuous Sample Space Continuous sample space
Outcome are numbers. The events consist of intervals of the real line or rectangular
regions in the plane.
Example 2.12: Pick a number between zero and one Sample space S = ?
The probability that the outcome falls in a subinterval of A = the length of the subinterval
A = the event when the outcome is at least 0.3 away from the center of the unit interval. P [A] = ?
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Conditional Probability Conditional probability
To determine whether two events A and B are related in the sense that knowledge about the occurrence of B alters the likelihood of occurrence of A.
[ ]| for [ ] 0[ ]
/ [ ]/ [ ]
A B A B
B B
P A BP A B P BP B
n n n P A Bn n n P B
A
B
A B
S
[ ] | [ ]
[ ] | [ ]
P A B P A B P B
P A B P B A P A
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example: Conditional Probability Example 2.24
A ball is selected from an urn containing two black balls (1, 2) and two white balls (3, 4). The sample space is S ={(1, b), (2, b), (3, w), (4, w)}. Assuming that the outcome are equally likely, find P [A|B], P [A|C], where A, B, and C are the following events:
A = {(1, b), (2, b), (3, w), (4, w)}, “black ball selected” B = {(1, b), (2, b), (3, w), (4, w)}, “even-numbered ball selected’” C = {(1, b), (2, b), (3, w), (4, w)}, “number of ball is greater than 2”
Example Tossing 2 dice Suppose the first die is ‘3’; given this information, what is the probability
that the sum of the 2 dice equals 8? There are 6 possible outcomes, given the first dice is ‘3’:
(3, 1), (3, 2), (3, 3), (3, 4), (3, 5), and (3, 6). The outcomes are equally probable. Hence, the probability that the sum is
8, given the first dice is ‘3’, is 1/6 .“” Conditional Probability Note
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Theorem On Total Probability Let B1 , B2 , …, Bn be mutually exclusive events whose union
equals the sample space S. Any event can be represented as the union of mutually exclusive events:
1 2 1
1 2
1 1 2 2
( ) ( ) ( )[ ] [ ] [ ] [ ][ ] [ | ] [ ] [ | ] [ ] [ | ] [ ]
n n
n
n n
A A S A B B B A B A BP A P A B P A B P A BP A P A B P B P A B P B P A B P B
B1B3
B2
A
Bn-1
Bn…
…
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example: Total Probability Example 2.28
A manufacturing process produces a mix of “good” memory chips and “bad” memory chips. The lifetime of good chips follows the exponential law with a rate of α. The life time of bad chips also follows the exponential law, but the rate of failure is 1000α. Suppose that the fraction of good chips is 1-p and of bad chips p. Find the probability that a randomly selected chip is still functioning after t seconds.
C = the event “chip still functioning after t seconds”
G = the event “chip is good”
B = the event “chip is bad”
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example: Total Probability Example
In a trial, the judge is 65% sure that Susan has committed a crime. Person F (friend) and Person E (enemy) are two witnesses who know whether Susan is innocent or guilty.
Person F is Susan’s friend and will lie with probability 0.25 if Susan is guilty. He will tell the true if Susan is innocent.
Person E is Susan’s enemy and will lie with probability 0.30 if Susan is innocent. Person E will tell the truth if Susan is guilty.
What is the probability that Person F and Person E will give conflicting testimony?
Solution Let I and G be the two mutually exclusive events that Susan is
innocent and guilty, respectively. Let C be the event that the two witnesses will give conflicting testimony. Find P[C] based on P[C|I] and P[C|G].
By the law of total probabilities
18
.2675.065.025.035.030.0][]|[][]|[][
GPGCPIPICPCP
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Bayes’ Rule Let B1 , B2 , …, Bn be a partition of a sample space S.
Suppose that event A occurs. What is the probability of event Bj ?
1
[ ] [ | ] [ ]|
[ ] [ | ] [ ]
j j jj n
k kk
P A B P A B P BP B A
P A P A B P B
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example: Bayes’ Rule Example 2.30
A manufacturing process produces a mix of “good” memory chips and “bad” memory chips. Suppose that in order to “weed out” the bad chips, every chip is tested for t seconds prior to leaving the factory.
Find the value of t for which 99% of chips sent out to customers are good.
C = the event “chip still functioning after t seconds” G = the event “chip is good” B = the event “chip is bad”
We want to find the value of t for which
Example 2.26 & 2.29 (Note)
[ ] 0.99
[ | ] [ ][ ] [ ] [ ] [ ]
P G C
P C G P GP C G P G P C B P B
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example: Bayes’ Rule Example
A judge is 65% sure that a suspect has committed a crime. During the course of the trial, a witness convinces the judge that there is an 85% chance that the criminal is left-handed. If 23% of the population is left-handed and the suspect is also left-handed. With this new information, how certain should the judge be of the guilt of the suspect?
SolutionG = event that the suspect is guiltyI = event that the suspect is innocentL = event that the suspect is left-handed
Since {G, I} forms a partition of the sample space, by Bayes’ Theorem:
21
.87.035.023.065.085.0
65.085.0][]|[][]|[
][]|[]|[
IPILPGPGLPGPGLPLGP
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Independence of Events Event A is independent of B if knowledge of the occurrence of B
does not alter the probability of A
Three events A, B, and C are independent if
[ ] [ ]
| and |
P A B P A P B
P A B P A P B A P B
[ ] [ ] [ ]P A B C P A P B P C
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Properties of independence and mutual exclusiveness
(1) If A ∩ B = φ, then P[A|B] = 0. A can never occur if B has occurred.
(2) If A ≠ φ, B ≠ φ and A and B are independent, then A and B
are not mutually exclusive. This is becauseP[A ∩ B] = P[A]P[B] ≠ 0.
Remarks we have seen that if A and B are mutually exclusive, then A and B cannot be independent.
(3) If A and B are independent and A ∩ B = φ, then either A = φ or B = φ or both. This is because 0 = P[A ∩ B] = P[A] P[B] so that either P[A] = 0, P[B] = 0 or both are equal.
23
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example: Independent Events Example 2.31
A ball is selected from an urn containing two black balls (1, 2) and two white balls (3, 4). The sample space is S = {(1, b), (2, b), (3, w), (4, w)}. Let events A, B, and C are defined as follows:
A = {(1, b), (2, b)}, “black ball selected” B = {(2, b), (4, w)}, “even-numbered ball selected’” C = {(3, w), (4, w)}, “number of ball is greater than 2”
Are events A and B independent? Are events A and C independent? In general if two events have nonzero probability and are mutually
exclusive, then they cannot be independent.
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example: Independent Events Example Two numbers x and y are selected at random
between zero and one. Let events A, B and C be defined by
However,
25
21yA
21xB
21,
21
21,
21 yxyxC
][][41][ BPAPBAP
][][41][ CPAPCAP
][][41][ CPBPCBP
81][][][0][][
,
CPBPAPPCBAP
soCBA
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Sequential Experiment Sequential of independent experiment
Suppose that a random experiment consists of performing experiments E1, E2, …, En.
Outcome of the experiment
Sample space of the experiment
Let A1 , A2 , …, An be events such that Ak concerns only the outcome of the kth sub-experiment. If the sub-experiments are independent, then
1 2( , , , )ns s s s
1 2 nS S S
1 2 1 2[ ] [ ] [ ]n nP A A A P A P A P A
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example: Sequential Events Example 2.36
Suppose that 10 numbers are selected at random from the interval [0, 1]. Find the probability that the first 5 numbers are less than ¼ and the last 5 numbers are greater than ½.
Let x1, x2, …, x10 be the sequence of 10 numbers.
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Binomial Probability Law Bernoulli trial
Performance an experiment once and nothing whether a particular event A occurs.
Outcome = “success” if A occurs, outcome = “fail” otherwise
Binomial probability law Probability of k successes in n independent repetitions of a
Bernoulli trial
(1 )
!! !
k n kn
np k p p
k
n nk k n k
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example: Binomial Probability Law Example 2.37
Suppose that a coin is tossed three times. Assume that the tosses are independent and the probability of heads is p
P [{HHH}] = P [{HHT}] = P [{HTH}] = P [{THH}] =P [{TTH}] = P [{THT}] =P [{HTT}] = P [{TTT}] =
Let k be the number of heads in three trials
P [k=0] =P [k=1] =P [k=2] =P[ k=3] =
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Example: Binomial Probability Law Example 2.39 Voice Communication System
Let k be the number of active speakers in a group of eight independent speakers. Suppose that a speaker is active with probability 1/3.
Find the probability that the number of active speakers is greater than six.
Example 2.40: Error Correction Coding A communication system transmits binary information over a
channel that introduces random bit errors with probability =10-3. the transmitter transmits each information bit three times and a decoder takes a majority vote of the received bits to decide on what the transmitted bit was.
Find the probability that the receiver will make an incorrect decision.
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Multinomial Probability Law Let B1,B2, …, BM be a partition of the sample space S of some
random experiment and P[Bj]=Pj. The events are mutually exclusive P1+P2+…+PM=1
Suppose the n independent repetitions of the experiment are performed. Let kj be the number of time event Bj occurs, then the vector (k1,k2,
…,kM) specifies the number of times each of the event Bj occurs
Multinomial probability law M=2: Binomial probability law 1 2
1 2 1 21 2
![ , , , ]! ! !
Mk k kM M
M
nP k k k p p pk k k
INHA UniversityINHA University
http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..
Geometric Probability Law A sequential experiment in which we repeat independent
Bernoulli trials until the occurrence of the first success. Let the outcome of this experiment be m.
Geometric probability law The probability p(m), that m trials are required.
1
1 0
[ ]
1 =1
m K j
m K j
K K
P m K p q pq q
pq qq
11 1,2,....mP m p p m