Embed Size (px)
Citation preview
Probability and Random Process for Electrical Engineering
主講人 : Huan ChenOffice : 427 E-Mail : [email protected] 助教 : 黃政偉 ([email protected]) 鄭志川 ([email protected])Lab : 223
Outline
2.1 Specifying Random Experiments 2.2 The Axioms of Probability 2.3 Computing Probabilities Using
Counting Methods 2.4 Conditional Probability 2.5 Independence of Events
2.1 Specifying Random ExperimentsA random experiment is specified by stating an experimental procedure
and a set of one or more measurements or observations.
EXAMPLE 2.1 :
Experiment E1 : Select a ball form an urn containing balls numbered 1
to 50. Note the number of the ball.
Experiment E2 : Select a ball form an urn containing balls numbered 1
to 4. Suppose that balls 1 and 2 are black and that balls 3 and 4 are
white. Note number and color of the ball you select.
Experiment E3 : Toss a coin three times and note the sequence of he
ads and tails.
Experiment E4 : Toss a coin three times and note the number of head
s.
Experiment E7 : Pick a number at random between zero and one.
Experiment E12 : Pick two numbers at random between zero and one.
Experiment E13 : Pick a number X at random between zero and one, t
hen pick a number Y at random between zero and X.
The sample space
We define an outcome or sample point of a random experiment as a
result that cannot be decomposed into other results.
The sample space S of a random experiment is defined as t
he set of all possible outcomes.
We will denote an outcome of an experiment by ζ, where ζ i
s an element or point in S.
10:,
1010:,
1,010:
3,2,1,0
,,,,,,,
,4,,3,,2,,1
50,......,2,1
13
12
7
4
3
2
1
xyyxS
yandxyxS
xxS
S
TTTHTTTHTTTHTHHHTHHHTHHHS
wwbbS
S
See Fig. 2.1 (a).
See Fig. 2.1 (c).
See Fig. 2.1 (d).
EXAMPLE 2.2 :
The sample spaces corresponding to the experiments in Example 2.
1 are given below using set notation :
We will call S a discrete sample space if S is countable; that is, its
outcomes can be put into one-to-one correspondence with the pos
itive integers.
We will call S a continuous sample space if S is not countable.
Events
We define an event as a subset of S.
Two events of special interest are the certain event, S, which consi
sts of all outcomes and hence always occurs, and the impossible
or null event, φ, which contains no outcomes and hence never occ
urs.
An event from a discrete sample space that consists of a single out
come is called an elementary event.
Set Operations
We can combine events using set operations to obtain other
events.
The union of two events A and B is denoted by .
The intersection of two events A and B is denoted by
Two events are said to be mutually exclusive if their
intersection is the null event, .
The complement of an event A is denoted by Ac
If an event A is a subset of an event B, that is, .
The events A and B are equal, , if they contain the same
outcomes.
BA
BA
BA
BA
BA
The following properties of set operations and their combin
ations are useful :
Commutative Properties :
Associative Properties :
Distributive Properties :
DeMorgan’s Rules :
(2.1) and ABBAABBA
(2.2) and CBACBACBACBA
and )()()( CABACBA (2.3) )()()( CABACBA
and ccc BABA )( (2.4) ccc BABA )(
A B A B
AAc
A B
A
B
BAa )( BAb )(
cAc)( BAd )(
BAe )(
EXAMPLE 2.4
For Experiment E10, let the events A, B, and C be defined by
, “magnitude of v is greater than 10 volts,”
, ” v is less than -5 volts,” and
, “v is positive.”
You should then verify that
10: vvA
5: vvB
0: vvC
.105:)(
,
,10:)(
,0:
,10:
,105:
vvBA
andCBA
vvCBA
vvC
vvBA
vorvvBA
c
c
2.2 The Axioms of Probability
Let E be a random experiment with sample spaces S. A probability
law for the experiment E is a rule that assigns to each event A a
number called the probability of A, that satisfies the following
axioms :
Axiom Ⅰ ,
Axiom Ⅱ ,
Axiom Ⅲ if , then
Axiom ’Ⅲ If is a sequence
of events such that
for all , then
,AP
AP0
1SP
BA BPAPBAP
ji AA ji
11 kk
kAPP
....., 21 AA
COROLLARY 1.
Proof : Since an event A and its complement Ac are mutually excl
usive, , we have form Axiom that Ⅲ
Since , by Axiom , Ⅱ
The corollary follows after solving for
APAP c 1
cAA
cc APAPAAP
cAAS
cc APAPAAPSP 1
cAP
COROLLARY 2.
Proof : From Corollary 1,
,
since
COROLLARY 3.
Proof : Let and in Corollary 1 :
1AP
11 cAPAP
0cAP
0P
SA cA
01 SPP
COROLLARY 4. If are pairwise mutually exclusive,
then
Proof :
Suppose that the result is true for some ; th
at is ,
and consider the n + 1 case
nAAA ,....,, 21
(2.5)
n
kkk
n
kAPAP
11
.211
nAPAP
n
kkk
n
k for
(2.6) 11
11
1
1
nk
n
knk
n
kk
n
kAPAPAAPAP
where we have applied Axiom to the second expression after notⅢing that the union of events A1 to An is mutually exclusive with An+1.
The distributive property then implies
Substitution of Eq. (2.5) into Eq. (2.6) gives the n + 1 case
.1
11
11
n
knk
n
knk
n
kAAAA
1
11
n
kkk
n
kAPAP
COROLLARY 5.
Proof : First we decompose , A, and B as unions of disjoin
t events. From the Venn diagram in Fig. 2.4,
By substituting and from the two lower equations i
nto the top equation, we obtain the corollary.
Corollary 5 is easily generalized to three events,
(2.
7)
BAPBPAPBAP
BA
BAPABPBP
BAPBAPAP
BAPABPBAPBAP
c
c
cc
cBAP cABP
BAPCPBPAPCBAP
,CBAPCBPCAP
FIGURE 2.4
Decomposition of into three disjoint sets.
A B
cBA BAc BA
BA
A
B
BAc
COROLLARY 6.
Proof is by induction (See Problems 18 and 19).
Since probabilities are nonnegative, Corollary 5 implies
that the probability of the union of two events is no greater than the
sum of the individual event probabilities
(2.8)
kjkj
n
jj
n
kk AAPAPAP
11
.1 11
nn AAP
BPAPBAP
COROLLARY 7. If , then .
Proof : In Fig. 2.5, B is the union of A and , thus
,
since .
Discrete Sample Spaces
First, suppose that the sample space is finite, is
given by
BA BPAP
BAc
APBAPAPBP c
0BAP c
''2
'1, maaaS
''2
'1 ,,, maaaPBP
(2.9) ''2
'1 maPaPaP
If S is countably infinite, then Axiom ’implies that the probability oⅢf an event such as is given by
If the sample space has n elements, , a proba
bility assignment of particular interest is the case of equally likely o
utcomes. The probability of the elementary events is
The probability of any event that consists of k outcomes, say
, is
'2
'1,bbD
(2.10) '2
'1 bPbPDP
naaS ,1
(2.11) n
aPaPaP n
121
''1, kaaB
2.12 n
kaPaPBP k ''
1
Thus, if outcomes are equally likely, then the probability of an event is equ
al to the number of outcomes in the event divided by the total number of outc
omes in the sample space.
EXAMPLE 2.6
An urn contains 10 identical balls numbered 0,1,…, 9. A random expe
riment involves selecting a ball from the urn and noting the number of
the ball. Find the probability of the following events :
A = “number of ball selected is odd,”
B = “number of ball selected is a multiple of 3,”
C = “number of ball selected is less than 5,”
and of and .
The sample space is , so the sets of outcomes
corresponding to the above events are
, , and .
If we assume that the outcomes are equally likely, then
.
.
From Corollary 5.
.
BA CBA
9,,1,0 S
9,7,5,3,1A 9,6,3B 4,3,2,1,0C
10
597531 PPPPPAP
10
3963 PPPBP
10
543210 PPPPPCP
10
6
10
2
10
3
10
5 BAPBPAPBAP
where we have used the fact that , so
. From Corollary 6,
9,3BA 102BAP
BAPCPBPAPCBAP
CBAPCBPCAP
10
1
10
1
10
2
10
2
10
5
10
3
10
5
10
9
EXAMPLE 2.7
Suppose that a coin is tossed three times. If we observe the sequenc
e of heads and tails, then there are eight possible outcomes
. If we assume t
hat the outcomes of S3 are equiprobable, then the probability of each
of the eight elementary events is 1/8. This probability assignment imp
lies that the probability of obtaining two heads in three tosses is, by C
orollary 3,
This second probability assignment predicts that the probability of obt
aining two heads in three tosses is
TTTHTTTHTTTHTHHHTHHHTHHHS ,,,,,,,
THHHTHHHTPtossesinheadsP ,,"32"
8
3 THHPHTHPHHTP
4
12"32" PtossesinheadsP
Continuous Sample Spaces
Probability laws in experiments with continuous sample spaces specify a rul
e for assigning numbers to intervals of the real line and rectangular regions
in the plane.
EXAMPLE 2.9
Consider the random experiment “pick a number x at random betwe
en zero and one.” The sample space S for this experiment is the unit
interval [0,1], which is uncountably infinite. If we suppose that all the
outcomes S are equally likely to be selected, then we would guess th
at the probability that the outcome is in the interval [0,1/2] is the sam
e as the probability that the outcome is in the interval [1/2,1].
We would also guess that the probability of the outcome being exactl
y equal to ½ would be zero since there are an uncountably infinite nu
mber of equally likely outcomes.
Consider the following probability law : “The probability that t
he outcome falls in a subinterval of S is equal to the length of the sub
interval,” that is,
where by
we mean the probability of the event corresponding to the interval [a,
b].
)(, abbaP )14.2(10 bafor
baP ,
We now show that the probability law is consistent with the
previous guesses about the probabilities of the events[0,1/2],[1/2,1],
and {1/2} :
In addition, if x0 is any point in S, then since individual points
have zero width.
Since the two intervals are disjoint, we
have by Axiom Ⅲ
5.05.011,5.0
5.005.05.0,0
P
P
0, 00 xxP
1,8.02.0,0 A
4.1,8.02.0,0 PPAP
EXAMPLE 2.10
Suppose that the lifetime of a computer memory chip is measured, a
nd we find that “the proportion of chips whose lifetime exceeds t decr
eases exponentially at a rate α” Find an appropriate probability law.
Let the sample space in this experiment be . If we i
nterpret the above finding as “the probability that a chip’s lifetime exc
eeds t decreases exponentially at a rate α,” we then obtain the follo
wing assignment of probabilities to events of the form (t,∞):
where α > 0. Axiom is satisfied since Ⅱ
),0( S
0,for ),( tetP t
.1),0( PSP
The probability that the lifetime is in the interval (r, s] is found by
noting in Fig.27 that so by Axiom Ⅲ,
By rearranging the above equation we obtain
We thus obtain the probability of arbitrary intervals in S
Figure 2.7
,,,, rssr
.,,, sPsrPrP
.,,, sr eesPrPsrP
,,, ssrr (
r
](
s
EXAMPLE 2.11
Consider Experiment E12, where we picked two number x and y at
random between zero and one. The sample space is then the unit
square shown in Fig. 2.8(a). If we suppose that all pairs of numbers
in the unit square are equally likely to be selected, then it is
reasonable to use a probability assignment in which the probability of
any region R inside the unit square is equal to the area of R. Find the
probability of the following events: , , and
.
Figures 2.8(b) through 2.8(c) show the regions
corresponding to the events A, B, and C. Clearly each of these
regions has area ½. Thus
5.0 xA 5.0 yB yxC
2
1AP
2
1BP
2
1CP
FIGURE 2.8 a Two-dimensional sample space and three events.
(a) Sample space
x
y
0 1
S
y
x
(b) Event
0 1x
y
2
1x
2
1x
(c) Event
0 1 x
y
2
1y
2
1y (d) Event
0 1 x
y
yx
yx
2.4 CONDITINAL PROBABILITY
The conditional probability, , of event A given that event B
has occurred. The conditional probability is defined by
(2.24)
FIGURE 2.11 If B is known to have occurred, then A can occur only if occurs.
BAP |
0for|
BPBP
BAPBAP
BA
BAA
B
S
Suppose that the experiment is performed n times, and suppose that event B occurs times, and that event occurs times. The relative frequency of interest is then
where we have implicitly assumed that . This is an agreement with Eq. (2.24)
BABn BAn
,BP
BAP
nn
nn
n
n
B
BA
B
BA
0BP
EXAMPLE 2.21
A ball is selected from an urn containing two black balls, numbered 1
and 2, and two white balls, numbered 3 and 4. The number and color
of the ball is noted, so the sample space is
Assuming that the four outcomes are equally likely, find and
, where A, B, and C are the following events:
Since and , Eq. (2.21) gives
.,4,,3,,2,,1 wwbb BAP |
CBP |
2"thengreaterisballofnumber",,4.,3
and"selected,ballnumberedeven",,4,,2
"selected,ballblack",,2,,1
wwC
wbB
bbA
bPBAP ,2 0 PCAP
APBP
BAPBAP
5.
5.
25.|
If we multiply both sides of the definition of by
we obtain
(2.25a)
Similarly we also have that
(2.25b)
EXAMPLE 2.23
Many communication systems can be modeled in the following way.
First, the user inputs a 0 or a 1 into the system, and a corresponding
signal is transmitted. Second, the receiver makes a decision about
what was the input it the system, based on the signal it received.
.0
5.
0| AP
CP
CAPCAP
BAP | BP
.| BPBAPBAP
.| APABPBAP
Suppose that the user sends 0s with probability 1-p and 1s with prob
ability p, and suppose that the receiver makes random decision error
s with probability ε. For let be the event “input was ” and
let be the event “receiver decision was ” Find the probabilities
The
tree diagram for this experiment is shown in Fig. 2.13. We then readil
y obtain the desired probabilities
,1,0i iA
,i iB .i
1,0and1,0for jiBAP ji
,1100 pBAP
,110 pBAP
and,01 pBAP
.111 pBAP
FIGURE 2.13 Probabilities of input-output
pairs in a binary transmission
system.
Input into binary channel
Output from binary channel
1
0
0 0
1
1
1-p p
1-ε1-εεε
(1-p)(1-ε) (1-p)ε pε p(1-ε)
Let be mutually exclusive events whose union
equals the sample space S as shown in Fig. 2.14. We refer to these
sets as a partition of S. Any event A can be represented as the union
of mutually exclusive events in the following way :
FIGURE 2.14 A partition of S into n disjoint sets.
nBBB ,,, 21
nBBBASAA 21
.21 nBABABA
1B
2B
3B 1nB
nB
A
(See Fig. 2.14.) By Corollary 4, the probability of A is
By applying Eq. (2.25a) to each of the terms on the right-hand side,
we obtain the theorem on total probability :
EXAMPLE 2.25
A manufacturing process produces a mix of “good” memory chips an
d “bad” memory chips. The lifetime of good chips follows the expone
ntial law introduced in Example 2.10, with a rate of failure α. The lifeti
me of bad chips also follows the exponential law, but the rate of failur
e is 1000 α.
.21 nBAPBAPBAPAP
)26.2(.||| 2211 nn BPBAPBPBAPBPBAPAP
Suppose that the fraction of good chips is 1-p and of bad chips, p. Fin
d the probability that a randomly selected chip is still functioning after
t seconds.
Let C be the event “chip still functioning after t seconds,” and
let G be the event “chip is good,” and B the event “chip is bad.” By th
e theorem on total probability we have
where we used the fact that
BPBCPGPGCPCP ||
pBCPpGCP |1|
,1 1000 tt peep
tt eBCPandeGCP 1000||
Bayes’ Rule
Let be a partition of a sample space S. Suppose that ev
ent A occurs; what is the probability of event B1? By the definition of c
onditional probability we have
where we used the theorem on total probability to replace .
Eq.(2.27) is called Bayes’ rule.
nBBB ,,, 21
)27.2(,|
||
1
n
kkk
jjjj
BPBAP
BPBAP
AP
BAPABP
AP
EXAMPLE 2.26 Binary Communication System
In the binary communication system in Example 2.23, find which inpu
t is more probable given that the receiver has output a 1. Assume tha
t, a priori, the input is equally likely to be 0 or 1. Let Ak be the
event that the input was k , k = 0, 1, then A0 and A1 are a partition of t
he sample space of input-output pairs. Let B1 be the event “receiver o
utput was a 1.” The probability of B1 is
Applying Bayes’ rule, we obtain the a posteriori probabilities
1110011 || APABPAPABPBP
.2
1
2
11
2
1
21
2||
1
00110 BP
APABPBAP
Thus, if εis less than 1/2, then input 1 is more likely than input 0 wh
en a 1 is observed at the output of the channel.
EXAMPLE 2.27 Quality Control
Consider the memory chips discussed in Example 2.25. Recall that a
fraction p of the chips are bad and tend to fail much more quickly tha
n good chips. Suppose that in order to “weed out” the bad chips, ever
y chip is tested for t seconds prior to leaving the factory. The chips th
at fail are discarded and the remaining chips are sent out to custome
rs. Find the value of t for which 99% of the chips sent out to customer
s are good.
.121
21||
1
11111
BP
APABPBAP
Let C be the event “chip still functioning after t seconds,” and
let G be the event “chip is good,” and B the event “chip is bad,” The p
roblem requires that we find the value of t for which
We find by applying Bayes’ rule :
.99.| CGP
CGP |
BPBCPGPGCP
GPGCPCGP
||
||
tt
t
peep
ep
10001
1
99.
11
11000
t
t
eppe
The above equation can then be solved for t :
For example, if 1/α=20,000 hours and p = .10, then t = 48 hours.
.1
99ln
999
1
p
pt
2.5 INDEPENDENCE OF EVENTSWe will define two events A and B to be independent if
Equation (2.28) then implies both
Note also that Eq. (2.29a) implies Eq. (2.28) when and
Eq. (2.29b) implies Eq. (2.28) when
)28.2(.BPAPBAP
)29.2(| aAPBAP
)29.2(| bBPABP
0BP
.0AP
EXAMPLE 2.28
A ball is selected from an urn containing two black balls, numbered 1
and 2, and two white balls, numbered 3 and 4. Let the events A, B,
and C be defined as follows:
Are events A and B independent? Are events A and C
independent?
the events A and
B are independent
2"thengreaterisballofnumber",,4.,3
and"selected,ballnumberedeven",,4,,2
"selected,ballblack",,2,,1
wwC
wbB
bbA
,2
1 BPAP
.4
1,2 bPBAP
.4
1BPAPBAP
by Equation (2.29b),
These two equations imply that because the proportion of
outcomes in S that lead to the occurrence of A is equal to the proportion of
outcomes in B that lead to A.
Events A and C are not independent since
so
2
1
21
41
,4,,2
,2|
wbP
bP
BP
BAPBAP
1
21
,4,,3,,2,,1
,2,,1
wwbbP
bbP
SP
APAP
BAPAP |
0 PCAP
5.0 APCAP
EXAMPLE 2.29
Two numbers x and y are selected at random between zero and one.
Let the events A, B, and C be defined as follows:
Are the events A and B independent? Are A and C independent?
Using Eq.(2.29a),
so events A and B are independent.
Using Eq.(2.29b),
so events A and C are not independent.
,5.0 xA and,5.0 yB .yxC
,
2
1
21
41| AP
BP
BAPBAP
,
2
1
4
3
21
83| AP
CP
CAPCAP
FIGURE 2.15
Examples of independent and nonindependent events.
0 1/2 1x
y
(a) Events A and B are independent
A
B
0 1/2 1x
y
(b) Events A and C are not
independent
A
C
What conditions should three events A, B, and C satisfy in or
der for them to be independent? First, they should be pairwise indep
endent, that is,
In addition, knowledge of the joint occurrence of any two, say A and
B, should not affect the probability of the third, that is
and
Then
Thus we conclude that three events A, B, and C are independent if the pro
bability of the intersection of any pair or triplet of events is equal to the pro
duct of the probabilities of the individual events.
and,, CPAPCAPBPAPBAP .CPBPCBP
.| CPBACP
.| CP
BAP
CBAPBACP
,CPBPAPCPBAPCBAP
EXAMPLE 2.30
Consider the experiment discussed in Example 2.29 where two
numbers are selected at random from the unit interval. Let the events
B, D, and F be defined as follows:
It can be easily verified that any pair of these events is independent :
2
1,
2
1xDyB
.2
1and
2
1
2
1and
2
1
yxyxF
,4
1DPBPDBP
and,4
1FPBPFBP
so,FDBsince.4
1 FPDPFDP
.8
10 FPDPBPPFDBP
FIGURE 2.16 Events B, D, and F are pairwise independent, but the triplet B, D, F are not independent events.
2
1)( yBa
2
1)( xDb
0 1/2 1x
y
D
0 1 x
y
B
1 1|2
0 1/2 1
1 1|2
x
y
F
F
2
1and
2
1
2
1and
2
1)( yxyxFc
The events A1, A2, …, An are said to be independent if for k = 2, …, n
where
EXAMPLE 2.31
Suppose a fair coin is tossed three times and we observe the
resulting sequence of heads and tails. Find the probability of the
elementary events.
Sample space The
assumption that the coin is fair, If we
assume that the outcomes of the coin tosses are independent,
)30.2(,2121 nn iiiiii APAPAPAAAP
.1 21 niii k
.,,,,,,, TTTHTTTHTTTHTHHHTHHHTHHHS .21 TPHP
,81
,81
,81
,81
,81
,81
,81
,81
TPTPTPTTTP
TPTPHPHTTP
TPHPTPTHTP
HPTPTPTTHP
HPHPTPTHHP
HPTPHPHTHP
TPHPHPHHTP
HPHPHPHHHP
2.6 SEQUENTIAL EXPERIMENTS
Sequences of Independent Experiments
Let A1, A2, …, An be events such that Ak concerns only the outcome of
the kth subexperiment. If the subexperiments are independent, then
This expression allows us to compute all probabilities of events of the
sequential experiment
)31.2(.2121 nn APAPAPAAAP
EXAMPLE 2.33
Suppose that 10 numbers are selected at random from the interval [0,
1]. Find the probability that the first 5 numbers are less than 1/4 and t
he last numbers are greater than 1/2 . Let
x1, x2,…, xn be the sequence of 10 numbers, then
If we assume that each selection of a number is independent of the o
ther selections, then
.10,,6for2
1
5,,1for4
1
kxA
kxA
kk
kk
10211021 APAPAPAAAP
.2
1
4
155
The Binomial Probability Law
EXAMPLE 2.34
Suppose that a coin is tossed three times. If we assume that the
tosses are independent and the probability of heads is p, then the
probability for the sequences of heads and tails is
,1
and,1
,1
,1
,1
,1
,1
,
3
2
2
2
2
2
2
3
pTPTPTPTTTP
ppTPTPHPHTTP
ppTPHPTPTHTP
ppHPTPTPTTHP
ppHPHPTPTHHP
ppHPTPHPHTHP
ppTPHPHPHHTP
pHPHPHPHHHP
Let k be the number of heads in three trials, then
THEOREM Let k be the number of successes in n independent B
ernoulli trials, then the probabilities of k are given by the binomial pro
bability law:
where is the probability of k successes in n trails,
is the binomial coefficient.
.3
and,13,,2
,13,,1
,10
3
2
2
3
pHHHPkP
ppTHHHTHHHTPkP
ppHTTTHTTTHPkP
pTTTPkP
)32.2(,,...,01 nkforppk
nkp knk
n
kpn
)33.2(!!
!
knk
n
k
n
We now prove the above theorem. Following Example 2.34
we see that each of the sequences with k successes and n – k
failures has the same probability, namely . Let be the
number of distinct sequences that have k successes and n – k
failures, then
The expression , is the number of ways of picking k positions
out o fn for the successes. It can be shown that
)34.2(.1 knknn ppkNkp
knk pp 1 kNn
kNn
)35.2(.
k
nkNn
EXAMPLE 2.35
Verify that Eq. (2.32) gives the probabilities found in Example 2.34.
In Example 2.34, let “toss results in heads” correspond to a “
success,” then
,01!0!3
!33
,131!1!2
!32
,131!2!1
!31
,11!3!0
!30
333
12123
2213
3303
pppp
ppppp
ppppp
pppp
Binomial theorem
If we let a = b = 1,
If we let a = p and b = 1 – p in Eq. (2.36), then
which confirms that the probabilities of the binomial probabilities sum
to 1.
)36.2(.0
knkn
k
n bak
nba
,200
kNk
n n
kn
n
k
n
,1100
kpppk
n n
kn
knkn
k
)37.2(
111 kp
pk
pknkp nn
EXAMPLE 2.36
Let k be the number of active (nonsilent) speakers in a group of eight
noninteracting (i.e., independent) speakers. Suppose that a speaker i
s active with probability 1/3 . Find the probability that the number of a
ctive speakers is greater than six.
For i = 1,…, 8, let Ai denote the event “ith speaker is active.”
The number of active speakers is then the number of successes in ei
ght Bernoulli trials with p = 1/3.
87
3
1
8
8
3
2
3
1
7
887
kPkP
00259.00015.00244.
EXAMPLE 2.37 Error Correction Coding
A communication system transmits binary information over a channel
that introduces random bit errors with probability . The tran
smitter transmits each information bit three times, and a decoder take
s a majority vote of the received bits to decide on what the transmitte
d bit was. Find the probability that the receiver will make an incorrect
decision.
310
.103001.3
3999.001.
2
32 632
kP
The Multinomial Probability Law
Suppose that n independent repetitions of the experiment are perfor
med. Let kj be the number of times event Bj occurs, then the vector (k
1, k2,…, kM ) specifies the number of times each of the events Bj occu
rs. The probability of the vector (k1, k2,…, kM ) satisfies the multinomia
l probability law :
where
)38.2(,!!!
!,,, 21
2121
21Mk
Mkk
MM ppp
kkk
nkkkP
.21 nkkk M
EXAMPLE 2.38
A dart is thrown nine times at a target consisting of three areas. Each
throw has a probability of .2, .3, and.5 of landing in areas 1, 2, and 3,
respectively. Find the probability that the dart lands exactly three tim
es in each of the areas. Let
n = 9 and p1 = .2, p2 = .3, and p3 =.5 :
04536.5.3.2.!3!3!3
!93,3,3 333 P
EXAMPLE 2.39
Suppose we pick 10 telephone numbers at random from a telephone
book and note the last digit in each of the numbers. What is the prob
ability that we obtain each of the integers form 0 to 9 only once?
Let M = 10, n = 10, and pj = 1/10 if we assume that the 10 int
egers in the range 0 to 9 are equiprobale.
.106.31.!1!1!1
!10 410
The Geometric Probability Law
Consider a sequential experiment in which we repeat independent B
ernoulli trials until the occurrence of the first success. Let the outcom
e of this experiment be m, the number of trials carried out until the oc
currence of the first success. The sample space for this experiment i
s the set of positive integers. The probability, p(m), that m trials are re
quired is found by noting that this can only happen if the first m -1 trial
s result in failures and the mth trial in success. The probability of this
event is
where Ai is the event “success in ith trial.” The probability assignment
specified by Eq. (2.39) is called the geometric probability law.
)39.2(,,2,11 1121
mppAAAAPmp mm
cm
cc
where q = 1 – p,
The probability that more than K trials are required before a success
occurs has a simple form :
01
1
j
jK
Km
m qpqqpKmP
qpqK
1
1
)40.2(.Kq
,11
1
1
1
1
qpqpmp
m
m
m
EXAMPLE 2.40 Error Control by Retransmission
Computer A sends a message to computer B over an unreliable telep
hone line. The message is encoded so that B can detect when errors
have been introduced into the message during transmission. If B det
ects an error, it request A to retransmit it . If the probability of a mess
age transmission error is q = .1, what is the probability that a messag
e needs to be transmitted more than two times ?
Each transmission of a message is a Bernoulli trial with prob
ability of success p = 1 – q. The probability that more than two transm
issions are required is given by Eq. (2.40):
.102 22 qmP
Sequences of Dependent Experiments
EXAMPLE 2.41
A sequential experiment involves repeatedly drawing a ball from one
of two urns, noting the number on the ball, and replacing the ball in it
s urn. Urn 0 contains a ball with the number 1 and two balls with the
number 0. The urn from which the first draw is made is selected at ra
ndom by flipping a fair coin. Urn 0 is used if the outcome is heads an
d urn 1 if the outcome is tails. Thereafter the urn used in a subexperi
ment corresponds to the number on the ball selected in the previous
subexperiment.
FIGURE 2.17 Trellis diagram for a Markov chain.
0
1
0
1
00
1 1
h
t0 0 0
0 0 0
1 1 1
1 1 1
1 2 3 4
3
2
0
1
0
1
00
1 1
2
1
2
1
3
1
3
13
1
6
1
6
16
1
6
56
56
5
3
2
3
2
(a) Each sequence of outcomes corresponds to a path through this trellis diagram
(b) The probability of a sequence of outcomes is the product of the probabilities along the associated path
To compute the probability of a particular sequence of outco
mes, say s0, s1, s2. Denote this probability by .
Let , then since
we have
Now note that in the above urn example the probability
depends only on since the most recen
t outcome determines which subexperiment is performed :
Therefore for the sequence of interest we have that
210 sssP 102 ssBsA and BPBAPBAP |
10102210 | ssPsssPsssP
)41.2(.|| 001102 sPssPsssP
10| nn sssP 1ns
)42.2(.|| 110 nnnn ssPsssP
)43.2(.|| 00112210 sPssPssPsssP
Sequential experiments that satisfy Eq. (2.42) are called Mar
kov chains. For these experiments, the probability of a sequence s0, s
1, …, sn is given by
EXA
MPLE 2.42 Find th
e probability of the sequence 0011 for the urn experiment introduced
in Example 2.41.
)44.2(|||,, 0012111,0 sPssPssPssPsssP nnnnn
,00|00|11|10011 PPPPP
3
20|0
3
10|1 PP and
12
10,
6
11|0
6
51|1 PPPP and
.54
5
2
1
3
2
3
1
6
50011
P
