C HAPTER 3 Discrete Random Variables

The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA Universityhttp://multinet.inha.ac.krhttp://multinet.inha.ac.kr

Multimedia Network Lab.Multimedia Network Lab.

CHAPTER 3

Discrete Random Variables

Prof. Sang-Jo [email protected]

http://multinet.inha.ac.kr

The Graduate School of Information Technology and Telecommunications, INHA UniversityThe Graduate School of Information Technology and Telecommunications, INHA University

http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..

What we are going to study? Random variable

A function that assigns a numerical value to the outcome of the experiment.

Contents Concept of a random variable Methods for calculating probabilities of events involving a

random variable. Probability mass function Expected value of random variable. Conditional probability mass function given partial information

about the random variable.

2



Random Variable (1/2) Usually interested not in the outcome itself, but numerical

attribute of the outcome. Number of heads in n tosses of a coin. The weight of a randomly selected student.

A measurement assigns a numerical value to the outcome of the random experiment. Since the outcomes are random, the results of the measurements

will also be random.

Random variable X A function that assigns a real number, X(ζ), to each outcome ζ in

the sample space of a random experiment. The sample space S is the domain of the random variable and

the set SX of all values taken on by X is the range of the random variable. Note that SX ⊂ R, R is set of all real numbers.



Random Variable (2/2)

Example 3.1 A coin is tossed 3 times. Let X be the number of heads in 3

tosses Outcomes ζ? Random variables X(ζ) ? The probability of the

event {X=k}? Sample space S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}.

Let X be the number of heads; then SX = {0, 1, 2, 3}..81]3[,

83]2[,

83]1[,

81]0[;2)( XPXPXPXPTHHX



Equivalent Events (1)

Sx= the set of values that can be taken on by X B = the subset of Sx

A = {ζ: X(ζ) in B}: the set of outcomes ζ in S that lead to values X(ζ) in B

Since event B in SX occurs whenever event A in S occurs, and vice versa. Hence P[B] = P[A] = P[{ ζ: X( ζ ) in B}]. A and B are called equivalent events with respect to X.

1( ) { : ( ) }A X B X B



Equivalent events(2) Example

Consider the random experiment of tossing 3 coinsS = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

X = no of heads in the 3 coins, SX = {0, 1, 2, 3} A1 = {HTT, TTT} A2 = {HHT, HTH, THH, TTT} A3 = {HTT, THT, TTH, TTT}

X(A1) = {0, 1} = set of all values taken by X(ζ ), ζ∈ A1

X(A2) = {0, 2} X-1({0, 1}) = set of all outcomes of elements in {0, 1}

= {HTT, THT, TTH, TTT} = A3. X-1({2, 3}) = {HHT, HTH, THH, HHH}

= set of all outcomes of elements in B = {2, 3}.

6



Discrete Random Variables Discrete random variable X

A random variable that assumes values from a countable set, that is

If its range is finite, then a discrete random variable X is said to be finite.

We are interested in finding the probabilities of events involving a discrete random variable X.

Probability mass function (pmf) of a discrete random variable X

Note 3-1

7

,,, 321 xxxSX

xXPxXPpX :



Probability Mass Function PMF pX(x) satisfies three properties:

Example 3.5 (Coin Tosses and Binomial Random Variable)

8

BxXX

kallk

Sx kallkXX

X

SBxpBinXP

APxpxp

xxp

X

where ii)

1 ii)

allfor 0 i)



Bernoulli Random VariableExample 3.8 (Bernoulli Random Variable) Let A be an event related to the outcomes of some random

experiment. Indicator function for A is defined by

Identify IA =1 with a “success”. IA is a discrete random variable. In this case, SI=range of IA

={0,1}. Let P [A ]=p, then pmf is PI (0)=1-p , PI (1)= p . Geometric Random

0 if not in ( )

1 if in A

AI

A



Geometric Random Variable Count M independent Bernoulli trials until the first

occurrence of a success. M is the geometric random variable.

Example 3.9 (Message Transmissions) Probability mass function

Cumulative distribution function

trialBernoulllieach in success"" of probablity theis where

,1,2,for 1 1

APpkppkXP k

k

j

kj qpqkXP1

1 1



Binomial Random Variable A random experiment is repeated n times

Let X be the number of times an event A occurs in n trials

Example 3.10 (Transmission Errors) Probability mass function

ariable random binomial theis)RVs Bernoulli theof sum(21

vXIIIX n

n,0,for 1

kpp

kn

kXP knk


http://multinet.inha.ac.kr Multimedia Network LabMultimedia Network Lab..Expected Value of Random Variables

15 repetitions of a random experiment X varies about 5, Y varies about 0 X is more spread out than Y Need parameters that quantify these properties

0 10 20 30 40 50 60 70 80 90 100 110 120 130 140 150-2

-1

0

1

2

3

4

5

6

7

8

Trial Number

Yi

Xi



Expected Value of a RV X The expected value of a discrete RV X

The expected value is defined if the above integral or sum converges absolutely,

Example 3.11 (Mean of Bernoulli Random Variable) Example 3.12 (Three Coin Tossses and Binomial Random Variable)

[ ] ( )k X kk

E X x p x

[ ] ( )k X kk

E X x p x



Expected Value of the Geometric Random Variable

Can we find a closed form for the above summed series? Recall

Hence,

Example The chance of getting “6” in the throw of a dice is 1/6. The

expected number of trials required to get the first “6” is . Does the answer sound reasonable?

1 1

1 1

[ ] .k k

k k

E X kpq p kq

2 3

22

1 1 ..., | | 1;1

1 1 2 3 ..., | | 1.(1 )

x x x for xx

x x for xx

21 1[ ] ( ) .1

E X pq p

1 61/ 6



Expected Value of Functions of a RV

Let X be a discrete RV, and let Z will assume a countable set of values of the form

Group the term xk that are mapped to each value zj

Let Z be the function

15

XgZ kxg

k

kXk xpxgXgEZE

ZEzpzxpzxpxgj

jZjj zxgx

kXjk

kXkjkk

:

cXbhXagZ

ccEcXEcXEXaEaXEXhEXgEXhXgE

cXhbEXgaEZE

,,

Example 3.17Example 3.19



Variance of a Random Variable (1/2)

The expected value provides us with very limited information.

The deviation of about its mean.

The variance of X

The standard deviation of X

2XEXXD

1

22

222

kkXXkX

SxX

XX

xpmxxpmx

mXEXEXEXVAR

X

2222

2222

2

2

XXX

XXXX

mXEmXEmXE

mXmXEmXEXVAR

2/1XVARXSTDX



Variance of a Random Variable (2/2)

Properties of variance

The nth moment of the random variable X

2

[ ] 0[ ] [ ]

[ ] [ ]

VAR cVAR X c VAR X

VAR cX c VAR X

1k

kXn

kn xpxXE



Conditional Probability Mass Function

In man situations we have partial information about a random variable X or about the outcome of its underlying random experiment. We are interested in how this information changes the

probability of events involving the random variable.

Conditional probability mass function Conditional expected value Note 3-2

18



Important Discrete Random Variables

1. Bernoulli Random Variable Remark

The Bernoulli RV is the value of the indicator function IA for some event A; X=1 if A occurs and 0 otherwise.

The variance is quadratic in p, with value zero at p=0 and p=1 and maximum at p=1/2.

19

0 10,1 , 1 ,XS p p q p p

0 1 10 1Xm E X p p p p

2 20 1 10 1E X p p p p

2 2 2 1XVAR X E X m p p p p pq

p

VAR

1/2

1/4




2. Binomial Random Variable Remark: X is the number of successes in n Bernoulli trials

and hence the sum of n independent, identically distributed Bernoulli RV.

20

0,1, ,

1 0,1, ,

X

n kkX

S n

nP X k p k p p k n

k

0 0 1

( 1) ( 1)1

1

'' ''

' 0

![ ] ( ) 1 1( )!( 1)!

( 1)! 1[( 1) ( 1)]!( 1)!

Let ' 1 and ' 1 , then'![ ] 1

( ' ')! '!

n n nn k n kk k

Xk k k

nn kk

k

nn kk

k

n nE X kp k k p p p pk n k k

nnp p pn k k

n n k knE X np p p np

n k k



Variance of Binomial RV

21

2 2

2 2 2

0

2 2

0 0

2 2 2 2 2

( ) [ ] [ ]

( 1)

( ) 1 , 1 .

nk n k

k

n nk n k k n k

k k

VAR X E X E Xn

k p q n pk

n nk k p q k p q n p

k k

n p np np n p npq np p q p

0

2 2 ( 2) ( 2)

2

2 2 2

( 1)

( 2)!( 1)( )!( 2)!

nn k

k

nk n k

k

nk k q

k

nn n p p qn k k

n p np




3. Geometric Random Variable Remark: X is the number of trials until the first success in a

sequence of independent Bernoulli trials.

Using

Setting and multiplying both side by p

22

10,1, , 1 0,1,kX XS P X k p k p p k

1

1

k

k

E X kpq

2 2

1 1[ ]1

pE X pp pq

1

20 0

1 1,1 1

k k

k k

x kxx x

x q

Example 3.30



Variance of Geometric RV Using

Setting x=q and multiplying both sides by pq , we obtain

Since

Therefore,

23

23

1

2 ( 1) , | | 1.(1 )

k

k

k k x for xx

2 1 1 22

1 1

2 [ ] [ ].(1 )

k k

k k

q k pq kpq E X E Xq

22 2

1 2 1 1[ ] [ ] .(1 )

q qE X so E Xp q p p

2 22 2 2 2

1 1 1[ ] [ ] [ ] q q pVAR X E X E Xp P p p



Memoryless property The discrete geometric random variable observes the

memoryless property:

If a success has not occurred in the earlier j trials, then the probability of having to perform at least k more trials to get a success is the same as the probability of initially having to perform at least k trials to get a success.

Proof First, observe that

We then have

1[ ] , 1 .kP X k q q p

[ | ] [ ] for all , 1.P X k j X j P X k j k

11[ ][ | ] .

[ ]

k jk

j

P X k j qP X k j X j qP X j q




4. Poisson Random Variable Remark: Counting the number of occurrences of an event in

a certain time period or a certain region in space, e.g. counts of emissions from radioactive substances.

The pmf for a Poisson random variable N is

where is the average number of event occurrences in a specified time interval or region in space.

Note

25

[ ] , 0,1,2,...!

k

P N k e kk

0

1.!

k

k

ek

Documents

C HAPTER 3 Discrete Random Variables