Iyer - Lecture 13

ECE 313 - Fall 2013

Functions of Random Variables, Expectation and Variance

ECE 313 Probability with Engineering Applications

Lecture 13 Professor Ravi K. Iyer

Dept. of Electrical and Computer Engineering University of Illinois at Urbana Champaign

Iyer - Lecture 13

ECE 313 - Fall 2013

Today’s Topics

•  Functions of a Random Variable •  Expectation of a Function of a Random Variable •  Variance

–  Variance of Exponential Distribution –  Variance of Normal Distribution

Iyer - Lecture 13

ECE 313 - Fall 2013

Functions of a Random Variable

•  Let As an example, X could denote the measurement error in a certain physical experiment and Y would then be the square of the error (recall the method of least squares).

•  Note that

2)( XXY ==φ

⎪⎩

⎪⎨

⎧ >−=

−−=

≤≤−=

≤=

≤=

>≤=

.,0

,0)],([2

1)(

),()(

)(

)(

)()(,0For .0for 0)(

is Y ofdensity ation thedifferentiby and

2

otherwise

yyfyyf

yFyF

yXyP

yXPyYPyF

yyyF

XY

XX

Y

Y

Iyer - Lecture 13

ECE 313 - Fall 2013

Functions of a Random Variable (cont.)

•  Let X have the standard normal distribution [N(0,1)] so that

•  This is a chi-squared distribution with one degree of freedom

⎪⎩

⎪⎨

⎧

≤

>=

≤

>

⎪⎩

⎪⎨

⎧⎟⎠

⎞⎜⎝

⎛+

=

∞<<∞−=

−

−−

−

,0.0,0

,21

)(

or

,0,0

,,0

21

21

21

)(

Then

.,21)(

2/

2/2/

2/2

yy

eyyf

yyee

yyf

xexf

y

Y

yy

Y

xX

π

ππ

π

Iyer - Lecture 13

ECE 313 - Fall 2013

Functions of a Random Variable (cont.) •  Let X be uniformly distributed on (0,1). We show that

has an exponential distribution with parameter Observe that Y is a nonnegative random variable implying

• 

•  This fact can be used in a distribution-driven simulation. In simulation programs it is important to be able to generate values of variables with known distribution functions. Such values are known as random deviates or random variates. Most computer systems provide built-in functions to generate random deviates from the uniform distribution over (0,1), say u. Such random deviates are called random numbers.

For y > 0, we haveFY (y) = P(Y ≤ y) = P[−λ−1 ln(1− X) ≤ y]

= P[ln(1− X) ≥ −λy]= P[(1− X) ≥ e−λy ] (since ex is an increasing function of x,)= P(X ≤1− e−λy )= FX (1− e−λy ).

But since X is uniform over (0,1), FX (x) = x, 0 ≤ x ≤1.Thus FY (y) =1− e−λy. Therefore Y is exponentially distributed with parameter λ.

)1ln(1 XY −−= −λ.0>λ

FY (y) = 0for y ≤ 0.

Iyer - Lecture 13

ECE 313 - Fall 2013

Example 1 •  Let X be uniformly distributed on (0,1). We obtain the cumulative

distribution function (CDF) of the random variable Y, defined by Y = Xn as follows: for

•  For instance, the probability density function (PDF) of Y is given by

n

nX

n

nY

yyFyXPyXPyYPyF

/1

/1

/1

)(}{}{

}{)(

=

=

≤=

≤=

≤=

,10 ≤≤ y

fY (y) =1ny1n−1

0

"

#$

%$

0 ≤ y ≤1otherwise

Iyer - Lecture 13

ECE 313 - Fall 2013

Expectation of a Function of a Random Variable

•  Given a random variable X and its probability distribution or its pmf/pdf •  We are interested in calculating not the expected value of X, but the

expected value of some function of X, say, g(X). •  One way: since g(X) is itself a random variable, it must have a probability

distribution, which should be computable from a knowledge of the distribution of X. Once we have obtained the distribution of g(X), we can then compute E[g(X)] by the definition of the expectation.

•  Example 1: Suppose X has the following probability mass function:

•  Calculate E[X2]. •  Letting Y=X2,we have that Y is a random variable that can take on one of

the values, 02, 12, 22 with respective probabilities

21.1][][71. thatNote

7.1)3.0(4)5.0(1)2.0(0][][Hence,

22

2

=≠=

=++==

XEXE

YEXE

3.0)2(,5.0)1(,2.0)0( === ppp

pY (0) = P{Y = 02} = 0.2

pY (1) = P{Y =12} = 0.5

pY (2) = P{Y = 22} = 0.3

Iyer - Lecture 13

ECE 313 - Fall 2013

Expectation of a Function of a Random Variable (cont.)

•  Proposition 2: (a) If X is a discrete random variable with probability mass function p(x), then for any real-valued function g,

•  (b) if X is a continuous random variable with probability density function f(x), then for any real-valued function g:

•  Example 3, Applying the proposition to Example 1 yields

•  Example 4, Applying the proposition to Example 2 yields

∑>

=0)(:

)()()]([xpx

xpxgXgE

dxxfxgXgE ∫∞

∞−= )()()]([

7.1)3.0)(2()5.0)(1()2.0(0][ 2222 =++=XE

41

)101 (since ][1

0

33

=

<<== ∫ x, f(x)dxxXE

Iyer - Lecture 13

ECE 313 - Fall 2013

Corollary

•  If a and b are constants, then •  The discrete case:

•  The continuous case:

bXaEbaXE +=+ ][][

bXaE

xpbxxpa

xpbaxbaXE

xpxxpx

xpx

+=

+=

+=+

∑∑

∑

>>

>

][

)()(

)()(][

0)(:0)(:

0)(:

bXaE

dxxfbdxxxfa

dxxfbaxbaXE

+=

+=

+=+

∫∫

∫∞

∞−

∞

∞−

∞

∞−

][

)()(

)()(][

Iyer - Lecture 13

ECE 313 - Fall 2013

Moments

•  The expected value of a random variable X, E[X], is also referred to as the mean or the first moment of X.

•  The quantity is called the nth moment of X. We have:

•  Another quantity of interest is the variance of a random variable X, denoted by Var(X), which is defined by:

1],[ ≥nXE n

E[Xn ]=

xn p(x),x:p(x )>0∑ if X is discrete

xn f (x)dx,−∞

∞

∫ if X is continuous

%

&

''

(

''

]])[[()( 2XEXEXVar −=

Iyer - Lecture 13

ECE 313 - Fall 2013

Variance of a Random Variable

•  Suppose that X is continuous with density f, let . Then,

•  So we obtain the useful identity:

µ=][XE

22

22

22

22

22

2

][2][

)()(2)(

)()2(

)]2[])[()(

µ

µµµ

µµ

µµ

µµ

µ

−=

+−=

+−=

+−=

+−=

−=

∫ ∫∫

∫∞

∞−

∞

∞−

∞

∞−

∞

∞−

XEXE

dxxfdxxxfdxxfx

dxxfxx

XXEXEXVar

22 ])[(][)( XEXEXVar −=

Iyer - Lecture 13

ECE 313 - Fall 2013

Variance of Normal Random Variable

•  Let X be normally distributed with parameters and . Find Var(X).

•  Recalling that , we have that:

•  Substituting yields:

•  Integrating by parts ( ) gives:

µ 2σ

µ=][XE

dxex

XEXVar

x∫∞

∞−

−−−=

−=

22 2/)(2

2

)(21

])[()(

σµµσπ

µ

σµ /)( −= xy

dyeyXVar y∫∞

∞−

−= 2/22

2

2)(

πσ

dyyedvyu y 2/2, −==

[ ] 22/2

2/2/2

222

22)( σ

πσ

πσ

==⎟⎟⎠

⎞⎜⎜⎝

⎛+−= ∫∫

∞

∞−

−∞

∞−

−∞

∞−− dyedyeyeXVar yyy

Iyer - Lecture 13

ECE 313 - Fall 2013

Variance of Exponential Distribution

•  From this, we determine the following proof: •  From this point, we need to use integration by parts to solve this equation:

•  Now we can use the integration by parts formula to continue solving:

f (x) =λe−λx; x ≥ 0

0; otherwise

#$%

&%

'(%

)%

dxexxE x∫∞ −=0

)( λλ

dxedvdxdu

evxu

x

x

λ

λ

λλ

λ

−

−

−==

−==

∫ ∫−= duvuvdvu

[ ]

⎟⎠

⎞⎜⎝

⎛−−=

⎥⎦

⎤⎢⎣

⎡+=

−+=

−−⎥⎦

⎤⎢⎣

⎡ −=

∞−

∞ −∞

∞ −∞−

∫

∫−

λ

λ

λλ

λλ

λ

λ

λλ

λ

10)(

0)(

)(

))(()(

0

00

00

xE

exE

dxexexE

dxeexxE

x

x

xx

x

Var(x) = E(x2 )−[E(x)]2

Iyer - Lecture 13

ECE 313 - Fall 2013

Variance of Exponential Distribution (cont.)

•  Now, we need to determine E(x2) so we can calculate the variance:

•  Now, we need to do integration by parts again:

•  Now we use the integration by parts formula again:

•  Now, remember that and we will be able to substitute it into the equation:

dxexxE xλλ −∞

∫= 0

22 )(

dxedvdxxdu

evxu

x

x

λ

λ

λλ

λ

−

−

−==

−==

2

2

[ ] dxexexxE

dxexexxE

xx

xx

λλ

λλ

λλ

λλ

λ

−∞∞−

∞ −∞−

∫

∫

+=

−+⎥

⎦

⎤⎢⎣

⎡ −=

0022

00

22

2)(

))(2())(()(

λλ λ 1)(

0== −∞

∫ dxexxE x

Iyer - Lecture 13

ECE 313 - Fall 2013

Variance of Exponential Distribution (cont.)

⎟⎠

⎞⎜⎝

⎛=

⎟⎠

⎞⎜⎝

⎛⎟⎠

⎞⎜⎝

⎛+=

22

2

2)(

120)(

λ

λλ

xE

xE

•  Now that we have found and we can substitute them into the equation to find the following holds true:

λλ2)( and 1)( == xExE22 )]([)()( xExExVar −=

2

22

1)(

12)(

λ

λλ

=

−=

xVar

xVar