Iyer - Lecture 13
ECE 313 - Fall 2013
Functions of Random Variables, Expectation and Variance
ECE 313 Probability with Engineering Applications
Lecture 13 Professor Ravi K. Iyer
Dept. of Electrical and Computer Engineering University of Illinois at Urbana Champaign
Iyer - Lecture 13
ECE 313 - Fall 2013
Today’s Topics
• Functions of a Random Variable • Expectation of a Function of a Random Variable • Variance
– Variance of Exponential Distribution – Variance of Normal Distribution
Iyer - Lecture 13
ECE 313 - Fall 2013
Functions of a Random Variable
• Let As an example, X could denote the measurement error in a certain physical experiment and Y would then be the square of the error (recall the method of least squares).
• Note that
2)( XXY ==φ
⎪⎩
⎪⎨
⎧ >−=
−−=
≤≤−=
≤=
≤=
>≤=
.,0
,0)],([2
1)(
),()(
)(
)(
)()(,0For .0for 0)(
is Y ofdensity ation thedifferentiby and
2
otherwise
yyfyyf
yFyF
yXyP
yXPyYPyF
yyyF
XY
XX
Y
Y
Iyer - Lecture 13
ECE 313 - Fall 2013
Functions of a Random Variable (cont.)
• Let X have the standard normal distribution [N(0,1)] so that
• This is a chi-squared distribution with one degree of freedom
⎪⎩
⎪⎨
⎧
≤
>=
≤
>
⎪⎩
⎪⎨
⎧⎟⎠
⎞⎜⎝
⎛+
=
∞<<∞−=
−
−−
−
,0.0,0
,21
)(
or
,0,0
,,0
21
21
21
)(
Then
.,21)(
2/
2/2/
2/2
yy
eyyf
yyee
yyf
xexf
y
Y
yy
Y
xX
π
ππ
π
Iyer - Lecture 13
ECE 313 - Fall 2013
Functions of a Random Variable (cont.) • Let X be uniformly distributed on (0,1). We show that
has an exponential distribution with parameter Observe that Y is a nonnegative random variable implying
•
• This fact can be used in a distribution-driven simulation. In simulation programs it is important to be able to generate values of variables with known distribution functions. Such values are known as random deviates or random variates. Most computer systems provide built-in functions to generate random deviates from the uniform distribution over (0,1), say u. Such random deviates are called random numbers.
For y > 0, we haveFY (y) = P(Y ≤ y) = P[−λ−1 ln(1− X) ≤ y]
= P[ln(1− X) ≥ −λy]= P[(1− X) ≥ e−λy ] (since ex is an increasing function of x,)= P(X ≤1− e−λy )= FX (1− e−λy ).
But since X is uniform over (0,1), FX (x) = x, 0 ≤ x ≤1.Thus FY (y) =1− e−λy. Therefore Y is exponentially distributed with parameter λ.
)1ln(1 XY −−= −λ.0>λ
FY (y) = 0for y ≤ 0.
Iyer - Lecture 13
ECE 313 - Fall 2013
Example 1 • Let X be uniformly distributed on (0,1). We obtain the cumulative
distribution function (CDF) of the random variable Y, defined by Y = Xn as follows: for
• For instance, the probability density function (PDF) of Y is given by
n
nX
n
nY
yyFyXPyXPyYPyF
/1
/1
/1
)(}{}{
}{)(
=
=
≤=
≤=
≤=
,10 ≤≤ y
fY (y) =1ny1n−1
0
"
#$
%$
0 ≤ y ≤1otherwise
Iyer - Lecture 13
ECE 313 - Fall 2013
Expectation of a Function of a Random Variable
• Given a random variable X and its probability distribution or its pmf/pdf • We are interested in calculating not the expected value of X, but the
expected value of some function of X, say, g(X). • One way: since g(X) is itself a random variable, it must have a probability
distribution, which should be computable from a knowledge of the distribution of X. Once we have obtained the distribution of g(X), we can then compute E[g(X)] by the definition of the expectation.
• Example 1: Suppose X has the following probability mass function:
• Calculate E[X2]. • Letting Y=X2,we have that Y is a random variable that can take on one of
the values, 02, 12, 22 with respective probabilities
21.1][][71. thatNote
7.1)3.0(4)5.0(1)2.0(0][][Hence,
22
2
=≠=
=++==
XEXE
YEXE
3.0)2(,5.0)1(,2.0)0( === ppp
pY (0) = P{Y = 02} = 0.2
pY (1) = P{Y =12} = 0.5
pY (2) = P{Y = 22} = 0.3
Iyer - Lecture 13
ECE 313 - Fall 2013
Expectation of a Function of a Random Variable (cont.)
• Proposition 2: (a) If X is a discrete random variable with probability mass function p(x), then for any real-valued function g,
• (b) if X is a continuous random variable with probability density function f(x), then for any real-valued function g:
• Example 3, Applying the proposition to Example 1 yields
• Example 4, Applying the proposition to Example 2 yields
∑>
=0)(:
)()()]([xpx
xpxgXgE
dxxfxgXgE ∫∞
∞−= )()()]([
7.1)3.0)(2()5.0)(1()2.0(0][ 2222 =++=XE
41
)101 (since ][1
0
33
=
<<== ∫ x, f(x)dxxXE
Iyer - Lecture 13
ECE 313 - Fall 2013
Corollary
• If a and b are constants, then • The discrete case:
• The continuous case:
bXaEbaXE +=+ ][][
bXaE
xpbxxpa
xpbaxbaXE
xpxxpx
xpx
+=
+=
+=+
∑∑
∑
>>
>
][
)()(
)()(][
0)(:0)(:
0)(:
bXaE
dxxfbdxxxfa
dxxfbaxbaXE
+=
+=
+=+
∫∫
∫∞
∞−
∞
∞−
∞
∞−
][
)()(
)()(][
Iyer - Lecture 13
ECE 313 - Fall 2013
Moments
• The expected value of a random variable X, E[X], is also referred to as the mean or the first moment of X.
• The quantity is called the nth moment of X. We have:
• Another quantity of interest is the variance of a random variable X, denoted by Var(X), which is defined by:
1],[ ≥nXE n
E[Xn ]=
xn p(x),x:p(x )>0∑ if X is discrete
xn f (x)dx,−∞
∞
∫ if X is continuous
%
&
''
(
''
]])[[()( 2XEXEXVar −=
Iyer - Lecture 13
ECE 313 - Fall 2013
Variance of a Random Variable
• Suppose that X is continuous with density f, let . Then,
• So we obtain the useful identity:
µ=][XE
22
22
22
22
22
2
][2][
)()(2)(
)()2(
)]2[])[()(
µ
µµµ
µµ
µµ
µµ
µ
−=
+−=
+−=
+−=
+−=
−=
∫ ∫∫
∫∞
∞−
∞
∞−
∞
∞−
∞
∞−
XEXE
dxxfdxxxfdxxfx
dxxfxx
XXEXEXVar
22 ])[(][)( XEXEXVar −=
Iyer - Lecture 13
ECE 313 - Fall 2013
Variance of Normal Random Variable
• Let X be normally distributed with parameters and . Find Var(X).
• Recalling that , we have that:
• Substituting yields:
• Integrating by parts ( ) gives:
µ 2σ
µ=][XE
dxex
XEXVar
x∫∞
∞−
−−−=
−=
22 2/)(2
2
)(21
])[()(
σµµσπ
µ
σµ /)( −= xy
dyeyXVar y∫∞
∞−
−= 2/22
2
2)(
πσ
dyyedvyu y 2/2, −==
[ ] 22/2
2/2/2
222
22)( σ
πσ
πσ
==⎟⎟⎠
⎞⎜⎜⎝
⎛+−= ∫∫
∞
∞−
−∞
∞−
−∞
∞−− dyedyeyeXVar yyy
Iyer - Lecture 13
ECE 313 - Fall 2013
Variance of Exponential Distribution
• From this, we determine the following proof: • From this point, we need to use integration by parts to solve this equation:
• Now we can use the integration by parts formula to continue solving:
f (x) =λe−λx; x ≥ 0
0; otherwise
#$%
&%
'(%
)%
dxexxE x∫∞ −=0
)( λλ
dxedvdxdu
evxu
x
x
λ
λ
λλ
λ
−
−
−==
−==
∫ ∫−= duvuvdvu
[ ]
⎟⎠
⎞⎜⎝
⎛−−=
⎥⎦
⎤⎢⎣
⎡+=
−+=
−−⎥⎦
⎤⎢⎣
⎡ −=
∞−
∞ −∞
∞ −∞−
∫
∫−
λ
λ
λλ
λλ
λ
λ
λλ
λ
10)(
0)(
)(
))(()(
0
00
00
xE
exE
dxexexE
dxeexxE
x
x
xx
x
Var(x) = E(x2 )−[E(x)]2
Iyer - Lecture 13
ECE 313 - Fall 2013
Variance of Exponential Distribution (cont.)
• Now, we need to determine E(x2) so we can calculate the variance:
• Now, we need to do integration by parts again:
• Now we use the integration by parts formula again:
• Now, remember that and we will be able to substitute it into the equation:
dxexxE xλλ −∞
∫= 0
22 )(
dxedvdxxdu
evxu
x
x
λ
λ
λλ
λ
−
−
−==
−==
2
2
[ ] dxexexxE
dxexexxE
xx
xx
λλ
λλ
λλ
λλ
λ
−∞∞−
∞ −∞−
∫
∫
+=
−+⎥
⎦
⎤⎢⎣
⎡ −=
0022
00
22
2)(
))(2())(()(
λλ λ 1)(
0== −∞
∫ dxexxE x
Iyer - Lecture 13
ECE 313 - Fall 2013
Variance of Exponential Distribution (cont.)
⎟⎠
⎞⎜⎝
⎛=
⎟⎠
⎞⎜⎝
⎛⎟⎠
⎞⎜⎝
⎛+=
22
2
2)(
120)(
λ
λλ
xE
xE
• Now that we have found and we can substitute them into the equation to find the following holds true:
λλ2)( and 1)( == xExE22 )]([)()( xExExVar −=
2
22
1)(
12)(
λ
λλ
=
−=
xVar
xVar