NIPRL

Chapter 2. Random Variables

2.1 Discrete Random Variables2.2 Continuous Random Variables2.3 The Expectation of a Random Variable2.4 The Variance of a Random Variable2.5 Jointly Distributed Random Variables2.6 Combinations and Functions of Random Variables

NIPRL

2.1 Discrete Random Variable2.1.1 Definition of a Random Variable (1/2)

• Random variable – A numerical value to each outcome of a particular

experiment

S

0 21 3-1-2-3

R

NIPRL

2.1.1 Definition of a Random Variable (2/2)

• Example 1 : Machine Breakdowns– Sample space : – Each of these failures may be associated with a repair

cost– State space : – Cost is a random variable : 50, 200, and 350

{ , , }S electrical mechanical misuse

{50,200,350}

NIPRL

2.1.2 Probability Mass Function (1/2)

• Probability Mass Function (p.m.f.)– A set of probability value assigned to each of the

values taken by the discrete random variable– and – Probability :

ip

ix

0 1ip 1iip

( )i iP X x p

NIPRL

2.1.2 Probability Mass Function (1/2)

• Example 1 : Machine Breakdowns– P (cost=50)=0.3, P (cost=200)=0.2,

P (cost=350)=0.5– 0.3 + 0.2 + 0.5 =1 50 200 350

0.3 0.2 0.5

ix

ip( )f x

0.5

0.3

50 200 350 Cost($)

0.2

NIPRL

2.1.3 Cumulative Distribution Function (1/2)

• Cumulative Distribution Function– Function : – Abbreviation : c.d.f

( ) ( )F x P X x :

( ) ( )y y x

F x P X y

( )F x1.0

0.5

0.3

50 200 3500 ($cost)x

NIPRL

2.1.3 Cumulative Distribution Function (2/2)

• Example 1 : Machine Breakdowns

50 ( ) (cost ) 0

50 200 ( ) (cost ) 0.3

200 350 ( ) (cost ) 0.3 0.2 0.5

350 ( ) (cost ) 0.3 0.2 0.5 1.0

x F x P x

x F x P x

x F x P x

x F x P x

NIPRL

2.2 Continuous Random Variables 2.2.1 Example of Continuous Random Variables (1/1)

• Example 14 : Metal Cylinder Production– Suppose that the random variable is the diameter

of a randomly chosen cylinder manufactured by the company. Since this random variable can take any value between 49.5 and 50.5, it is a continuous random variable.

X

NIPRL

2.2.2 Probability Density Function (1/4)

• Probability Density Function (p.d.f.)– Probabilistic properties of a continuous random variable

( ) 0f x

( ) 1statespace

f x dx

NIPRL

2.2.2 Probability Density Function (2/4)

• Example 14– Suppose that the diameter of a metal cylinder has a p.d.f

2( ) 1.5 6( 50.2) for 49.5 50.5

( ) 0, elsewhere

f x x x

f x

( )f x

x49.5 50.5

NIPRL

2.2.2 Probability Density Function (3/4)

• This is a valid p.d.f.

50.5 2 3 50.549.549.5

3

3

(1.5 6( 50.0) ) [1.5 2( 50.0) ]

[1.5 50.5 2(50.5 50.0) ]

[1.5 49.5 2(49.5 50.0) ]

75.5 74.5 1.0

x dx x x

NIPRL

2.2.2 Probability Density Function (4/4)

• The probability that a metal cylinder has a diameter between 49.8 and 50.1 mm can be calculated to be

50.1 2 3 50.149.849.8

3

3

(1.5 6( 50.0) ) [1.5 2( 50.0) ]

[1.5 50.1 2(50.1 50.0) ]

[1.5 49.8 2(49.8 50.0) ]

75.148 74.716 0.432

x dx x x

( )f x

x49.5 50.549.8 50.1

NIPRL

2.2.3 Cumulative Distribution Function (1/3)

• Cumulative Distribution Function

( ) ( ) ( )

( )( )

xF x P X x f y dy

dF xf x

dx

( ) ( ) ( )

( ) ( )

( ) ( )

P a X b P X b P X a

F b F a

P a X b P a X b

NIPRL

2.2.2 Probability Density Function (2/3)

• Example 14

2

49.5

349.5

3 3

3

( ) ( ) (1.5 6( 50.0) )

[1.5 2( 50.0) ]

[1.5 2( 50.0) ] [1.5 49.5 2(49.5 50.0) ]

1.5 2( 50.0) 74.5

x

x

F x P X x y dy

y y

x x

x x

3

3

(49.7 50.0) (50.0) (49.7)

(1.5 50.0 2(50.0 50.0) 74.5)

(1.5 49.7 2(49.7 50.0) 74.5)

0.5 0.104 0.396

P X F F

NIPRL

2.2.2 Probability Density Function (3/3)

( )F x

( 49.7) 0.104P X

x49.5 50.549.7 50.0

1

( 50.0) 0.5P X

(49.7 50.0) 0.396P X

NIPRL

2.3 The Expectation of a Random Variable 2.3.1 Expectations of Discrete Random Variables (1/2)

• Expectation of a discrete random variable with p.m.f

• Expectation of a continuous random variable with p.d.f f(x)

• The expected value of a random variable is also called the mean of the random variable

( )i iP X x p

( ) i ii

E X p x

state space( ) ( )E X xf x dx

NIPRL

2.3.1 Expectations of Discrete Random Variables (2/2)

• Example 1 (discrete random variable)– The expected repair cost is

(cost) ($50 0.3) ($200 0.2) ($350 0.5) $230E

NIPRL

2.3.2 Expectations of Continuous Random Variables (1/2)

• Example 14 (continuous random variable)– The expected diameter of a metal cylinder is

– Change of variable: y=x-50

50.5 2

49.5( ) (1.5 6( 50.0) )E X x x dx

0.5 2

0.5

0.5 3 2

0.5

4 3 2 0.50.5

( ) ( 50)(1.5 6 )

( 6 300 1.5 75)

[ 3 / 2 100 0.75 75 ]

[25.09375] [ 24.90625] 50.0

E x y y dy

y y y dy

y y y y

NIPRL

2.3.2 Expectations of Continuous Random Variables (2/2)

• Symmetric Random Variables– If has a p.d.f that is

symmetric about a point so that

– Then, (why?)

– So that the expectation of the random variable is equal to the point of symmetry

( )f x

( ) ( )f x f x

x

( )E X

( )E X ( )f x

x

NIPRL

( ) ( )

( ) ( )

2

( ) ( )

E X xf x dx

xf x dx xf x dx

y x

xf x dx yf y dy

-

- -

+

+

NIPRL

2.3.3 Medians of Random Variables (1/2)

• Median– Information about the “middle” value of the random

variable

• Symmetric Random Variable– If a continuous random variable is symmetric about a

point , then both the median and the expectation of the random variable are equal to

( ) 0.5F x

NIPRL

2.3.3 Medians of Random Variables (2/2)

• Example 14

3( ) 1.5 2( 50.0) 74.5 0.5F x x x

50.0x

NIPRL

2.4 The variance of a Random Variable 2.4.1 Definition and Interpretation of Variance (1/2)

• Variance( )– A positive quantity that measures the spread of the

distribution of the random variable about its mean value– Larger values of the variance indicate that the

distribution is more spread out

– Definition:

• Standard Deviation– The positive square root of the variance– Denoted by

2

2 2

Var( ) (( ( )) )

( ) ( ( ))

X E X E X

E X E X

2

NIPRL

2.4.1 Definition and Interpretation of Variance (2/2)

2

2 2

2 2

2 2

Var( ) (( ( )) )

( 2 ( ) ( ( )) )

( ) 2 ( ) ( ) ( ( ))

( ) ( ( ))

X E X E X

E X XE X E X

E X E X E X E X

E X E X

Two distribution with identical mean values but different variances

( )f x

x

NIPRL

2.4.2 Examples of Variance Calculations (1/1)

• Example 1

2 2

2 2 2

2

Var( ) (( ( )) ) ( ( ))

0.3(50 230) 0.2(200 230) 0.5(350 230)

17,100

i ii

X E X E X p x E X

17,100 130.77

NIPRL

• Chebyshev’s Inequality– If a random variable has a mean and a variance ,

then

– For example, taking gives

2

2

1( ) 1

for 1

P c X cc

c

2c

2

1( 2 2 ) 1 0.75

2P X

2.4.3 Chebyshev’s Inequality (1/1)

NIPRL

• Proof

2 2 2 2 2

| | | |

( ) ( ) ( ) ( ) ( ) .x c x c

x f x dx x f x dx c f x dx

2(| | ) 1/P x c c

2(| | ) 1 (| | ) 1 1/P x c P x c c

NIPRL

2.4.4 Quantiles of Random Variables (1/2)

• Quantiles of Random variables– The th quantile of a random variable X

– A probability of that the random variable takes a value less than the th quantile

• Upper quartile– The 75th percentile of the distribution

• Lower quartile– The 25th percentile of the distribution

• Interquartile range– The distance between the two quartiles

p

( )F x pp

p

NIPRL

2.4.4 Quantiles of Random Variables (2/2)

• Example 14

– Upper quartile :

– Lower quartile :

– Interquartile range :

3( ) 1.5 2( 50.0) 74.5 for 49.5 50.5F x x x x ( ) 0.75F x 50.17x

( ) 0.25F x 49.83x

50.17 49.83 0.34

NIPRL

2.5 Jointly Distributed Random Variables 2.5.1 Jointly Distributed Random Variables (1/4)

• Joint Probability Distributions– Discrete

– Continuous

( , ) 0

satisfying 1

i j ij

iji j

P X x Y y p

p

state space( , ) 0 satisfying ( , ) 1f x y f x y dxdx

NIPRL

2.5.1 Jointly Distributed Random Variables (2/4)

• Joint Cumulative Distribution Function– Discrete

– Continuous

( , ) ( , )i jF x y P X x Y y

: :

( , )i j

iji x x j y y

F x y p

( , ) ( , )x y

w zF x y f w z dzdw

NIPRL

2.5.1 Jointly Distributed Random Variables (3/4)

• Example 19 : Air Conditioner Maintenance– A company that services air conditioner units in

residences and office blocks is interested in how to schedule its technicians in the most efficient manner

– The random variable X, taking the values 1,2,3 and 4, is the service time in hours

– The random variable Y, taking the values 1,2 and 3, is the number of air conditioner units

NIPRL

2.5.1 Jointly Distributed Random Variables (4/4)

• Joint p.m.f

• Joint cumulative distribution function

Y=number of units

X=service time

1 2 3 4

1 0.12 0.08 0.07 0.05

2 0.08 0.15 0.21 0.13

3 0.01 0.01 0.02 0.07

0.12 0.18

0.07 1.00

iji j

p

11 12 21 22(2,2)

0.12 0.18 0.08 0.15

0.43

F p p p p

NIPRL

2.5.2 Marginal Probability Distributions (1/2)

• Marginal probability distribution– Obtained by summing or integrating the joint probability

distribution over the values of the other random variable– Discrete

– Continuous

( ) i ijj

P X i p p

( ) ( , )Xf x f x y dy

NIPRL

2.5.2 Marginal Probability Distributions (2/2)

• Example 19– Marginal p.m.f of X

– Marginal p.m.f of Y

3

11

( 1) 0.12 0.08 0.01 0.21jj

P X p

4

11

( 1) 0.12 0.08 0.07 0.05 0.32ii

P Y p

NIPRL

• Example 20: (a jointly continuous case)

• Joint pdf:

• Marginal pdf’s of X and Y:

( , )f x y

( ) ( , )

( ) ( , )

X

Y

f x f x y dy

f y f x y dx

NIPRL

2.5.3 Conditional Probability Distributions (1/2)

• Conditional probability distributions– The probabilistic properties of the random variable X

under the knowledge provided by the value of Y– Discrete

– Continuous

– The conditional probability distribution is a probability distribution.

|

( , )( | )

( )ij

i jj

pP X i Y jp P X i Y j

P Y j p

|

( , )( )

( )X Y yY

f x yf x

f y

NIPRL

2.5.3 Conditional Probability Distributions (2/2)

• Example 19– Marginal probability distribution of Y

– Conditional distribution of X

3( 3) 0.01 0.01 0.02 0.07 0.11P Y p

131| 3

3

0.01( 1| 3) 0.091

0.11Y

pp P X Y

p

NIPRL

2.5.4 Independence and Covariance (1/5)

• Two random variables X and Y are said to be independent if– Discrete

– Continuous

– How is this independency different from the independence among events?

for all values of and ofij i jp p p i X j Y

( , ) ( ) ( )X Yf x y f x f y for all x and y

NIPRL

2.5.4 Independence and Covariance (2/5)

• Covariance

– May take any positive or negative numbers.– Independent random variables have a covariance of zero– What if the covariance is zero?

Cov( , ) (( ( ))( ( )))

( ) ( ) ( )

X Y E X E X Y E Y

E XY E X E Y

Cov( , ) (( ( ))( ( )))

( ( ) ( ) ( ) ( ))

( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( )

X Y E X E X Y E Y

E XY XE Y E X Y E X E Y

E XY E X E Y E X E Y E X E Y

E XY E X E Y

NIPRL

2.5.4 Independence and Covariance (3/5)

• Example 19 (Air conditioner maintenance)

( ) 2.59, ( ) 1.79E X E Y 4 3

1 1

( )

(1 1 0.12) (1 2 0.08)

(4 3 0.07) 4.86

iji j

E XY ijp

Cov( , ) ( ) ( ) ( )

4.86 (2.59 1.79) 0.224

X Y E XY E X E Y

NIPRL

2.5.4 Independence and Covariance (4/5)

• Correlation:

– Values between -1 and 1, and independent random variables have a correlation of zero

Cov( , )Corr( , )

Var( )Var( )

X YX Y

X Y

NIPRL

2.5.4 Independence and Covariance (5/5)

• Example 19: (Air conditioner maintenance)

Var( ) 1.162, Var( ) 0.384X Y

Cov( , )Corr( , )

Var( )Var( )

0.2240.34

1.162 0.384

X YX Y

X Y

NIPRL

• What if random variable X and Y have linear relationship, that is,

where

That is, Corr(X,Y)=1 if a>0; -1 if a<0.

Y aX b 0a

2 2

2 2

( , ) [ ] [ ] [ ]

[ ( )] [ ] [ ]

[ ] [ ] [ ] [ ]

( [ ] [ ]) ( )

Cov X Y E XY E X E Y

E X aX b E X E aX b

aE X bE X aE X bE X

a E X E X aVar X

2

( , ) ( )( , )

( ) ( ) ( ) ( )

Cov X Y aVar XCorr X Y

Var X Var Y Var X a Var X

NIPRL

2.6 Combinations and Functions of Random Variables

2.6.1 Linear Functions of Random Variables (1/4)• Linear Functions of a Random Variable

– If X is a random variable and for some numbers thenand

• Standardization-If a random variable has an expectation of and a variance of ,

has an expectation of zero and a variance of one.

Y aX b ,a bR ( ) ( )E Y aE X b

2Var( ) Var( )Y a X

X 2 1X

Y X

NIPRL

2.6.1 Linear Functions of Random Variables (2/4)

• Example 21:Test Score Standardization– Suppose that the raw score X from a particular testing

procedure are distributed between -5 and 20 with an expected value of 10 and a variance 7. In order to standardize the scores so that they lie between 0 and 100, the linear transformation is applied to the scores.

4 20Y X

NIPRL

2.6.1 Linear Functions of Random Variables (3/4)

– For example, x=12 corresponds to a standardized score of y=(4ⅹ12)+20=68

( ) 4 ( ) 20 (4 10) 20 60E Y E X

2 2Var( ) 4 Var( ) 4 7 112Y X

112 10.58Y

NIPRL

2.6.1 Linear Functions of Random Variables (4/4)

• Sums of Random Variables– If and are two random variables, then

and

– If and are independent, so that then

1X 2X

1 2 1 2( ) ( ) ( ) ( ?)E X X E X E X why

1 2 1 2 1 2Var( ) Var( ) Var( ) 2Cov( , )X X X X X X

1X 2X 1 2Cov( , ) 0X X

1 2 1 2Var( ) Var( ) Var( )X X X X

NIPRL

• Properties of 1 2( , )Cov X X

1 2 1 2 1 2( , ) [ ] [ ] [ ]Cov X X E X X E X E X

1 2 2 1( , ) ( , )Cov X X Cov X X

1 1 1( , ) ( )Cov X X Var X 2 2 2( , ) ( )Cov X X Var X

1 2 1 1 1 2 1( , ) ( , ) ( , )Cov X X X Cov X X Cov X X

1 2 1 2 1 1 1 2

2 1 2 2

( , ) ( , ) ( , )

( , ) ( , )

Cov X X X X Cov X X Cov X X

Cov X X Cov X X

1 2 1 2 1 2( ) ( ) ( ) 2 ( , )Var X X Var X Var X Cov X X

NIPRL

2.6.2 Linear Combinations of Random Variables (1/5)

• Linear Combinations of Random Variables– If is a sequence of random variables and

and are constants, then

– If, in addition, the random variables are independent, then

1, , nX X1, , na a

b

1 1 1 1( ) ( ) ( )n n n nE a X a X b a E X a E X b

2 21 1 1 1Var( ) Var( ) Var( )n n n na X a X b a X a X

NIPRL

2.6.2 Linear Combinations of Random Variables (2/5)

• Averaging Independent Random Variables– Suppose that is a sequence of independent

random variables with an expectation and a variance .

– Let

– Then

and

– What happened to the variance?

1, , nX X 2

1 nX XX

n

( )E X

2

Var( )Xn

NIPRL

2.6.2 Linear Combinations of Random Variables (3/5)

1 1

1 1 1 1( ) ( ) ( )

1 1

n nE X E X X E X E Xn n n n

n n

2 2

1 1

2 2 22 2

1 1 1 1Var( ) Var Var( ) Var( )

1 1

n nX X X X Xn n n n

n n n

NIPRL

2.6.2 Linear Combinations of Random Variables (4/5)

• Example 21– The standardized scores of the two tests are

– The final score is

1 1 2 2

10 5 50and

3 3 3Y X Y X

1 2 1 2

2 1 20 5 50

3 3 9 9 9Z Y Y X X

NIPRL

2.6.2 Linear Combinations of Random Variables (5/5)

– The expected value of the final score is

– The variance of the final score is

1 2

20 5 50( ) ( ) ( )

9 9 920 5 50

18 309 9 9

62.22

E Z E X E X

1 2

2 2

1 2

2 2

20 5 50Var( ) Var

9 9 9

20 5Var( ) Var( )

9 9

20 524 60 137.04

9 9

Z X X

X X

NIPRL

2.6.3 Nonlinear Functions of a Random Variable (1/3)

• Nonlinear function of a Random Variable– A nonlinear function of a random variable X is another

random variable Y=g(X) for some nonlinear function g.

– There are no general results that relate the expectation and variance of the random variable Y to the expectation and variance of the random variable X

2 , , XY X Y X Y e

NIPRL

2.6.3 Nonlinear Functions of a Random Variable (2/3)

– For example,

– Consider

– What is the pdf of Y?

( ) 1 for 0 1

( ) 0 elsewhereXf x x

f x

0 1 x

f(x)=1 E(x)=0.

5

1.0 2.718 y

f(y)=1/y E(y)=1.71

8

( ) for 0 1XF x x x

where 1 2.718

X

Y

Y e

NIPRL

2.6.3 Nonlinear Functions of a Random Variable (3/3)• CDF methd

– The p.d.f of Y is obtained by differentiation to be

• Notice that

( ) ( ) ( ) ( ln( )) (ln( )) ln( )XY xF y P Y y P e y P X y F y y

for 1 2.718( ) 1

( ) YY y

dF yf y

dy y

2.718 2.718

1 1

( ) 0.5

( ) ( ) 1 2.718 1 1.718

( ) 1.649

Yz z

E X

E Y zf z dz dz

E Y e e

NIPRL

• Determining the p.d.f. of nonlinear relationship between r.v.s:

Given and ,what is ?

If are all its real roots, that is,

then where

( )Xf x ( )Y g X ( )Yf y

1 2, , , nx x x

1 2( ) ( ) ( )ny g x g x g x

1

( )( )

| '( ) |

nX i

Yi i

f xf y

g x

( )'( )

dg xg x

dx

NIPRL

• Example: determine

-> one root is possible in

( ) 1 for 0 1

( ) 0 elsewhereXf x x

f x

where 1 2.718

X

Y

Y e

( )Yf y

( )

ln

xy g x e

y x

xdge y

dx

1 1( )

| |Yf yy y

0 1x