LECTURE 6

TRANSFORMATION OF RANDOM VARIABLES

TRANSFORMATION OF FUNCTION OF A RANDOM VARIABLE

UNIVARIATE TRANSFORMATIONS

TRANSFORMATION OF RANDOM VARIABLES

• If X is an rv with cdf F(x), then Y=g(X) is also an rv.

• If we write y=g(x), the function g(x) defines a mapping from the original sample space of X, S, to a new sample space, , the sample space

of the rv Y.

g(x): S

1-TO-1 TRANSFORMATION OF RANDOM VARIABLES

• Let y=g(x) define a 1-to-1 transformation. That is, the equation y=g(x) can be solved uniquely:

• Ex: Y=X-1 X=Y+1 1-to-1

• Ex: Y=X² X=± sqrt(Y) not 1-to-1

• When transformation is not 1-to-1, find disjoint partitions of S for which

transformation is 1-to-1.

)y(gx 1

If X is a discrete r.v. then S is countable. The sample space for Y=g(X) is ={y:y=g(x),x S}, also

countable. The pmf for Y is

1 1

Y

x g y x g y

f y P Y y P X x f x

Example

• Let X~GEO(p). That is,

• Find the p.m.f. of Y=X-1

• Solution: X=Y+1

• P.m.f. of the number of failures before the first success

• Recall: X~GEO(p) is the p.m.f. of number of Bernoulli trials required to get the first success

,...3,2,1xfor)p1(p)x(f 1x

,...2,1,0yfor)p1(p)1y(f)y(f yXY

Example

• Suppose X~Poisson with

• Let Y=4X X=Y/4

• Then

,...2,1,0!

)(

xx

exp

x

/ 4

( / 4) 0,4,8,...( / 4)!

yep x y y

y

CONTINUOUS RANDOM VARIABLE

• Let X be an rv of the continuous type with pdf f. Let y=g(x) be differentiable for all x and non-zero. Then, Y=g(X) is also an rv of the continuous type with pdf given by

..0

|)(|))(()(

11

wo

yforygdy

dygf

yh

• Let X have the density

1, 0 1

0, otherwise

xf x

Let Y=eX. X=g1 (y)=lnY dx=(1/y)dy.

11. ,0 log 1

1, 1

0, otherwise

h y yy

y eyh y

Example

Example

• Suppose X has an exponential distribution with

• Let Y=4X-2 X=(Y+2)/4

/1, 0, 0x

Xf x e x

X=g1 (y)= (Y+2)/4 dx=(1/4)dy.

( 2) / 41 1| | 2, 0

( ) 4

0 . .

ye yh y

o w

TRANSFORMATION THAT ARE NOT 1-TO-1

• Let y=g(x) if the equation y=g(x) can not be solved uniquely then not one to one transformation should be used.

When transformation is not 1-to-1, find disjoint partitions of S for which transformation is 1-to-1 and use 1-to-1 transformation or use cumulative method.

Example

• Let X be an rv with pmf

Let Y=|X| S ={2, 1,0,1,2} ={0,1,4}

4 1( ) 2, 1,0,1,2

31 2

x

p x x

4(0)

31

10(1) ( 1) (1)

31

17(2) ( 2) (2)

31

y

y x x

y x x

p

p p p

p p p

Example

• Let X be an rv with pmf

1/5, 2

1/ 6, 1

1/5, 0

1/15, 1

11/30, 2

x

x

p x x

x

x

Let Y=X2. S ={2, 1,0,1,2} ={0,1,4} 1/5, 0

( ) 7 /30, 1

17 /30, 4

y

p y y

y

Example • Stores located on a linear city with density f(x)=0.05

-10≤x ≤ 10, 0 otherwise

• Courier incurs a cost of U=16X2 when she delivers to a store located at X (her office is located at 0)

1600080

)()(

160004044

05.005.0)()(

44

416

2/1

4

4

2

uu

du

udFuf

uuuu

dxuUPuF

uX

uuU

uXuXuU

UU

u

uU

• Let X have the density

2 / 21, .

2

xf x e x

Let W=X2. Find the pdf of W.

Example

First step

Find the distribution function of W

G(w) = P[W ≤ w] = P[ X2 ≤ w]

if 0P w X w w

2

21

2

w x

w

e dx

F w F w

where 2

21

2

x

F x f x e

d wd w

F w F wdw dw

Second step

Find the density function of W

g(w) = G'(w).

1 1

2 2 2 21 1 1 1

2 22 2

w w

e w e w

1 1

2 21 1

2 2f w w f w w

1

2 21

if 0.2

w

w e w

TRANSFORMATION OF FUNCTION OF TWO OR MORE RANDOM

VARIABLES

BIVARIATE TRANSFORMATIONS

DISCRETE CASE

• Let X1 and X2 be a bivariate random vector with a

known probability distribution function.

• Consider a new bivariate random vector (U, V)

defined by U=g1(X1, X2) and V=g2(X1, X2) where

g1(X1, X2) and g2(X1, X2) are some functions of X1

and X2 .

• Then, the joint pmf of (U,V) is

VUAxx

XXVU xxfvVuUvuf,21

21

,

21,, ,,Pr,

EXAMPLE • Let X1 and X2 be independent Poisson distribution

random variables with parameters 1 and 2 with joint pmf .

• Find the distribution of Y1=X1+X2. We need to define

new variable Y2=X2. Then Y1=0,1,2,…, Y2=0,1,2,…,Y1 and X2=Y2 and X1=Y1-Y2.

1 2 1 2

1 21 2 1 2

1 2

( , ) 0,1,2,..., 0,1,2,...,! !

x xe

p x x x xx x

1 2 2 1 2

1 21 2 1 2 1

1 2 2

( , ) 0,1,2,..., 0,1,2,...,( )! !

y y ye

p y y y y yy y y

1 2 2 1 1 21 21 1

2 2

1 2 1 1 21 1 20 0

1 1 2 2 1

! ( )( ) ( , )

! ( )! ! !

y y y yy y

y y

y eep y p y y

y y y y y

CONTINUOUS CASE • Let X=(X1, X2, …, Xn) have a continuous joint

distribution for which its joint pdf is f, and consider

the joint pdf of new random variables Y1, Y2,…, Yk

defined as

*

X,,X,XgY

X,,X,XgY

X,,X,XgY

nkk

n

n

21

2122

2111

• If the transformation T is one-to-one and onto, then

there is no problem of determining the inverse

transformation, and we can invert the equation in (*)

and obtain new equations as

**

y,,y,ygx

y,,y,ygx

y,,y,ygx

nknn

k

k

211

211

22

211

11

Assuming that the partial derivatives exist at every point (y1, y2,…,yk=n).

ii y/g 1

• Under these assumptions, we have the following determinant J

called as the Jacobian of the transformation specified by (**). Then, the joint pdf of Y1, Y2,…,Yk can be obtained by using the change of variable technique of multiple variables.

n

nn

n

y

g

y

g

y

g

y

g

detJ1

1

1

11

1

11

• As a result, the new p.d.f. is defined as follows:

otherwise

yyyJgggfyyyg

nnXX

nn

,0

,,,for |,|,,,,,,

21

11

2

1

1,,

211

Example

Let X1 and X2 ~ Exp(1)

Consider the random variables Y1=X1, Y2=X1+X2. Find pdf of Y1 and Y2?

X1=Y1 and X2=Y2-Y1 the Jacobian is

Then

1 2

1 2

( )

, 1 2 1 2( , ) 0, 0x x

x xf x x e x x

1 01

1 1

2

1 2 1 2, 1 2 , 1 2 1 1 2( , ) ( , ) 0y

y y x xf y y f y y y e y y

METHOD OF CONDITIONING

• U=h(X1,X2)

• Find f(u|x2) by transformations (Fixing X2=x2)

• Obtain the joint density of U, X2:

• f(u,x2) = f(u|x2)f(x2)

• Obtain the marginal distribution of U by integrating joint density over X2

222 )()|()( dxxfxufufU

Example • X1~Beta(a2,b2 X2~Beta(a3,b1 Independent

• U=X1X2

• Fix X2=x2 and get f(u|x2)

10))ln(1(18

)ln(1818018)ln(181818

18)()|()(

1011831

)/1)(/(6)()|(),(

01

)/1)(/(6)|(

/1/

103)(10)1(6)(

221

2

2

22

1

2

21

222

2

2

2

2

2

22222

2

2

222

21

2121

2

2

221111

uuuuu

uuuuxuuxdxx

uudxxfxufuf

xux

uux

xxuxuxfxufxuf

xux

xuxuxuf

xdU

dXxUXxXU

xxxfxxxxf

uuuU

Example

• X1, X2 independent Exponential(q)

• f(xi)=q-1e-xi/q xi>0, q>0, i=1,2

• f(x1,x2)= q-2e-(x1+x2)/q x1,x2>0

• U=X1+X2

),2(~01

11)(

11

111

1

11)(

,

/

2

/

2

////

2

/)(

02

/

02

/)(/

0

20

//

021

//

0 0 2

21221

2121

22222

21221

2

qbaq

qqqq

qqq

qq

q

qqqqq

qqqq

qqqq

GammaUuue

eu

eeufuee

dxedxedxee

dxeedxdxeeuUP

XuXuXuXXuU

xuXuXXuU

u

uuu

U

uu

xuxu

xu

xuxu

xuxx

uxx

u xu

M.G.F. Method

• If X1,X2,…,Xn are independent random variables with MGFs Mxi (t), then the MGF of is

n

1i

iXY )t(M)...t(M)t(MnX1XY

Example

Then find the pmf of

Let ~ ,independent

i iX Bin n p

1 2

1

1

1

...1

~ , .

( ) ( )... ( )

( ) ...( )

( )

k

i ki

Y X Xk

nnt t k

n nt k

X Bin n n n p

M t M t M t

pe q pe q

pe q

1

k

ii

Y X

1

1

...1

( ) ( )... ( )

( ) ...( )

( )

Y X Xk

nnt t k

n nt k

M t M t M t

pe q pe q

pe q

Example

ba

bbb

b

ba

aaa

a

,~

)1()1()1(

)()()(

,...,1)1()(

nt)(independe,...,1),(~

11

)...(

1

11

1

11

n

i

i

n

i

i

XX

tXtXXXttY

Y

n

i

i

X

ii

GammaXY

ttt

tMtMeeEeEeEtM

XY

nittM

niGammaX

n

i in

n

nn

i

i

Example

n

i

ii

n

i

ii

n

i

ii

n

i iin

i iinn

nn

nXX

XtaXtaXaXattY

Y

i

n

i

ii

iiX

iii

aaNormalXaY

tata

tata

tata

taMtaMeeEeEeEtM

aXaY

nit

ttM

niNormalX

n

nnnn

i

1

22

11

2

1

22

1

222

1

2

111

1

)...(

1

22

2

,~

2exp

2exp

2exp

)()()(

constants fixed }{

,...,12

exp)(

nt)(independe,...,1),(~

1

1111

ORDER STATISTICS

ORDER STATISTICS

• Let X1, X2,…,Xn be a r.s. of size n from a distribution of continuous type having pdf f(x), a<x<b. Let X(1) be the smallest of Xi, X(2) be the second smallest of Xi,…, and X(n) be the largest of Xi.

1 2 na X X X b

• X(i) is the i-th order statistic.

1 21

1 2

min , , ,

max , , ,

n

nn

X X X X

X X X X

ORDER STATISTICS

• It is often useful to consider ordered random sample.

• Example: suppose a r.s. of five light bulbs is tested and the failure times are observed as (5,11,4,100,17). These will actually be observed in the order of (4,5,11,17,100). Interest might be on the kth smallest ordered observation, e.g. stop the experiment after kth failure. We might also be interested in joint distributions of two or more order statistics or functions of them.

JOINT PDF OF THE OREDER STATISTICS

• If X1, X2,…,Xn is a r.s. of size n from a population with continuous pdf f(x), then the joint pdf of the order statistics X(1), X(2),…,X(n) is

1 2 1 2, , , !

n ng x x x n f x f x f x

The joint pdf of ordered sample is not same as the joint pdf of unordered sample.

)n()1( x...xfor

Order statistics are not independent.

Note: For discrete distributions, we need to take ties into account (two X’s being equal).

Find the joint pdf of the order statistics for the uniform distribution, the standard exponential distribution and normal distribution?

Solution: p.d.f for the uniform is:

( ) 1, 0 1f x x

1 1 1 1( , ,..., ) !, 0 ..., 1,n ng y y y n y y y

Example

Solution: p.d.f for the standard Exponential distribution is:

( ) , 0xf x e x

1

1 1 1 1( , ,..., ) ! , 0 ..., ,

n

i

i

y

n ng y y y n e y y y

Solution: p.d.f for the standard normal distribution is:

2

21

( ) ,2

x

f x e x

2

12

1 1 1 1

!( , ,..., ) , ..., ,

2

ni

i

y

n n

ng y y y e y y y

Example

• Suppose that X1,X2 and X3 represent a random sample of size 3 from population with pdf

• Joint pdf of order statistics Y1,Y2 and Y3?

• Marginal pdf of Y1

( ) 2 0 1f x x x

1 2 3 1 2 3 1 2 3 1 2 3( , , ) 3!(2 )(2 )(2 ) 48 0 1g y y y y y y y y y y y y

1 1

2 2

1 1 1 2 3 2 3 1 1 1

1 2

( ) 48 6 (1 ) 0 1y y

g y y y y dy dy y y y

THE MARGINAL DISTRIBUTIONS FOR THE ODER STATISTICS

Theorem: (p.d.f of the rth order statistics) If X1, X2,…,Xn be a r.s. of size n from a population with continuous pdf f(x), then the p.d.f. of the rth order statistics X(r) is given as:

1 1!( ) ( ) [ ( ) ] [1 ( ) ] ,

( 1)!( )!

r n

r r r r r r

ng y f y F y F y y

r n r

• r-th Order Statistic

y y1 y2 yr-1 yr yr+1 yn … …

P(X<yr) P(X>yr)

f(yr)

# of possible orderings n!/{(r1)!1!(n r)!}

1 1

!( ) ( )

( 1)!( )!

[ ( ) ] [1 ( ) ] ,

r r r

r n

r r r

ng y f y

r n r

F y F y y

Example

• X~Uniform(0,1). A r.s. of size n is taken. Find the p.d.f. of kth order statistic.

• Solution: Let Yk be the kth order statistic.

)1kn,k(Beta~Y

1y0for)y1(y)1kn()k(

)1n(

1)y1(y)!kn()!1k(

!n)y(g

k

kn1k

kn1kkY

Example

• Suppose that X1,X2,…,Xn represent a random sample of size n from population with pdf

• Marginal pdf of Y1 and Yn?

( ) 2 0 1f x x x

2( ) 0 1F x x x

2 1

1 1 1 1 1

2 1

1

( ) 2 (1 ) 0 1

( ) 2 ( ) 0 1

n

n

n n n n

g y ny y y

g y ny y y

Theorem (p.d.f of the largest order statistics): If X1, X2,…,Xn be a r.s. of size n from a population with continuous pdf f(x), then the p.d.f. of the Largest order statistics Y(n) is given as:

1( ) ( )[ ( )] ,n

n n n n ng y n f y F y y

Theorem: (p.d.f of the smallest order statistics) If X1,

X2,…,Xn be a r.s. of size n from a population with continuous pdf f(x), then the p.d.f. of the smallest order statistics X(1) is given as:

1

1 1 1 1( ) ( )[1 ( )] ,n

ng y n f y F y y

Example

Let are an O. S. of sample size n = 6 and the p.d.f. of this sample is

Find:

Solution:

1( ) , 0 2

2f x x

1 2 6...Y Y Y

1 1 6 6( ) , ( ) , ( )r rg y g y g y

1 6 1

6

6!( ) [2 ] , 0 2

( 1)!(6 )!2

r

r r r r rg y y y yr r

5

1 1 1 16

6( ) [ 2 ] , 0 2

2g y y y

5

6 6 6 66

6( ) , 0 2

2g y y y

JOINT P.D.F. OF i-TH AND j-TH ORDER

STATISTIC (FOR i<j)

Theorem:

If X1, X2,…,Xn be a r.s. of size n from a population with continuous pdf f(x), and Y1< Y2<…<Yn are the order statistics of that sample, then the p.d.f. of the two order statistics Yi< Yj , i<j and i,j = 1,2, …,n is given as

1 1!

( , ) [ ( )] ( )[ ( ) ( )] ( )[1 ( )]( 1)!( 1)!( )!

i j i n j

ij i j i i j i j j

ng y y F y f y F y F y f y F y

i j i n j

• Joint p.d.f. of i-th and j-th Order Statistic (for i<j)

y y1 y2 yi-1 yi yi+1 yn … …

P(X<yi) P(yi<X<yj)

f(yi)

# of possible orderings n!/{(i1)!1!(j-i-1)!1!(n j)!}

yj-1 yj yj+1

i-1 items j-i-1 items n-j items 1 item 1 item

P(X>yj)

f(yj)

1 1!( , ) [ ( )] ( )[ ( ) ( )] ( )[1 ( )]

( 1)!( 1)!( )!

i j i n j

ij i j i i j i j j

ng y y F y f y F y F y f y F y

i j i n j

Example

• Suppose that X1,X2,…,Xn represent a random sample of size n from population with pdf

• Find the density of range R=Yn-Y1?

( ) 2 0 1f x x x

2( ) 0 1F x x x

2 2 2

1, 1 1 1 1

!( , ) 2 2 ( ) 0 1

( 2)!

n

n n n n

ng y y y y y y y

n