Lecture 14: Multivariate Distributions

Probability Theory and Applications

Fall 2005

October 25

Multivariate Distributions

Distributions may have more than one R.V.

Example: S=size of house - real RV P=price of house - real RV A=Age of house - real RV C= condition of house Excellent, Very Good, Good, Poor

- discrete RV

Since variables are not-independent need a multivariate distribution to describe them: f(S,P,A,C)

Bivariate Random Variables

Given R.V. X and YCases1. X,Y both discrete number of blue and red jelly beans picked from jar2. X,Y both continuous height and weight3. X discrete and Y continuous date and stock price

Both Discrete

The joint distribution of (X,Y) is specified by

• The value set of (X,Y)

• The joint probability function

f(x,y)=P(X=x,Y=y)

Note:

• f(x,y)≥0 for any (x,y)

( , ) 1x y

f x y

Discrete Example

Box contains jewels H=high quality

M=medium quality D=defective

You pick two jewels w/o replacementX=# of HY =#of M

3 H

2 M

2 D

2

2 1( 0, 0)

7 21

2

P X Y

Joint Probability Function

X\Y 0 1 2

0 1/21 4/21 1/21 6/21

1 6/21 6/21 0/21 12/21

2 3/21 0/21 0/21 3/21

10/21 10/21 1/21

Joint Probability Function

X\Y 0 1 2

0 1/21 4/21 1/21

1 6/21 6/21 0/21

2 3/21 0/21 0/21

Marginal Probability Functions

X\Y 0 1 2

0 1/21 4/21 1/21 6/21

1 6/21 6/21 0/21 12/21

2 3/21 0/21 0/21 3/21

10/21 10/21 1/21 1( )Yf y

( )Xf x

Definitions

The marginal distribution of X is

Note this is exactly the same as pdf of X

The joint cumulative density function of X,Y is

( ) ( , )Xy valueset

f x f x y

( , ) ( , )F x y P X x Y y

Questions

P(You get one high quality and one medium jewel)?

P(You pick at least one high quality jewel)?

Conditional Distributions

The conditional distribution of Y given X is

In our example:

( , )( | ) ( | )

( )

P Y y X xf y x P Y y X x

P X x

(1)

( 1, ) (1, )( | 1)

( 1) x

P x y f yf y x

P X f

Conditional Probability Functions

X\Y 0 1 2

0 1/21 4/21 1/21 6/21

1 6/21 6/21 0/21 12/21

2 3/21 0/21 0/21 3/21

10/21 10/21 1/21 1( )Yf y

( )Xf x

Y 0 1 2

f(y|X=1)1

6 / 21

2 / 21 1

0 / 21

2 / 211

6 / 21

2 / 21

Conditional Probability Functions

X\Y 0 1 2

0 1/21 4/21 1/21 6/21

1 6/21 6/21 0/21 12/21

2 3/21 0/21 0/21 3/21

10/21 10/21 1/21 1( )Yf y

( )Xf x

X 0 1 2

f(x|Y=1) 4/10 6/10 0

Find distribution ofX given Y=1

Question

Given that exactly one jewel picked is medium quality, what is the probability that the other is high quality?

6/10

Given that at least one jewel picked is medium quality, what is the probability that the other is high quality?

6/11

X,Y both Continuous

The joint pdf, f(x,y) defined over R2 has properties:

• f(x,y)≥0

To calculate probabilities, integrate joint pdf over X,Y over the area

Or more generally if we want P((x,y)A)

( , ) 1f x y dxdy

( , ) ( , )b d

a c

P a X b c Y d f x y dydx

( , ) ) ( , )A

P X Y A f x y dA

X,Y both Continuous

More generally if we want P((x,y)A)

The c.d.f.

( , ) ) ( , )A

P X Y A f x y dA

( , ) ( , )

( , )u v

F x y P X x Y y

f u v dvdu

2 ( , )( , ) . .

F x yf x y c d f

x y

Marginals and Conditionals

The marginal pdf of X

The marginal pdf of Y

The conditional pdf of X given Y=y

( ) ( , )xf x f x y dy

( ) ( , )yf y f x y dx

( , )( | )

( )y

f x yf x y

f y

Examples

The joint pdf of (x,y) is

Find c

( , ) (1 ) 0 1, 0 2f x y cx y for X y

1 2 1 200 0 0

1

0

1 (1 ) (1 ) |

4 2

1/ 2

c x y dydx c x y dx

c xdx c

c

continued

Find pdf of X

Find pdf of Y

2 2

0 0

1( ) ( , ) (1 ) 2

2Xf x f x y dy x y dy x

1 1

0 0

1( ) ( , ) (1 ) 1/ 4(1 )

2Yf y f x y dx x y dx y

2 0 1( )

0 . .X

x xf x

o w

1/ 4(1 ) 0 2( )

0 . .Y

y yf y

o w

continued

Find marginal of X given Y=1

Note this is the same as marginal of X!

X and Y are independent!

( , 1) 1/ 2 (1 1)( | 1) 2

(1) 1/ 4(1 1)Y

f x y xf x y x

f

|

2 0 1( | 1)

0 . .X Y

x xf x y

o w

( , ) 1/ 2 (1 )( | ) 2 0 1

( ) 1/ 4(1 )Y

f x y x yf x y x x

f y y

continued

Find P(X>Y)

1

0 0

21

00

13 3 4

1 2

00

( ) (1 )2

1

2 2

1 1 1(1/ 3 1/ 8) 11/ 48

2 2 2 3 8 2

x

x

xP X Y y dydx

yx y dx

x x xx dx

? ?

? ?( ) (1 )

2

xP X Y y dydx 0 X 1

2

Y

Mixed Continuous and Discrete

Let L a be R.V. that is 1 if candy corn manufactured from Line 1 and 0 if line 0

Let X=weight of candy corn

The joint pdf is

What is the marginal distribution of X – the weight of the candy corn?

2

2

( 7.05)

2

( 10.1)

2(1.44),

10.25

12

1( , ) 0.75

02 1.2

0 . .

x

x

X L

xe

L

xf x l e

L

o w

Mixed Continuous and Discrete

The joint pdf is

Sum over L to find the marginal of X

22 ( 10.1)( 7.05)2(1.44)2

( ) ( ,1) ( ,0)

1 10.25 0.75

2 2 1.2

x

xx

f x f x f x

e e x

2

2

( 7.05)

2

( 10.1)

2(1.44),

10.25

12

1( , ) 0.75

02 1.2

0 . .

x

x

X L

xe

L

xf x l e

L

o w

Conditional Distribution

What is the marginal of L?

What is the conditional X given L?

2

2

( 7.05)

2

( 10.1)

2(1.44)

10.25 0.25 1

2

1( , ) 0.75 0.75 0

2 1.2

.

x

x

L

e dx L

f x l e dx L

2

2

( 7.05)

2

( 10.1)

2(1.44)

11

2

01( | )

2 1.2

.

x

x

Le

x

Lf x l e

x

L is Bernoulli R.V. p=0.25

If candy corn is from Line 1,weight is normal with mean 7.05 and s.d. = 1.If candy corn is from Line 0,weight is normal withmean 10.1 and s.d. = 1.2.

Mixture Model

X is a mixture of two different normals

0 5 10 15

0

0.05

0.1

0.15

0.2

0.25

x

0.25 1/(sqrt(2 )) exp(-(x-7.05)2/2)+0.75 1/(sqrt(2 1.44)) exp(-(x-10.1)2/(2 1.44))

Example 5

Harry Potter plays flips a magical coin 10 times and records the number of heads.

The coin is magical because each day the probability of getting heads changes.

Let Y, the probability of getting heads on a given day, be uniform [0,1]

Let X be the number of heads of 10 gotten on a given day with the magic coin.

What is the pdf of X?

Example 5 continued

Y is uniform [0,1] so

X|Y is binomial n=10 p=Y

So f(X,Y)

( ) 1 0 1Yf y y

1010( , ) (1 ) 0,1, ,10, 0 1x xf X Y y y x y

x

110

0

10( ) (1 ) Note this is a Beta Integral!

10 ( 1) (10 1) 10! !(10 )!_

( 1 10 1) (10 )! ! 11!

10,1, ,10

11

x xXf X y y dy

x

x x x x

x x x x x

x

X is discrete uniformAll values equally likely

Fact

You can compute the joint from a marginal and a conditional.

Be careful how you compute the value sets!

( , ) ( | ) ( )xf x y f y x f x

Example 2 – Two Continuous

The joint pdf of X and Y is

Find marginal of X

( , ) (1 ) 0 1f x y cx y x y

O X 1

Y

1

11 2

2

( ) (1 ) / 2

1 1/ 2 / 2

X x xf x cx y dy cx y y

cx x x

2 3/ 2 / 2 0 1( )

0 . .X

c x x x xf x

o w

Example 2

Still need c

You check:

12 3

0

1 / 2 / 2 / 24 1 24c x x x dx c c

212 (1 ) 0 1( )

0 . .X

x x xf x

o w

212 (1 ) 0 1( )

0 . .Y

y y yf y

o w

continued

Find P(Y<2X)

1 / 2

0 0( , ) 1/ 4

yf x y dxdy 1

0 / 2( , ) 3 / 4

y

yf x y dxdy

OX 1

Y

1

OX 1

Y

1

P(Y≥2X)

Conditional distribution

Find conditional pdf of Y and X=1/2

O X 1

Y

1

(1/ 2, )( | 1/ 2)

(1/ 2)x

f yf y x

f ? ?y 1/ 2 1y

2

24 /(1/ 2)(1 )( | 1/ 2)

12 / 2(1 1/ 2)

8(1 )

yf y x

y

\ 1/ 2

8(1 ) 1/ 2 1( | 1/ 2)

0 . .Y X

y yf y x

o w

Conditional distribution

Find conditional pdf of Y and X=x 0<x<1

O X 1

Y

1 2\ 1/ 2

2(1 )1

( | ) 10 . .

Y X

yx y

f y x xo w

Independence

R.V. X and Y are independent if and only any of the following hold

1. F(x,y)=FX(x)FY(y)

P(X≤x,Y≤y)= P(X≤x)P(Y≤y)

2. f(x,y)=fX(x)fY(y)

3. f(y|x)=fY(y)

Example 3

Given the joint pdf of X,Y

Use the marginal of X and the conditional pdf of Y given X=x to determine if X and Y are independent?

( , ) 8 0 1f x y xy x y

Answer

Find marginal of X

Find conditional of Y given X

1

2

( ) 8

4 (1 ) 0 1

x

x

f x xydy

x x x

O

Y

1

2 2

8 2( | ) 0 1

4 (1 ) 1

xy yf y x x y

x x x

Answer continued

Are they independent?

No

2 3( ) ( ) 4 (1 )4 8 ( , )x yf x f y x x y xy f x y

3

0

( ) 8 4 0 1y

yf y xydx y y

Note

P(Y≤3/4|x=1/2) and P(Y≤3/4|x ≤1/2) are very different things!

Let’s calculate each one

P(Y≤3/4|X=1/2)

The pdf of Y given X=1/2 is

so3/ 4 3/ 4

1/ 2 1/ 2

2( 3/ 4 | 1/ 2) ( | 1/ 2) 5 /12

0.75

yP Y X f y x dy dy

2

21/ 2 1

1 (1/ 2)

yy

P(Y≤3/4|X ≤ 1/2)

The probability Y given X ≤ 1/2 is

where1/ 2 1/ 2

2

0 0

( 1/ 2) ( ) 4 (1 ) 7 /16xP X f x dx x x dx

( 3 / 4, 1/ 2)( 3 / 4 | 1/ 2)

( 1/ 2)

P Y XP Y X

P X

P(Y≤3/4|X ≤ 1/2)

The probability P(Y≤3/4,X ≤ 1/2)

The probability

1/ 2 3/ 4

0

1/ 23/ 42

0

1/ 22

0

( 3 / 4, 1/ 2) 8

4

4 (9 /16 ) 7 / 32

x

x

P Y X xydydx

x y dx

x x dx

O

1

7 / 32( 3/ 4 | 1/ 2) 1/ 2

7 /16P Y X

Example 4

Suppose X has the Gamma distribution with parameters with K=2 and theta=1 and

the conditional distribution of Y given X.

(X>0) is

Find P( X<4| Y=2)

( | ) 1/ 0f y x x y x

( ) 0xxf x e x x

Example 4

We know f(x,y)=f(x|y)fx(x) so the joint is

The marginal of Y is

Thus conditional of X given Y is

1( , ) 0xf x y e x y x

x

( ) 0x yy

y

f y e dx e y

( | ) 0x

y

ef x y y x for y

e

Example 4 continued

So

Thus

Exercise try: P(X>4|Y>2)

1/ 2( | 1/ 2) 1/ 2

xef x y x

e

42

1/ 21/ 2

( 4 | 1/ 2)xe

P x y dx ee

Example 5 – Two Discretes

You write a paper with an average rate of 10 errors per paper. Assume the number of errors per papers follows a Poisson distribution.

You roommate proofreads it for you, and he/she has .8 percent of correcting each error.

What is the joint distributions of the number of errors and the number of corrections?

What is the distribution of the number of errors after you roommate reads the paper?

answer

Let X be the number of errors

Y be the number of errors after correction

Clearly Y depends on X.

Given

What is pdf of Y|X?

binomial(n=X,p=.2)

( ) ~ ( 10)Xf x Poisson

( | ) .2 .8 0, ,y x yxf y x y x

y

answer

Let X be the number of errors

Y be the number of errors after correction

,

x 10

( , ) ( | ) ( )

10.2 .8 0, 0, ,

x!

X Y x

y x y

f x y f y x f x

x ex y x

y

Extra Credit: if you can figure out marginal of Y.