Jointly distributed Random variables

Multivariate distributions

Quite often there will be 2 or more random variables (X, Y, etc) defined for the same random experiment.

Example:

A bridge hand (13 cards) is selected from a deck of 52 cards.

X = the number of spades in the hand.

Y = the number of hearts in the hand.

In this example we will define:

p(x,y) = P[X = x, Y = y]

The function

p(x,y) = P[X = x, Y = y]

is called the joint probability function of X and Y.

Note:

The possible values of X are 0, 1, 2, …, 13

The possible values of Y are also 0, 1, 2, …, 13 and X + Y ≤ 13.

, ,p x y P X x Y y

13 13 26

13

52

13

x y x y

The total number of ways of choosing the 13 cards for the hand

The number of ways of choosing the x spades for the hand

The number of ways of choosing the y hearts for the hand

The number of ways of completing the hand with diamonds and clubs.

Table: p(x,y)

0 1 2 3 4 5 6 7 8 9 10 11 12 13

0 0.0000 0.0002 0.0009 0.0024 0.0035 0.0032 0.0018 0.0006 0.0001 0.0000 0.0000 0.0000 0.0000 0.0000

1 0.0002 0.0021 0.0085 0.0183 0.0229 0.0173 0.0081 0.0023 0.0004 0.0000 0.0000 0.0000 0.0000 -

2 0.0009 0.0085 0.0299 0.0549 0.0578 0.0364 0.0139 0.0032 0.0004 0.0000 0.0000 0.0000 - -

3 0.0024 0.0183 0.0549 0.0847 0.0741 0.0381 0.0116 0.0020 0.0002 0.0000 0.0000 - - -

4 0.0035 0.0229 0.0578 0.0741 0.0530 0.0217 0.0050 0.0006 0.0000 0.0000 - - - -

5 0.0032 0.0173 0.0364 0.0381 0.0217 0.0068 0.0011 0.0001 0.0000 - - - - -

6 0.0018 0.0081 0.0139 0.0116 0.0050 0.0011 0.0001 0.0000 - - - - - -

7 0.0006 0.0023 0.0032 0.0020 0.0006 0.0001 0.0000 - - - - - - -

8 0.0001 0.0004 0.0004 0.0002 0.0000 0.0000 - - - - - - - -

9 0.0000 0.0000 0.0000 0.0000 0.0000 - - - - - - - - -

10 0.0000 0.0000 0.0000 0.0000 - - - - - - - - - -

11 0.0000 0.0000 0.0000 - - - - - - - - - - -

12 0.0000 0.0000 - - - - - - - - - - - -

13 0.0000 - - - - - - - - - - - - -

13 13 26

13

52

13

x y x y

Bar graph: p(x,y)

0 1 2 3 4 5 6 7 8 9 10 11 12 13

0

3

6

9

12

-

0.0100

0.0200

0.0300

0.0400

0.0500

0.0600

0.0700

0.0800

0.0900

x

y

p(x,y)

Example:

A die is rolled n = 5 times

X = the number of times a “six” appears.

Y = the number of times a “five” appears.

Now

p(x,y) = P[X = x, Y = y]

The possible values of X are 0, 1, 2, 3, 4, 5.

The possible values of Y are 0, 1, 2, 3, 4, 5.

and X + Y ≤ 5

A typical outcome of rolling a die n = 5 times will be a sequence F5FF6 where F denotes the outcome {1,2,3,4}. The probability of any such sequence will be:

51 1 4

6 6 6

x y x y

where

x = the number of sixes in the sequence and

y = the number of fives in the sequence

Now

p(x,y) = P[X = x, Y = y]

51 1 4

6 6 6

x y x y

K

Where K = the number of sequences of length 5 containing x sixes and y fives.

5 5 5

5

x x y

x y x y

5 !5! 5!

! 5 ! ! 5 ! ! ! 5 !

x

x x y x y x y x y

Thus

p(x,y) = P[X = x, Y = y]

55! 1 1 4

! ! 5 ! 6 6 6

x y x y

x y x y

if x + y ≤ 5 .

Table: p(x,y)

0 1 2 3 4 5

0 0.1317 0.1646 0.0823 0.0206 0.0026 0.00011 0.1646 0.1646 0.0617 0.0103 0.0006 02 0.0823 0.0617 0.0154 0.0013 0 03 0.0206 0.0103 0.0013 0 0 04 0.0026 0.0006 0 0 0 05 0.0001 0 0 0 0 0

55! 1 1 4

! ! 5 ! 6 6 6

x y x y

x y x y

0 1 2 3 4 5

0

12

34

5

0.0000

0.0200

0.0400

0.0600

0.0800

0.1000

0.1200

0.1400

0.1600

0.1800

Bar graph: p(x,y)

x

y

p(x,y)

General properties of the joint probability function;

p(x,y) = P[X = x, Y = y]

1. 0 , 1p x y

2. , 1x y

p x y

3. , ,P X Y A p x y ,x y A

Example:

A die is rolled n = 5 times

X = the number of times a “six” appears.

Y = the number of times a “five” appears.

What is the probability that we roll more sixes than fives

i.e. what is P[X > Y]?

Table: p(x,y)

0 1 2 3 4 5

0 0.1317 0.1646 0.0823 0.0206 0.0026 0.00011 0.1646 0.1646 0.0617 0.0103 0.0006 02 0.0823 0.0617 0.0154 0.0013 0 03 0.0206 0.0103 0.0013 0 0 04 0.0026 0.0006 0 0 0 05 0.0001 0 0 0 0 0

55! 1 1 4

! ! 5 ! 6 6 6

x y x y

x y x y

, 0.3441P X Y p x y x y

Marginal and conditional distributions

Definition:

Let X and Y denote two discrete random variables with joint probability function

p(x,y) = P[X = x, Y = y]

Then

pX(x) = P[X = x] is called the marginal probability function of X.

pY(y) = P[Y = y] is called the marginal probability function of Y.

and

Note: Let y1, y2, y3, … denote the possible values of Y.

Xp x P X x

1 2, ,P X x Y y X x Y y

1 2, ,P X x Y y P X x Y y

1 2, ,p x y p x y

, ,jj y

p x y p x y Thus the marginal probability function of X, pX(x) is obtained from the joint probability function of X and Y by summing p(x,y) over the possible values of Y.

Also

Yp y P Y y

1 2, ,P X x Y y X x Y y

1 2, ,P X x Y y P X x Y y

1 2, ,p x y p x y

, ,ii x

p x y p x y

Example:

A die is rolled n = 5 times

X = the number of times a “six” appears.

Y = the number of times a “five” appears.

0 1 2 3 4 5 p X (x )

0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001 0.40191 0.1646 0.1646 0.0617 0.0103 0.0006 0 0.40192 0.0823 0.0617 0.0154 0.0013 0 0 0.16083 0.0206 0.0103 0.0013 0 0 0 0.03224 0.0026 0.0006 0 0 0 0 0.00325 0.0001 0 0 0 0 0 0.0001

p Y (y ) 0.4019 0.4019 0.1608 0.0322 0.0032 0.0001

Conditional Distributions

Definition:

Let X and Y denote two discrete random variables with joint probability function

p(x,y) = P[X = x, Y = y]

Then

pX |Y(x|y) = P[X = x|Y = y] is called the conditional probability function of X given Y = y

and

pY |X(y|x) = P[Y = y|X = x] is called the conditional probability function of Y given X = x

Note

X Yp x y P X x Y y

, ,

Y

P X x Y y p x y

P Y y p y

Y Xp y x P Y y X x

, ,

X

P X x Y y p x y

P X x p x

and

• Marginal distributions describe how one variable behaves ignoring the other variable.

• Conditional distributions describe how one variable behaves when the other variable is held fixed

Example:

A die is rolled n = 5 times

X = the number of times a “six” appears.

Y = the number of times a “five” appears.

0 1 2 3 4 5 p X (x )

0 0.1317 0.1646 0.0823 0.0206 0.0026 0.0001 0.40191 0.1646 0.1646 0.0617 0.0103 0.0006 0 0.40192 0.0823 0.0617 0.0154 0.0013 0 0 0.16083 0.0206 0.0103 0.0013 0 0 0 0.03224 0.0026 0.0006 0 0 0 0 0.00325 0.0001 0 0 0 0 0 0.0001

p Y (y ) 0.4019 0.4019 0.1608 0.0322 0.0032 0.0001

x

y

The conditional distribution of X given Y = y.

0 1 2 3 4 5

0 0.3277 0.4096 0.5120 0.6400 0.8000 1.00001 0.4096 0.4096 0.3840 0.3200 0.2000 0.00002 0.2048 0.1536 0.0960 0.0400 0.0000 0.00003 0.0512 0.0256 0.0080 0.0000 0.0000 0.00004 0.0064 0.0016 0.0000 0.0000 0.0000 0.00005 0.0003 0.0000 0.0000 0.0000 0.0000 0.0000

pX |Y(x|y) = P[X = x|Y = y]

x

y

0 1 2 3 4 5

0 0.3277 0.4096 0.2048 0.0512 0.0064 0.00031 0.4096 0.4096 0.1536 0.0256 0.0016 0.00002 0.5120 0.3840 0.0960 0.0080 0.0000 0.00003 0.6400 0.3200 0.0400 0.0000 0.0000 0.00004 0.8000 0.2000 0.0000 0.0000 0.0000 0.00005 1.0000 0.0000 0.0000 0.0000 0.0000 0.0000

The conditional distribution of Y given X = x.

pY |X(y|x) = P[Y = y|X = x]

x

y

Example

A Bernoulli trial (S - p, F – q = 1 – p) is repeated until two successes have occurred.

X = trial on which the first success occurs

and

Y = trial on which the 2nd success occurs.

Find the joint probability function of X, Y.

Find the marginal probability function of X and Y.

Find the conditional probability functions of Y given X = x and X given Y = y,

Solution

A typical outcome would be:

FFF…FSFFF…FSx - 1

x y

y – x - 1

1 1 2 2 if x y x yq pq p q p y x

, ,p x y P X x Y y

2 2 if

, 0 otherwise

yq p y xp x y

p(x,y) - Table

y

1 2 3 4 5 6 7 8

x

1 0 p2 p2q p2q2 p2q3 p2q4 p2q5 p2q6

2 0 0 p2q p2q2 p2q3 p2q4 p2q5 p2q6

3 0 0 0 p2q2 p2q3 p2q4 p2q5 p2q6

4 0 0 0 0 p2q3 p2q4 p2q5 p2q6

5 0 0 0 0 0 p2q4 p2q5 p2q6

6 0 0 0 0 0 0 p2q5 p2q6

7 0 0 0 0 0 0 0 p2q6

8 0 0 0 0 0 0 0 0

The marginal distribution of X

Xp x P X x ,y

p x y

2 1 2 2 1 2 2x x x xp q p q p q p q

2 2

1

y

y x

p q

2 1 2 31xp q q q q

2 1 11

1x xp q pq

q

This is the geometric distribution

The marginal distribution of Y

Yp y P Y y ,x

p x y

2 21 2,3,4,

0 otherwise

yy p q y

This is the negative binomial distribution with k = 2.

The conditional distribution of X given Y = y

X Yp x y P X x Y y

, ,

Y

P X x Y y p x y

P Y y p y

2 2

1

y

x

p q

pq

1 for 1, 2, 3y xpq y x x x

This is the geometric distribution with time starting at x.

The conditional distribution of Y given X = x

Y Xp y x P Y y X x

, ,

X

P X x Y y p x y

P X x p x

2 2

2 2

1

1 1

y

y

p q

y p q y

This is the uniform distribution on the values 1, 2, …(y – 1)

for 1,2,3, , 1x y

SummaryDiscrete Random Variables

The joint probability function;

p(x,y) = P[X = x, Y = y]

1. 0 , 1p x y

2. , 1x y

p x y

3. , ,P X Y A p x y ,x y A

Continuous Random Variables

Definition: Two random variable are said to have joint probability density function f(x,y) if

1. 0 ,f x y

2. , 1f x y dxdy

3. , ,P X Y A f x y dxdy A

If 0 ,f x y

,f x y dxdyA

then ,z f x y

defines a surface over the x – y plane

Multiple Integration

,f x y dxdyA

A

f(x,y)

If the region A = {(x,y)| a ≤ x ≤ b, c ≤ y ≤ d} is a rectangular region with sides parallel to the coordinate axes:

x

y

d

c

a b

,f x y dxdyA

f(x,y)

, ,d b b d

c a a c

f x y dx dy f x y dy dx

Then

,f x y dxdyA

f(x,y)

,d b

c a

f x y dxdy

To evaluate

,d b

c a

f x y dx dy

Then evaluate the outer integral

,b

a

G y f x y dxFirst evaluate the inner integral

,d b d

c a c

f x y dxdy G y dy

x

y

d

c

a b

y

f(x,y)

,b

a

G y f x y dx = area under surface above the line where y is constant

,d b d

c a c

f x y dxdy G y dy

dy

Infinitesimal volume under surface above the line where y is constant

,f x y dxdyA

f(x,y)

,b d

a c

f x y dydx

The same quantity can be calculated by integrating first with respect to y, than x.

,b d

a c

f x y dy dx

Then evaluate the outer integral

,d

c

H x f x y dyFirst evaluate the inner integral

,b d b

a c a

f x y dydx H x dx

x

y

d

c

a bx

f(x,y)

,d

c

H x f x y dy = area under surface above the line where x is constant

,b d b

a c a

f x y dydx H x dx

dx

Infinitesimal volume under surface above the line where x is constant

f(x,y)

1 1 1 1

2 3 2 3

0 0 0 0

x y xy dxdy x y xy dydx

Example: Compute

Now

1 1 1 1

2 3 2 3

0 0 0 0

x y xy dxdy x y xy dx dy

11 3 2

3

0 03 2

x

x

x xy y dy

11 2 4

3

0 0

1 1 1 1

3 2 3 2 2 4

y

y

y yy y dy

1 1 7

6 8 24

f(x,y)

The same quantity can be computed by reversing the order of integration

1 1 1 1

2 3 2 3

0 0 0 0

x y xy dydx x y xy dy dx

11 2 4

2

0 02 4

y

y

y yx x dx

11 3 22

0 0

1 1 1 1

2 4 2 3 4 2

x

x

x xx x dx

1 1 7

6 8 24

Integration over non rectangular regions

Suppose the region A is defined as follows

A = {(x,y)| a(y) ≤ x ≤ b(y), c ≤ y ≤ d}

x

y

d

c

a(y) b(y)

,f x y dxdyA

,b yd

c a y

f x y dx dy

Then

If the region A is defined as follows

A = {(x,y)| a ≤ x ≤ b, c(x) ≤ y ≤ d(x) }

x

y

ba

d(x)

c(x)

,f x y dxdyA

,d xb

a c x

f x y dy dx

Then

In general the region A can be partitioned into regions of either type

x

y

A1

A3

A4

A2

A

f(x,y)

Example: Compute the volume under f(x,y) = x2y + xy3 over the region A = {(x,y)| x + y ≤ 1, 0 ≤ x, 0 ≤ y}

x

y

x + y = 1

(1, 0)

(0, 1)

f(x,y)

Integrating first with respect to x than y

x

y

x + y = 1

(1, 0)

(0, 1)

(0, y) (1 - y, y)

11

2 3

0 0

y

x y xy dx dy

11

2 3 2 3

0 0

y

x y xy dxdy x y xy dxdy

A

and

11

2 3

0 0

y

x y xy dx dy

11 3 2

3

0 03 2

x y

x

x xy y dy

3 21

3

0

1 1

3 2

y yy y dy

1 2 3 4 3 4 5

0

3 3 2

3 2

y y y y y y ydy

31 1 1 2 12 4 5 4 5 61

3 2

1 1 1 1 1 1 16 3 4 15 8 5 12 20 40 30 8 15 24 10 3 1

120 120 40

Now integrating first with respect to y than x

x

y

x + y = 1

(1, 0)

(0, 1)

(x, 0)

(x, 1 – x )

1 1

2 3

0 0

x

x y xy dy dx

1 1

2 3 2 3

0 0

x

x y xy dydx x y xy dydx

A

Hence

1 1

2 3

0 0

x

x y xy dy dx

11 2 4

2

0 02 4

y x

y

y yx x dx

2 412

0

1 1

2 4

x xx x dx

1 2 3 4 2 3 4 5

0

2 4 6 4

2 4

x x x x x x x xdx

1 2 3 4 5

0

2 2 2

4

x x x x xdx

15 20 15 12 61 1 1 1 1 48 6 8 10 20 120 120

Continuous Random Variables

Definition: Two random variable are said to have joint probability density function f(x,y) if

1. 0 ,f x y

2. , 1f x y dxdy

3. , ,P X Y A f x y dxdy A

Definition: Let X and Y denote two random variables with joint probability density function f(x,y) then

the marginal density of X is

,Xf x f x y dy

the marginal density of Y is

,Yf y f x y dx

Definition: Let X and Y denote two random variables with joint probability density function f(x,y) and marginal densities fX(x), fY(y) then

the conditional density of Y given X = x

,

Y XX

f x yf y x

f x

conditional density of X given Y = y

,

X YY

f x yf x y

f y

The bivariate Normal distribution

Let

2 2

1 1 1 1 2 2 2 2

1 1 2 2

1 2 2

2

,1

x x x x

Q x x

1 21

,2

1 2 21 2

1, e

2 1

Q x xf x x

where

This distribution is called the bivariate Normal distribution.

The parameters are 1, 2 , 1, 2 and

Surface Plots of the bivariate Normal distribution

Note:

2 2

1 1 1 1 2 2 2 2

1 1 2 2

1 2 2

2

,1

x x x x

Q x x

1 21

,2

1 2 21 2

1, e

2 1

Q x xf x x

is constant when

is constant.

This is true when x1, x2 lie on an ellipse centered at 1, 2 .

x

y

f(x,y)

x

y

f(x,y)

x

y

f(x,y)

The Bivariate Normal Distribution

x

y y y

x x1

2

1 1

2 2

Contour Plots of the Bivariate Normal Distribution

x

y y y

x x1

2

1 1

2 2

Scatter Plots of data from the Bivariate Normal Distribution

1 2 1 2 1 2

1 2 1 2 1 2

1 21 2

1 2

Marginal and Conditional distributions

1 1 1 2 2,f x f x x dx

Marginal distributions for the Bivariate Normal distribution

Recall the definition of marginal distributions for continuous random variables:

and

It can be shown that in the case of the bivariate normal distribution the marginal distribution of xi is Normal with mean i and standard deviation i.

2 2 1 2 1,f x f x x dx

The marginal distributions of x2 is

2 2 1 2 1,f x f x x dx

1 2

1,

212

1 2

1e

2 1

Q x xdx

where

2 2

1 1 1 1 2 2 2 2

1 1 2 2

1 2 2

2

,1

x x x x

Q x x

Proof:

Now:

2 2

1 1 1 1 2 2 2 2

1 1 2 2

1 2 2

2

,1

x x x x

Q x x

2 2 2

1 112 2 2

2x a x a a

c x cb b b b

21 1 2 2

12 2 22 1

21 1 1

x xx

22

2 2 2 2112 2 2 2 2

1 2 1 2

21 1 1

x x

Hence 2 2 211 or 1 b b

Also

1 2 22 2 2

2 11 1

xa

b

11 22

2

1

1x

and 11 2

2

a x

Finally

222

2 2 2 2112 2 2 2 2 2

1 2 1 2

21 1 1

x xac

b

22 2

2 2 2 211 22 2 2 2 2

1 2 1 2

21 1 1

x x ac

b

22

2 2 2 2112 2 2 2 2

1 2 1 2

21 1 1

x x

2

11 2

2

21

x

and

2

22 1 11 1 2 2 2 222 2

2 21

12

1c x x

2

11 2 2

2

x

2

2212 222 2

21

11

1x

2

2 2

2

x

Summarizing

2 2

1 1 1 1 2 2 2 2

1 1 2 2

1 2 2

2

,1

x x x x

Q x x

2

1x ac

b

where 21 1 b

11 2

2

a x

2

2 2

2

xc

and

Thus 2 2 1 2 1,f x f x x dx

1 2

1,

212

1 2

1e

2 1

Q x xdx

211

2

121 2

1e

2 1

x ac

bdx

2112

212

1 2

2 1e

22 1

x acbbe

dxb

22 2

2

1

2

2

1

2

x

e

Thus the marginal distribution of x2 is Normal with mean 2 and standard deviation 2.

Similarly the marginal distribution of x1 is Normal with mean 1 and standard deviation 1.

1 2

1 2122 2

,f x xf x x

f x

Conditional distributions for the Bivariate Normal distribution

Recall the definition of conditional distributions for continuous random variables:

and

It can be shown that in the case of the bivariate normal distribution the conditional distribution of xi given xj is Normal with:

1 2

2 1211 1

,f x xf x x

f x

andmean

standard deviation

ii j ji j

j

x

21ii j

Proof

1 2

2 1211 1

,f x xf x x

f x

1 2

22 2

2

1,

2

21 2

1

2

2

e

2 1

1

2

Q x x

x

e

222

1 2 22 21 2

22

1 11 1,

2 22 2

2 21 1

e e

2 1 2 1

x a xx cQ x xb

where 21 1 b

11 2

2

a x

2

2 2

2

xc

and

2

11

21 212

1

2

x a

bf x x eb

Hence

Thus the conditional distribution of x2 given x1 is Normal with:

andmean

standard deviation

11 2 212

2

a x

2112 1b

Bivariate Normal Distribution with marginal distributions

Bivariate Normal Distribution with conditional distribution

(1, 2)

x2

x1

Regression

Regression to the mean

22 1 221

1

x

Major axis of ellipses

Suppose that a rectangle is constructed by first choosing its length, X and then choosing its width Y.

Its length X is selected form an exponential distribution with mean = 1/ = 5. Once the length has been chosen its width, Y, is selected from a uniform distribution form 0 to half its length.

Find the probability that its area A = XY is less than 4.

Example:

Solution:

, X Y Xf x y f x f y x

151

5 for 0xXf x e x

1 if 0 2

2Y Xf y x y xx

1 15 51 2

5 5

1= if 0 2, 0

2x x

xe e y x xx

xy = 4y = x/2

2 4y x x 2 8 or 8 2 2x x

2 2 2 2 2y x

2 2, 2

2 2, 2

4

2 2 2

0 0 02 2

4 , ,

xx

P XY f x y dydx f x y dydx

4

2 2 2

0 0 02 2

4 , ,

xx

P XY f x y dydx f x y dydx

1 15 5

42 2 2

2 25 5

0 0 02 2

xx

x x

x xe dydx e dydx

1 15 5

2 22 2 4

5 2 50 2 2

x xxx x xe dx e dx

1 15 5

2 2281

5 50 2 2

x xe dx x e dx

This part can be

evaluated

This part may require Numerical evaluation

multivariate distributions

k ≥ 2

Definition

Let X1, X2, …, Xk denote k discrete random variables, then

p(x1, x2, …, xk )

is joint probability function of X1, X2, …, Xk if

1

12. , , 1n

nx x

p x x

11. 0 , , 1np x x

1 13. , , , ,n nP X X A p x x 1, , nx x A

Definition

Let X1, X2, …, Xk denote k continuous random variables, then

f(x1, x2, …, xk )

is joint density function of X1, X2, …, Xk if

1 12. , , , , 1n nf x x dx dx

11. , , 0nf x x

1 1 13. , , , , , ,n n nP X X A f x x dx dx

A

Example: The Multinomial distribution

Suppose that we observe an experiment that has k possible outcomes {O1, O2, …, Ok } independently n times.

Let p1, p2, …, pk denote probabilities of O1, O2, …, Ok respectively.

Let Xi denote the number of times that outcome Oi occurs in the n repetitions of the experiment.

Then the joint probability function of the random variables X1, X2, …, Xk is

1 21 1 2

1 2

! , ,

! ! !kxx x

n kk

np x x p p p

x x x

Note:

is the probability of a sequence of length n containing

x1 outcomes O1

x2 outcomes O2

…

xk outcomes Ok

1 21 2

kxx xkp p p

1 21 2

!

! ! ! kk

nnx x xx x x

is the number of ways of choosing the positions for the x1 outcomes O1, x2 outcomes O2, …, xk outcomes Ok

1 21

31 2

k

k

n x x xn n x

x xx x

1 1 2

1 1 2 1 2 3 1 2 3

! !!

! ! ! ! ! !

n x n x xn

x n x x n x x x n x x x

1 2

!

! ! !k

n

x x x

is called the Multinomial distribution

1 21 1 2

1 2

! , ,

! ! !kxx x

n kk

np x x p p p

x x x

1 21 2

1 2

kxx xk

k

np p p

x x x

Example:Suppose that a treatment for back pain has three possible outcomes:

O1 - Complete cure (no pain) – (30% chance)

O2 - Reduced pain – (50% chance)

O3 - No change – (20% chance)

Hence p1 = 0.30, p2 = 0.50, p3 = 0.20. Suppose the treatment is applied to n = 4 patients suffering back pain and let X = the number that result in a complete cure, Y = the number that result in just reduced pain, and Z = the number that result in no change.Find the distribution of X, Y and Z. Compute P[X + Y ≥ Z]

4! , , 0.30 0.50 0.20 4

! ! !

x y zp x y z x y z

x y z

Table: p(x,y,z)z

x y 0 1 2 3 4

0 0 0 0 0 0 0.00160 1 0 0 0 0.0160 00 2 0 0 0.0600 0 00 3 0 0.1000 0 0 00 4 0.0625 0 0 0 01 0 0 0 0 0.0096 01 1 0 0 0.0720 0 01 2 0 0.1800 0 0 01 3 0.1500 0 0 0 01 4 0 0 0 0 02 0 0 0 0.0216 0 02 1 0 0.1080 0 0 02 2 0.1350 0 0 0 02 3 0 0 0 0 02 4 0 0 0 0 03 0 0 0.0216 0 0 03 1 0.0540 0 0 0 03 2 0 0 0 0 03 3 0 0 0 0 03 4 0 0 0 0 04 0 0.0081 0 0 0 04 1 0 0 0 0 04 2 0 0 0 0 04 3 0 0 0 0 04 4 0 0 0 0 0

P [X + Y ≥ Z]z

x y 0 1 2 3 4

0 0 0 0 0 0 0.00160 1 0 0 0 0.0160 00 2 0 0 0.0600 0 00 3 0 0.1000 0 0 00 4 0.0625 0 0 0 01 0 0 0 0 0.0096 01 1 0 0 0.0720 0 01 2 0 0.1800 0 0 01 3 0.1500 0 0 0 01 4 0 0 0 0 02 0 0 0 0.0216 0 02 1 0 0.1080 0 0 02 2 0.1350 0 0 0 02 3 0 0 0 0 02 4 0 0 0 0 03 0 0 0.0216 0 0 03 1 0.0540 0 0 0 03 2 0 0 0 0 03 3 0 0 0 0 03 4 0 0 0 0 04 0 0.0081 0 0 0 04 1 0 0 0 0 04 2 0 0 0 0 04 3 0 0 0 0 04 4 0 0 0 0 0

= 0.9728

Example: The Multivariate Normal distribution

Recall the univariate normal distribution

2121

2

x

f x e

the bivariate normal distribution

221

22 12

2

1 ,

2 1

x xx x y yx xx x y y

x y

f x y e

The k-variate Normal distribution

112

1 / 2 1/ 2

1 , ,

2k kf x x f e

x μ x μx

where

1

2

k

x

x

x

x

1

2

k

μ

11 12 1

12 22 2

1 2

k

k

k k kk

Marginal distributions

Definition

Let X1, X2, …, Xq, Xq+1 …, Xk denote k discrete random variables with joint probability function

p(x1, x2, …, xq, xq+1 …, xk )

1

12 1 1 , , , ,q n

q q nx x

p x x p x x

then the marginal joint probability function

of X1, X2, …, Xq is

Definition

Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function

f(x1, x2, …, xq, xq+1 …, xk )

12 1 1 1 , , , ,q q n q nf x x f x x dx dx

then the marginal joint probability function

of X1, X2, …, Xq is

Conditional distributions

Definition

Let X1, X2, …, Xq, Xq+1 …, Xk denote k discrete random variables with joint probability function

p(x1, x2, …, xq, xq+1 …, xk )

1

1 11 11 1

, , , , , ,

, ,k

q q kq q kq k q k

p x xp x x x x

p x x

then the conditional joint probability function

of X1, X2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is

Definition

Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function

f(x1, x2, …, xq, xq+1 …, xk )

then the conditional joint probability function

of X1, X2, …, Xq given Xq+1 = xq+1 , …, Xk = xk is

Definition

11 11 1

1 1

, , , , , ,

, ,k

q q kq q kq k q k

f x xf x x x x

f x x

Definition

Let X1, X2, …, Xq, Xq+1 …, Xk denote k continuous random variables with joint probability density function

f(x1, x2, …, xq, xq+1 …, xk )

then the variables X1, X2, …, Xq are independent of Xq+1, …, Xk if

Definition – Independence of sets of vectors

1 1 1 1 1 , , , , , ,k q q q k q kf x x f x x f x x

A similar definition for discrete random variables.

Definition

Let X1, X2, …, Xk denote k continuous random variables with joint probability density function

f(x1, x2, …, xk )

then the variables X1, X2, …, Xk are called mutually independent if

Definition – Mutual Independence

1 1 1 2 2 , , k k kf x x f x f x f x

A similar definition for discrete random variables.

Example

Let X, Y, Z denote 3 jointly distributed random variable with joint density function then

2 0 1,0 1,0 1, ,

0 otherwise

K x yz x y zf x y z

Find the value of K.

Determine the marginal distributions of X, Y and Z.

Determine the joint marginal distributions of

X, Y

X, Z

Y, Z

Solution

1 1 1

2

0 0 0

1 , ,f x y z dxdydz K x yz dxdydz

Determining the value of K.

11 1 1 13

0 0 0 00

1

3 3

x

x

xK xyz dydz K yz dydz

11 12

0 00

1 1 1

3 2 3 2

y

y

yK y z dz K z dz

12

0

1 1 71

3 4 3 4 12

z zK K K

12if

7K

1 1

21

0 0

12, ,

7f x f x y z dydz x yz dydz

The marginal distribution of X.

11 122 2

0 00

12 12 1

7 2 7 2

y

y

yx y z dz x z dz

12

2 2

0

12 12 1 for 0 1

7 4 7 4

zx z x x

1

212

0

12, , ,

7f x y f x y z dz x yz dz

The marginal distribution of X,Y.

122

0

12

7 2

z

z

zx z y

212 1 for 0 1,0 1

7 2x y x y

Find the conditional distribution of:

1. Z given X = x, Y = y,

2. Y given X = x, Z = z,

3. X given Y = y, Z = z,

4. Y , Z given X = x,

5. X , Z given Y = y

6. X , Y given Z = z

7. Y given X = x,

8. X given Y = y

9. X given Z = z

10. Z given X = x,

11. Z given Y = y

12. Y given Z = z

The marginal distribution of X,Y.

212

12 1, for 0 1,0 1

7 2f x y x y x y

Thus the conditional distribution of Z given X = x,Y = y is

2

212

12, , 7

12 1,7 2

x yzf x y z

f x yx y

2

2

for 0 112

x yzz

x y

The marginal distribution of X.

21

12 1 for 0 1

7 4f x x x

Thus the conditional distribution of Y , Z given X = x is

2

21

12, , 7

12 17 4

x yzf x y z

f xx

2

2

for 0 1,0 114

x yzy z

x