ENEE 324:Conditional Expectation

Richard J. La

Fall 2004

Conditional Expectation Example: Toss a coin 3 times

X = number of heads in 3 independent tosses

Y = length of the longest run of heads

Compute

3

1

2

3

3/8

1/8

1/8

1/8

1/4

Conditional Expectation

Example #2: (Tom and Jenny): Compute

Ans:

In general is a deterministic number which can be computed from the given value of

Similarly,

Conditional Expectation Example #2: (Tom and Jenny)

can be thought of as a function of the rv , i.e., given the value we can compute the value of the function

Similarly,

Conditional expectation A function of rv => a derived rv !!!! = value of the function evaluated at

Since is a rv, we can calculate its PMF, expected value, etc.

Conditional Expectation

Conditional expectation E[X|Y] A function of rv Y (i.e, f(Y)) f(y) = E[X|Y=y] PMF of rv E[X|Y] :

Expected value of rv E[X|Y]

Conditional Expectation In general,

Example: Toss a coin 3 times

X = number of heads in 3 independent tosses

Y = length of the longest run of heads

3

1

2

3

3/8

1/8

1/8

1/8

1/4

Conditional Expectation

3

1

2

3

3/8

1/8

1/8

1/8

1/4

Independent RVs Recall that two events A and B are independent if

Definition: Two discrete rvs X and Y are independent if and only if for all i.e., events {X=x} and {Y=y} are independent for all Since

Independent RVs Example: Roll two six-sided dice

X = number of dots on die #1

Y = number of dots on die #2

x

y

X and Y are independent

Independent RVs Example #2: Toss a coin 3 times

X = number of heads in 3 independent tosses

Y = maximum number of consecutive heads

3

1

2

3

3/8

1/8

1/8

1/8

1/4

X and Y not independent

Useful Fact In general,

X, Y independent => X, Y uncorrelated ( Cov(X,Y) = 0 )

However, the converse is not true in general !

Useful Fact Example: Uncorrelated but not independent rvs

1

20.2

0.2

0.2

0.2

0.2

21

X and Y NOT INDEPENDENT !!

Multiple Discrete RVs Suppose be N rvs defined on the

same underlying experiment Definition: Joint PMF

Multiple Discrete RVs Example: Suppose that the instructor plays a tennis

match with Anna Kournikova. Let be the number of games that the instructor wins in set 1, 2, 3, respectively.

Definition: Marginal PMF

Two RV case:

Multiple Discrete RVs Definition: Discrete rvs are

independent if and only if

for all

Example: Roll N dice, and let be the number on the i-th die.

where

Multiple Discrete RVs Functions of multiple rvs: Let

PMF:

Expected value:

Summary: Multiple Discrete RVs

1. Joint PMF of X and Y :

2. Margin PMF :

3. Function of RVs X and Y : PMF -

Expected Value –

4. Conditional probability

Summary: Multiple Discrete RVs

5. Independent RVs

Problems Problem #1: The PMF for rvs H and B is given in

the following table. Find the marginal PMFs and

h=-1

h= 0h= 1

b=0 b=2 b=4

0 0.4 0.20.1 0 0.10.1 0.1 0

h

b

0.1

0.1

0.1

0.10.4

0.2

0.2 0.5 0.3

0.60.20.2

0.2

0.4 0.1

0.1 0.1

0.1

0.1

0.1

0.10.1

0.4

0.2

Problems:

Problem #2: A bin of 5 transistors is known to contain 2 that are defective. The transistors are to be tested, one at a time, until the defective ones are identified. Denote by N1 the number of tests made till the first defective is identified and by N2 the number of additional tests until the second defective is identified. Find the joint PMF of N1 and N2.

Problems:

Rvs X and Y have the joint PMF as shown. Let B = {X + Y <= 3}. Find the conditional PMF of X and Y given B.

3

1

2

3

1/8

1/4

1/16

1/12

1/8

1/12

1/12

1/16

1/16

1/16

1 2 3

1

2

3

1/8

1/4 1/8

1/12

1/12 1/16

1 2

Normalize byP(B) = 35/48

Problems: The marginal PMF of rv A is

The conditional PMF of rv B given A is given by

(a) Find the joint PMF of rvs A and B.

(b) If B = 0, what is the conditional PMF ?

(c) If A = 2, what the conditional expected value

?

Problems: Rvs X and Y have joint PMF given by the following

matrix

Are X and Y independent? Are they uncorrelated?

y = -1 y = 0 y = 10 0.25 00.25 0.25 0.25

x = -1x = 1