STAT 430/510: Lecture 17jpiette/stat430-su10/lecture17.pdfAdmin. Stuff Covariance Correlation Conditional Expectation To Do STAT 430/510: Lecture 17 James Piette June 28, 2010

Admin. Stuff Covariance Correlation Conditional Expectation To Do

STAT 430/510: Lecture 17

James Piette

June 28, 2010


Updates

HW4 is due today.HW2 grades are (finally) up, as are the HW3 solutions. I’llget around to grading that soon.Discuss HW5.


Formalization

Def: The covariance between two r.v.’s X and Y is definedas

Cov(X ,Y ) = E [(X − E(X ))(Y − E(Y ))]

An alternative form of covariance is

Cov(X ,Y ) = E [XY ]− E [X ]E [Y ]

Remembering back, Cov comes up when we look at X andY not independent and . . .

Var(X + Y ) = Var(X ) + Var(Y ) + 2Cov(X ,Y )


Properties

Cov(X ,Y ) = Cov(Y ,X ).Cov(X ,X ) = Var(X ).Cov(aX ,bY ) = ab · Cov(X ,Y ).Cov(

∑i Xi ,

∑j Yj) =

∑i∑

j Cov(Xi ,Yj).

Var(∑n

i=1 Xi) =∑n

i=1 Var(Xi) + 2∑

i<j Cov(Xi ,Xj).If Xi are pairwise independent, then

Var(n∑

i=1

Xi) =n∑

i=1

Var(Xi)


Example 1

Let

X =

1 w.p. 1/30 w.p. 1/3−1 w.p. 1/3

and

Y =

{1 if X = 00 if X 6= 0

Question: What is XY and E [XY ]?Solution: XY = 0, because Y is 0 if X is not and Y is not0 if X is.Thus, the E [XY ] = 0.Question: Are X and Y independent?Solution: No, even though Cov(X ,Y ) = 0.


Example 2

Think back to example 2c, in this chapter.Let X1, . . . ,Xn be i.i.d r.v.’s having expected values µ andvariance σ2.Let X̄ =

∑ni=1

Xin be the sample mean.

Let the quantities Xi − X̄ be called deviations.These are all the differences between the individual dataand the sample mean.

The r.v. S2 =∑n

i=1(Xi−X̄)2

n−1 is called the sample variance.

Question: What is Var(X̄ )?


Example 2 (cont.)

Solution:

Var(X̄ ) = Var

(n∑

i=1

Xi

n

)

=

(1n

)2

Var

(n∑

i=1

Xi

)

=

(1n

)2 n∑i=1

Var(Xi) by independence

=σ2

n

Question: E [S2]?


Example 2 (cont.)

Solution: Start off by multiplying by (n − 1) to eliminatethe constant. Then . . .

(n − 1)S2 =n∑

i=1

(Xi − X̄ )2

=n∑

i=1

(Xi − µ+ µ− X̄ )2 (adding 0)

=n∑

i=1

(Xi − µ)2 +n∑

i=1

(X̄ − µ)2 − 2(X̄ − µ)n∑

i=1

(Xi − µ)

=n∑

i=1

(Xi − µ)2 + n(X̄ − µ)2 − 2(X̄ − µ)n(X̄ − µ)

=n∑

i=1

(Xi − µ)2 − n(X̄ − µ)2


Example 2 (cont.)

Now, take the expectation of that:

(n − 1)E [S2] =n∑

i=1

E [(Xi − µ)2]− nE [(X̄ − µ)2]

= nσ2 − nVar(X̄ )

= (n − 1)σ2

Thus, E [S2] = σ2, which is what we would want whenestimating the variance of some data.


Example 3

Let X = X1 + . . .+ Xn, where X1, . . . ,Xn are i.i.d Bernoullitrials with prob. of success p. Then, X is . . .Binomial with parameters (n,p). We’ve talked about thevariance of a Binomial r.v. before; this is how we can proveit:

Var(X ) = Var(n∑

i=1

Xi)

=n∑

i=1

Var(Xi)


Example 3 (cont.)

Note that:

Var(Xi) = E [X 2i ]− (E [Xi ])

2

= E [Xi ]− (E [Xi ])2

= p − p2

Then,Var(X ) = np(1− p)


Formalization

Def: The correlation of two r.v.’s X and Y , denoted byρ(X ,Y ), is defined as

ρ(X ,Y ) =Cov(X ,Y )√

Var(X )Var(Y )

Note that−1 ≤ ρ(X ,Y ) ≤ 1

If ρ(X ,Y ) = 0, then X and Y are said to be uncorrelated.X and Y are uncorrelated if and only if

E [XY ] = E [X ]E [Y ]

Correlation indicates the strength of a linear relationshipbetween two variables.


Properties

Covariance depends on the unit of measurement, thus,making it difficult to interpret a computed value.Correlation is scale independent:

ρ is not affected by a linear change in the units ofmeasurement (e.g. pound← kilo).If b and d are both positive or both negative, thenρ(a + bX , c + dY ) = ρ(X ,Y ).

When |ρ(X ,Y )| = 1, then Y = a + bX for some a,b.If X and Y are independent, then ρ = 0.However, ρ = 0 does not imply independence between Xand Y , as was seen in the previous example.


Example 4

Let

IA =

{1 if A occurs0 otherwise

, IB =

{1 if B occurs0 otherwise

Question: What is Cov(IA, IB)?


Example 4 (cont.)

Solution: We know that:

E [IA] = P(A)

E [IB] = P(B)

E [IAIB] = P(AB)

Then,

Cov(IA, IB) = P(AB)− P(A)P(B)

= P(B)[P(A|B)− P(A)]

The indicator variables for A and B are either positivelycorrelated, uncorrelated, or negatively correlated,depending on how B affects the prob. of A occurring.


Example 5

Let X be the number of 1’s and Y the number of 2’s thatoccur in n rolls of a fair die.Question: What is Corr(X ,Y )?Solution: X and Y and be talked about as the sum of . . .

Xi =

{1 roll i lands on 10 otherwise

, Yi =

{1 roll i lands on 20 otherwise


Example 5 (cont.)

Then, the covariance between Xi and Yi is . . .

Cov(Xi ,Yi) = E [XiYj ]− E [Xi ]E [Yj ]

=

{− 1

36 i = j1

36 −136 = 0 i 6= j

So, the covariance must be

Cov(∑

i

Xi ,∑

j

Yj) =∑

i

∑j

Cov(Xi ,Yj)

= − n36


Example 5 (cont.)

To finish this off, we need the variance of X and Y :

Var(X ) = Var(∑

i

Xi)

=∑

i

Var(Xi)

= n16

56

= n5

36= Var(Y )

So, the correlation must be:

Corr(X ,Y ) =Cov(X ,Y )√

Var(X )Var(Y )

=−n/36

(5n)/36= −1

5


Formalization

Def: If X and Y are jointly discrete r.v.’s, then theconditional expectation of X given Y = y , for all valuesof y s.t. pY (y) > 0, is defined as

E [X |Y = y ] =∑

x

x · pX |Y (x |y)

Def: If X and Y are jointly continuous r.v.’s, then theconditional expectation of X given Y = y , provided thatfY (y) > 0, is defined as

E [X |Y = y ] =

∫ ∞−∞

x · fX |Y (x |y)dx


Properties

Proposition 7.5.1:

E [X ] = E [E [X |Y ]]

That is, for a discrete r.v.,

E [X ] =∑

y

E [X |Y = y ]P(Y = y)

And, for a continuous r.v.,

E [X ] =

∫ ∞−∞

E [X |Y = y ]fY (y)dy

Law of Total Variance:

Var(X ) = Var(E(X |Y )) + E(Var(X |Y ))


Example 6

A miner is trapped in a mine containing 3 doors. The firstdoor leads to a tunnel that will take him to safety after 3hours of travel. The second door leads to a tunnel that willreturn him to the mine after 5 hours. The third door leadsto a tunnel that will return him to the mine after 7 hours.Question: If we assume that the miner is at all timesequally likely to choose any one of the doors, what is theexpected length of time until he reaches safety?


Example 6 (cont.)

Solution: Let X denote the amount of time (in hours) untilthe miner reaches safety and let Y denote the door heinitially chooses. So, E [X ] is . . .

E [X ] = E [X |Y = 1]P(Y = 1) + E [X |Y = 2]P(Y = 2)

+E [X |Y = 3]P(Y = 3)

=13

(E [X |Y = 1] + E [X |Y = 2] + E [X |Y = 3])

What are each of those conditional expectations?

E [X |Y = 1] = 3E [X |Y = 2] = 5 + E [X ]

E [X |Y = 3] = 7 + E [X ]


Example 6 (cont.)

Thus, plugging this back in, we get . . .

E [X ] =13

(3 + 5 + E [X ] + 7 + E [X ])

⇒ 13

E [X ] = 5

or E [X ] = 15


Now, covered section 7.4 and are on 7.5.Putting up HW5 after class.

Documents

STAT 430/510: Lecture 17jpiette/stat430-su10/lecture17.pdfAdmin. Stuff Covariance Correlation Conditional Expectation To Do STAT 430/510: Lecture 17 James Piette June 28, 2010