EC 402 (20/21) - MTWK 3 - VASSILISPS2

RACNIR'S SPEAKING NOTES (UNOFFICIAL CONTENT)

141 As mentioned in"Items

"

of my"status check

"

slides,

we need to know how to obtain Mitt) where XNN(y ,E) .

Without this,we can't even attempt this question .

So here we go . . .

Step 1 : let's recall the univariate case . . .

Say Z ~Nloi) and X÷µtoZThen Mzttt . -- EFTi f. jet? afge.EE de

⇐HI E- ate + EGI. I ate

Eat⇒dz

i.¥-I - t' = E- atz = f.of agg e-talk - tKtY dz

-""

÷?÷÷÷.of a Nlt , 1) PDF .

i. e. M It) = Efetxf = E fetµtoZ)]X=EM Efet}

=

ettmz.to/--etMe4zEE--etm+IEotBrilliant ! Let's extend to the multivariate case

(w/o proof because that adds no extra intuition)

ELEMENTARY MULTIVARIATE RESULTS USED IN Ps2 :

A) Remember last week,we did Eff and Varf./ rules?Efa = a EH1

,

and% '

varfaxf = a. Varfxl , for a EIR and X a scalar Rv.

(B) Well,vectors work the Same way :

eg : let X be an nxl vector- valuedRV

.Let at Rn

.

Then,Efa'XI = a' Efxl

,and

varfoixfava.lyaNote l : I := Varfxf = EL4 - EK1N-EASY /

,an nxn matrix s.tnAn{2170 and 5=5!

Notes : xnNlµE)⇐ fµh=knT"detLEFexpfµµfEi←µffor any teeth .

If additionally E.=EIn and say µi=µj=m for i.ft . . . ,n then also

Xi NE NIt ,8) for i=l , . . . ,n .

Step 2 : Say ×n×TNµn× , 'Em) .Analogous to the univariate case , we have that for

Some t E IR",

Mx 4) = exp ftp.t tat'Etf .

Stop 3 : Define Y : = DX for some at Rn

(usual stuff from last week - i.e. we have Xand Etc) . We also have a transformation Y . Find Ely) .)

let's see how . . .

My ftp.ifiEfettf.tn . Eft'

a'Xf BEHE felatixfDAE M×④sE%xpf tape + that atf

Job done .3 proofs in one shot ! Why? Because . . .

Bythe uniqueness/propertyof MGFS ,we recognise

that

a) .Y N N (a'µ , a

( let's also add'

r'eat"

to keep things easier)

(5) PROPOSITION,

5.a. FordRD . Symmetric matrix A there exists a

a square N .9. matrix P

'

s.t.

A- pp !

PROOF IF 5.a .

Ii ) lemma-

5. a. 1.

Since A is a real symmetric matrix , thereexists a representation

A- = VAV'

where I :=diag{ 414 , . . . ,dnLHf isa diagonal matrix of ordered eigenvaluesof A ,and

where V := fvi.. . . .vn/ is an orthogonal

matrix of corresponding eigenvectors .

this is called ←Note : ri 'S = {9 ;forging j"orthonormality

"

of vectors for i, f- f. . . ,n .

Proof of 5. a. 1 The stated result is exactly-

.

the spectral theorem for Hermitian matrices .

(For those who have been Pat before , this shouldbe extremely familiar ! )

Iii ) Lemma,

5.a.2. di LAI 30 . for it , . . . ,n .

Proof of I. a.2 .

The stated result is just thedefinition of positive definiteness .

µ I've defined this in Q2 already.

liii ) lemma I. a.3 diff for i= f., n are all real .

Proof of I. a.3 . By definition , Ari = dittlri ,where ri denotes the

ith eigenvector of A fos i -- f . . . ,h .• Then

, vi.* A'

= LATE , fori = I

,. . . ,n , where

ni denotes the

complex coivgate of some matrixM and MEAT

.

. Since AIA,ri*A=Mri* for

i =L,. . . in .

. If we post - multiply both tides by ri ,

we have vi*Avi=IiHri*ri,

or

di vi. = IfAJri*ri ,

for it , . . . ,n .

- This ensures that iiLA3= ELA}for i =L , . . in , which completes theproof of Lemma 5. a .-3

④I Finally , given lemmas 5. a. 1 , 5. a.2 , and 5. a.3 , there clearlyexists a diagonal matrix

11" diagLXA.FI . . ..int#ss.t.A--VdV'=vNkNIiv!

Defining E- VNK , the result follows and thiscompletes (a sketch of ) the

'

proof for Proposition 5. a .

Proposition 5. b. For nxl random vector

X N N (0,%) sit . rank = n

,

and 4,'0

,we have that Z :-. x

'E' x NX2N .

PRIOF . I has to be a I.b. Symmetric matrix because .fyontellme]. So by lemma 5.a. 1. ,there exist matrices Paid D sit

.

I =PDP'where D= diagfd.IS , . . . . in5231 and

P is the orthogonal matrix of correspondingeigenvectors of A .

- Clearly , is E-'

exists, by Proposition 2.a. ;

E-'

is symmetric since

I'=4Dp 'II F'ftp.i-pfp'and f-(potty! post'= ;

'

Iii) £ is RD .

since di {Lif t

for it . . . ,n .""-i

. It then follows from Proposition 5. a. that there exist matrices£4 :=Ps"r it

. E-'=IKEY? see slide titled

- Consider the vector 4 : = E-"E×

.

"ELEMENTARY. . .

"

Clearly , E41 = E-"i EH40

,and }Vor = E-"

I Varfxf E-"' = In .

- Combining the above with the distributional result in ①4 ,4- NL0

,In) .

"t also to";*%%¥i¥%d elementoft ,}

as mentioned in my"

status check"

- This means that 44 = II Yi NX2N Tides litem 4) ,you may need to reviewbasic samplingdistributions IN

,t,IF)

< The proof is completedby noting thatto understand this .

44 = ×' -242 EYE ×

= X' E-

'

×

which means that ×' E' X N XI .

4) Not much for me to tag here .

See official towtion .

(7) Proposition 7

mama

PR10F

Lemmy 7. I M has eigenvalues either 0 or I .Proof of71 . By definition , Mri = hi {MI ri for iii. in .

. Pre -multiplying both sides by M ,MMri = Mii LHri or Mri-- difmfriso that dilmtri = dilmfri for iii.in .

- the result follows .

- fiver henna 7.1 and Lemma 5. a . I,

there exist matricesD

,a diagonal matrix of 0's and I's , ie eigenvalues of M ; and

C,an orthogonal matrix of corresponding eigenvectors St .

M = Cbc?

- Since rankleµ = J,it must be the case that M has J I's

and Ln ) 0 's as its eigenvalues .

. Then,X' Mx = x'Cbc4 = 44)'b¢x) .

- Defining Y : = dx,we have that Y u N 10 , In)

and link NL0 , 1) for it . . . . ,n ,where Yi is the ith element of Y for i. 1 . . . . . n .

Thus,X' Mx = 4

'by = IfI? ditMI -XI ,

which

completes the proof of Proposition7 .

(f)[ SEE OFFICIAL Solution FOR Q8]

The main point is that AZ is basicall justthe residuals from a regression of Z on a constant(for some Nxt vector Z) .

To see this, recognise that

4) My := In - XK'

for some Nxk full rank matrix X4) Let X : = i , where i is an NH vector of ones .

Then,

Mi = In - ili'if i'

= In - YIN)= A

,as defined in the question .

CAN You SEE NOW WHYit's No BIG SURPRISE If At any AA -

.A ?!

Next,

At IN - i I'i¥2- - kN) ii't

= E - 4N) i E.For--←

I:[

No BIG SURPRISE SINCE %, FOR MODEL ¥ p + Eq for off . . . ,NIS JUST Z

,RIGHT?