Exercises Chapter 1 Estimation Theory - univ-orleans.fr · Exercises Chapter 1 Estimation Theory Advanced Econometrics - HEC Lausanne Christophe Hurlin University of OrlØans November

Exercises Chapter 1

Estimation Theory

Advanced Econometrics - HEC Lausanne

Christophe Hurlin

University of Orléans

November 2013

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne November 2013 1 / 68

Exercise 1

Rayleigh distribution


ProblemWe consider two continuous independent random variables X and Ynormally distributed with N

0, σ2

. The transformed variable R dened

by:R =

pX 2 + Y 2

has a Rayleigh distribution with a parameter σ2 :

R Rayleighσ2

with a pdf fRr ; σ2

dened by:

fRr ; σ2

=r

σ2exp

r2

2σ2

8r 2 [0,+∞[

E (R) = σ

rπ

2V (R) =

4 π

2

σ2


Problem (contd)

We consider an i .i .d . sample fR1,R2, ..,RNg and an estimator (MLE) ofσ2 dened by

bσ2 = 12N

N

∑i=1R2i

Question 1: Show that bσ2 is an unbiased estimator of σ2.


Solution:

We have: bσ2 = 12N

N

∑i=1R2i

We want to calculate Ebσ2 .

Since the sample fR1,R2, ..,RNg is i .i .d ., we have:

Ebσ2 = E

12N

N

∑i=1R2i

!=

12N

N

∑i=1

ER2i


Solution (contd):

Ebσ2 = E

12N

N

∑i=1R2i

!=

12N

N

∑i=1

ER2i

We know that:

E (Ri ) = σ

rπ

2V (Ri ) =

4 π

2

σ2

So, we have:

ER2i= V (Ri ) +E (Ri )

2

=

4 π

2

σ2 +

π

2σ2

= 2σ2


Solution (contd):

Ebσ2 =

12N

N

∑i=1

ER2i

=12N

N

∑i=12σ2

=N2σ2

2N

So, we have:Ebσ2 = σ2

The estimator bσ2 is unbiased.Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne November 2013 7 / 68

Remark: Sometimes, the Rayleigh distribution is parametrized by σ. But,it is easier to show that bσ2 is unbiased than to show that bσ is unbiased...

bσ =vuut 12N

N

∑i=1R2i

Then to study the bias, we have to calculate:

E (bσ) = E

0@vuut 12N

N

∑i=1R2i

1A ???

since for a nonlinear function g (.) , E (g (x)) 6= g (E (x)) ... The onlysolution is to compute the integral

E (bσ) = R ∞0 x fbσ x ; σ2 dx


Problem (contd)

Question 2: Show that bσ2 is a (weakly) consistent estimator of σ2. Weadmit that the raw moments of R are dened by:

ERki= σk 2

k2 Γ1+

k2

k 2 N

and where Γ (.) denotes the gamma function with:

Γ (x) =∞Z0

tx1 exp (t) dt.

andΓ (x) = (x 1)! for x 2 N


Solution:

First, calculate Vbσ2 . Since the sample fR1,R2, ..,RNg is i .i .d ., we

have:

Vbσ2 = V

12N

N

∑i=1R2i

!=

14N2

N

∑i=1

VR2i

What is the value of VR2i?

VR2i= E

R4iER2i2

We shown thatER2i= 2σ2


Solution (contd):

VR2i= E

R4iER2i2

= ER4i 4σ4

What is the value of ER4i? For any k 2 N :

ERki= σk 2

k2 Γ1+

k2

For k = 4, we have:

ER4i= σ4 22 Γ (3)

withΓ (3) = (3 1)! = 2! = 2

So, we have:ER4i= σ4 23 = 8σ4


Solution (contd):

The variance of R2i is equal to:

VR2i= E

R4i 4σ4 = 8σ4 4σ4 = 4σ4

As a consequence

Vbσ2 =

14N2

N

∑i=1

VR2i

=14N2

N

∑i=14σ4

=N4σ4

4N2

=σ4

N


Solution (contd):

To sum up:Ebσ2 = σ2 (unbiased estimator)

limN!∞

Vbσ2 = lim

N!∞

σ4

N= 0

So, the estimator bσ2 is a (weakly) consistent estimator of σ2:

bσ2 p! σ2


Problem (contd)Question 3: Sometimes, the Rayleigh distribution is parametrized by σ(and not by σ2) with

R Rayleigh (σ)Propose a (weakly) consistent estimator for σ.


Solution:

Since σ > 0, a natural estimator for σ is dened by:

bσ =vuut 12N

N

∑i=1R2i =

pbσ2We shown that the estimator bσ2 is a (weakly) consistent estimator of σ2 :

bσ2 p! σ2

By applying the Continuous Mapping Theorem (CMP) for the continuousfunction g (x) =

px , we get immediately:

gbσ2 p! g

σ2

or bσ p! σ


Problem (contd)

Question 4: For any value of σ2, what is the nite sample (or exactsampling) distribution of the estimator bσ2 dened by

bσ2 = 12N

N

∑i=1R2i


Solution:

The estimator is dened by

bσ2 = 12N

N

∑i=1R2i

We know that R1,R2, ..,RN are i .i .d . random variables with a Rayleighdistribution.

Ri Rayleighσ2

Reminder: if X and Y are independent and normally distributed N0, σ2

random variables, the transformed variable R =

pX 2 + Y 2 has a Rayleigh

distribution.

The exact distribution of R2 = X 2 + Y 2 is unknown (it is not a χ2...),and as a consequence the nite sample distribution of bσ2 is unknown.


Problem (contd)Question 5: Write a Matlab code in order to approximate the true(unknown) nite sample distribution of bσ2 for a sample size N = 10, atrue value of σ2 = 16 and by using S = 1, 000 simulations.

(1) Plot an histogram of the 1,000 realisations of the estimator bσ2.(2) plot the Kernel estimator of the density fbσ2 (x) by using the Matlabbuilt-in function ksdensity.


Denition (Kernel density estimator)Let consider a sample X1, ..,XN , where X has a distribution characterizedby the pdf fX (x) , for x 2 R. A consistent (kernel) estimator of fX (x) forany x 2 R is given by:

bfX (x) = 1λN

N

∑i=1Kx xi

λ

where K (.) denotes a kernel function and λ is bandwidth parameter.

bfX (x) p! fX (x) 8x 2 R

For more details (and a discussion on the optimal choice of λ), see:

Lecture notes "Econométrie Non Paramétrique", Hurlin (2008), MasterEconométrie and Statistique Appliquée, Université dOrléans.


Denition (Kernel function)

A kernel function K (u) satises the following properties:(i) K (u) 0(ii)

RK (u) du = 1

(iii) K (u) reaches its maximum for u = 0 and decreases with juj.(iv) K (u) is symmetric, i.e. K (u) = K (u) .


Some examples of Kernel functions

Normal : K (u) =1p2π

expu

2

2

u 2 R

Triangular : K (u) = 1 juj u 2 [1, 1]

Quartic or BiWeight :K (u) =1516

1 u2

2u 2 [1, 1]

Epanechnikov :K (u) =34

1 u2

u 2 [1, 1]

Triweight :K (u) =3532

1 u2

3u 2 [1, 1]



5 10 15 20 25 30 350

50

100

150

200

250


0 5 10 15 20 25 30 35 400

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08


Problem (contd)

Question 6: For the special case where σ2 = 1, what is the nite sample(or exact sampling) distribution of the estimator bσ2 dened by

bσ2 = 12N

N

∑i=1R2i


Solution:

The estimator is dened by

bσ2 = 12N

N

∑i=1R2i

We know that R1,R2, ..,RN are i .i .d . random variables withRi Rayleigh (1)Reminder 1 : if Ri Rayleigh (1), then R =

pX 2 + Y 2 where X and Y

are independent and standard normally distributed N (0, 1) randomvariables.

Reminder 2: if X N (0, 1) , then X 2 χ2 (1)

Reminder 3: if X χ2 (v1) and Y χ2 (v2) , and X and Y areindependent, then X + Y χ2 (v1 + v2)


Solution (contd):

So, if X and Y are independent and standard normally distributedN (0, 1) random variables

Ri =pX 2 + Y 2 Rayleigh (1)

R2i = X2 + Y 2 χ2 (2)

The sum of independent chi-squared distributed random variable has achi-squared distribution.

N

∑i=1R2i χ2 (2N)


Solution (contd):

2Nbσ2 = N

∑i=1R2i χ2 (2N)

In the special case where σ2 = 1, the transformed variable 2Nbσ2 has anexact sampling (nite sample) distribution that corresponds to achi-squared distribution with 2N degrees of freedom.


Problem (contd)Question 7: Write a Matlab code in order to approximate the true(unknown) nite sample distribution of the transformed variable 2Nbσ2 fora sample size N = 10 in the special case where σ2 = 1 by usingS = 10, 000 simulations.

(1) Plot the Kernel estimator of the density f2Nbσ2 (x) by using the Matlabbuilt-in function ksdensity.

(2) Compare this estimated density function to the pdf of a chi-squareddistribution with 2N degrees of freedom.



0 10 20 30 40 50 60 700

0.01

0.02

0.03

0.04

0.05

0.06

0.07Estimated finite sample pdfTheoretical pdf of a chisquared


Problem (contd)

Question 8: What is the asymptotic distribution of the estimator bσ2 ?bσ2 = 1

2N

N

∑i=1R2i


Solution:

We know that R21 ,R22 , ..,R

2N are i .i .d . variables with

ER2i= 2σ2 V

R2i= 4σ4

Step 1: By applying the Lindberg-Levy univariate Central LimitTheorem (CLT), we get:

pN

1N

N

∑i=1R2i 2σ2

!d! N

0, 4σ4


Solution (contd):

Step 2: By denition, we have

bσ2 = 12N

N

∑i=1R2i = g

1N

N

∑i=1R2i

!pN

1N

N

∑i=1R2i 2σ2

!d! N

0, 4σ4

with g (x) = x/2. So, g (.) is a continuous and continuously di¤erentiablefunction with g

2σ26= 0 and not involving N, then the delta method

implies

pN

g

1N

N

∑i=1R2i

! g

2σ2! d! N

0,

∂g (x)∂x

2σ2

24σ4

!

g2σ2=2σ2

2= σ2

∂g (x)∂x

2σ2=

∂x/2∂x

2σ2=12


Solution (contd):

pN

g

1N

N

∑i=1R2i

! g

2σ2! d! N

0,12

24σ4

!

The estimator bσ2 is asymptotically normally distributedpNbσ2 σ2

d! N

0, σ4


Problem (contd)

Question 9: What is the asymptotic variance of the estimator bσ2 ?bσ2 = 1

2N

N

∑i=1R2i


Solution:

We know that:

pNbσ2 σ2

d! N

0, σ4

or equivalently bσ2 asy N

σ2,

σ4

N

The asymptotic variance of bσ2 is equal to:

Vasy

bσ2 = σ4

N


Problem (contd)Question 10: Write a Matlab code in order to approximate theasymptotic distribution of the transformed variable

Z =

pNbσ2 σ2

σ2

for a sample size N = 10, 000, a true value of σ2 = 16 by usingS = 10, 000 simulations.

(1) Plot the Kernel estimator of the density fZ (x) by using the Matlabbuilt-in function ksdensity.

(2) Compare this estimated density function to the pdf of a standardnormal distribution.



5 0 50

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0.45

Estimated finite sample pdfTheoretical pdf of a standard normal


Problem (contd)

Question 11: What is the asymptotic distribution of the estimator bσ ofthe parameter σ dened by:

bσ =vuut 12N

N

∑i=1R2i


Solution:

Step 1 : We know that:

pNbσ2 σ2

d! N

0, σ4

Since bσ > 0, we have:

bσ =vuut 12N

N

∑i=1R2i =

pbσ2 = g bσ2where g (x) =

px is a continuous and continuously di¤erentiable function

with gσ26= 0 and that does not depend on N


Solution (contd):

Step 2: We have pNbσ2 σ2

d! N

0, σ4

The delta method for g (x) =

px implies

pNgbσ2 g σ2 d! N

0,

∂g (x)∂x

σ2

2σ4

!

with

gσ2= σ

∂g (x)∂x

σ2=

∂px

∂x

σ2=

1

2p

σ2=

12σ


Solution (contd):

Step 2 (contd):

pNgbσ2 g σ2 d! N

0,

∂g (x)∂x

σ2

2σ4

!

gσ2= σ

∂g (x)∂x

σ2=

∂px

∂x

σ2=

1

2p

σ2=

12σ

So, we have:pN (bσ σ)

d! N0,

σ2

4


Exercise 2

CAPM


Problem (CAPM)The empirical analogue of the CAPM is given by:

rit rft| z excess return of security i for time t

= αi + βi (rmt rft )| z market excess return for time t

+ εt

where εt is an i .i .d . error term. We assume that

erit = rit rft ermt = rmt rftE (εt ) = 0 V (εt ) = σ2 E ( εt jermt ) = 0


Problem (CAPM, contd)Consider the model erit = αi + βiermt + εt

Data: Microsoft, SP500 and Tbill (closing prices) from 11/1/1993 to04/03/2003

0.10

0.05

0.00

0.05

0.10

0.06 0.04 0.02 0.00 0.02 0.04 0.06 0.08

RSP500

RM

SFT

0.08

0.04

0.00

0.04

0.08

500 1000 1500 2000

RSP500 RMSFT


Problem (CAPM, contd)We consider the CAPM model rewritten as follows

erit = x>t β+ εt t = 1, ..T

where xt = (1 ermt )> is 2 1 vector of random variables, β = (αi βi )> is

2 1 vector of parameters, and where the error term εt satisesE (εt ) = 0, V (εt ) = σ2 and E ( εt jermt ) = 0.


Problem (CAPM, contd)Question 1: show that the OLS estimator

bβ = T

∑t=1xtx>t

!1 T

∑t=1xterit

!

satises pTbβ β0

d! N

0, σ2E1

x>t xt


Solution:

1 let us rewrite the OLS estimator as:

bβ = T

∑t=1xtx>t

!1 T

∑t=1xterit

!= β0 +

T

∑t=1xtx>t

!1 T

∑t=1xt εt

!

2 Normalize the vector bβ β0

pTbβ β0

=

1T

T

∑t=1xtx>t

!1 pT1T

T

∑t=1xt εt

!


Solution (contd):

3. Using the WLLN and the CMP: 1T

T

∑t=1xtx>t

!1p! E1

xtx>t

4. Using the CLT:

pT

1T

T

∑t=1xt εt E (xt εt )

!d! N (0,V (xt εt ))

with E ( εt jermt ) = 0 and E ( εt j 1) = E (εt ) = 0 =) E (xt εt ) = 0and

V (xt εt ) = Ext εt εtx>t

= E

Ext εt εtx>t

xt= E

xtV ( εt j xt ) x>t

= σ2E

xtx>t


Solution (contd):

So, we have 1T

T

∑t=1xtx>t

!1p! E1

xtx>t

pT

1T

T

∑t=1xt εt

!d! N

0, σ2E

xtx>t


Solution (contd):

By using the Slutskys theorem (for a convergence in distribution), wehave:

pTbβ β0

=

1T

T

∑t=1xtx>t

!1 pT1T

T

∑t=1xtµt

!d! N (Π,Ω)

withΠ = E1

xtx>t

0 = 0

Ω = E1xtx>t

σ2E

xtx>t

E1

xtx>t

= σ2E1

xtx>t

Finally, we have:

pTbβ β0

d! N

0, σ2E1

xtx>t


Problem (CAPM, contd)Question 2: What is the asymptotic variance-covariance matrix of theOLS estimator bβ ?

bβ = T

∑t=1xtx>t

!1 T

∑t=1xterit

!


Solution:

We shown thatpTbβ β0

d! N

0, σ2E1

xtx>t

or equivalently bβ asy N

β0,

σ2

TE1

xtx>t

The asymptotic variance-covariance matrix of bβ is equal to:

Vasy

bβ = σ2

TE1

xtx>t

| z

22


Remarks:

1 The asymptotic variance covariance matrix is a 2 2 symmetricmatrix:

Vasy

bβ = σ2

TE1

xtx>t

=

0@ Vasy (bα) covbα, bβ

covbβ,bα Vasy

bβ1A

2 Since xt = (1 ermt )>, we have:Extx>t

= E

1 ermtermt er2mt

=

1 E (ermt )

E (ermt ) Eer2mt


Problem (CAPM, contd)

Question 3: Let us consider a consistent estimator of σ2 dened by:

bσ2 = 1T 2

T

∑t=1bε2t = 1

T 2T

∑t=1

erit x>t bβ2Propose a consistent estimator of the asymptotic variance Vasy

bβ?


Solution:

We know that

Vasy

bβ = σ2

TE1

xtx>t

Using the LLN, if xt is i .i .d ., we get:

1T

T

∑t=1xtx>t

p! Extx>t

Using the CMP:

1T

T

∑t=1xtx>t

!1p! E1

xtx>t


Solution (contd):

So, we have

Vasy

bβ = σ2

TE1

xtx>t

and

1T

T

∑t=1xtx>t

!1p! E1

xtx>t

bσ2 p! σ2

By using the Slutskys theorem:

bσ2T

1T

T

∑t=1xtx>t

!1p! σ2

TE1

xtx>t

= Vasy

bβ


Solution (contd):

A consistent estimator of the asymptotic variance Vasy

bβ is dened bybVasy

bβ = bσ2T

1T

T

∑t=1xtx>t

!1Or equivalently by

bVasy

bβ = bσ2 T

∑t=1xtx>t

!1with bσ2 = 1

T 2T

∑t=1bε2t = 1

T 2T

∑t=1

erit x>t bβ2


Remark:

bVasy

bβ = bσ2 T

∑t=1xtx>t

!1Since xt = (1 ermt )>:

T

∑t=1xtx>t =

T ∑T

t=1 ermt∑Tt=1 ermt ∑T

t=1 er2mt


Problem (CAPM, contd)Question 4: Using the excel le capm.xls, write a Matlab code toestimate the beta and the alpha for MSFT. Compare your results with thefollowing table of estimation results (Eviews).



Perfect....


Problem (CAPM, contd)Question 5: Using the excel le capm.xls, write a Matlab code

(1) to estimate the variance of the error term εt

(2) to estimate the asymptotic standard errors of the estimators bβ(3) Compare your results with the table of estimation results (Eviews).



Perfect too....


End of Exercices - Chapter 1

Christophe Hurlin (University of Orléans)


Documents

Exercises Chapter 1 Estimation Theory - univ-orleans.fr · Exercises Chapter 1 Estimation Theory Advanced Econometrics - HEC Lausanne Christophe Hurlin University of OrlØans November