Exercises Chapter 1 Estimation Theory - univ-orleans.fr · Exercises Chapter 1 Estimation Theory...

Exercises Chapter 1

Estimation Theory

Advanced Econometrics - HEC Lausanne

Christophe Hurlin

University of Orléans

November 2013

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne November 2013 1 / 68

Exercise 1

Rayleigh distribution

ProblemWe consider two continuous independent random variables X and Ynormally distributed with N

0, σ2

. The transformed variable R dened

by:R =

pX 2 + Y 2

has a Rayleigh distribution with a parameter σ2 :

R Rayleighσ2

with a pdf fRr ; σ2

dened by:

fRr ; σ2

σ2exp

8r 2 [0,+∞[

E (R) = σ

2V (R) =

Problem (contd)

We consider an i .i .d . sample fR1,R2, ..,RNg and an estimator (MLE) ofσ2 dened by

bσ2 = 12N

∑i=1R2i

Question 1: Show that bσ2 is an unbiased estimator of σ2.

Solution:

We have: bσ2 = 12N

∑i=1R2i

We want to calculate Ebσ2 .

Since the sample fR1,R2, ..,RNg is i .i .d ., we have:

Ebσ2 = E

∑i=1R2i

∑i=1

Solution (contd):

Ebσ2 = E

∑i=1R2i

∑i=1

We know that:

E (Ri ) = σ

2V (Ri ) =

So, we have:

ER2i= V (Ri ) +E (Ri )

= 2σ2

Solution (contd):

Ebσ2 =

∑i=1

∑i=12σ2

=N2σ2

So, we have:Ebσ2 = σ2

The estimator bσ2 is unbiased.Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne November 2013 7 / 68

Remark: Sometimes, the Rayleigh distribution is parametrized by σ. But,it is easier to show that bσ2 is unbiased than to show that bσ is unbiased...

bσ =vuut 12N

∑i=1R2i

Then to study the bias, we have to calculate:

E (bσ) = E

0@vuut 12N

∑i=1R2i

1A ???

since for a nonlinear function g (.) , E (g (x)) 6= g (E (x)) ... The onlysolution is to compute the integral

E (bσ) = R ∞0 x fbσ x ; σ2 dx

Problem (contd)

Question 2: Show that bσ2 is a (weakly) consistent estimator of σ2. Weadmit that the raw moments of R are dened by:

ERki= σk 2

k2 Γ1+

and where Γ (.) denotes the gamma function with:

Γ (x) =∞Z0

tx1 exp (t) dt.

andΓ (x) = (x 1)! for x 2 N

Solution:

First, calculate Vbσ2 . Since the sample fR1,R2, ..,RNg is i .i .d ., we

Vbσ2 = V

∑i=1R2i

∑i=1

What is the value of VR2i?

VR2i= E

R4iER2i2

We shown thatER2i= 2σ2

Solution (contd):

VR2i= E

R4iER2i2

= ER4i 4σ4

What is the value of ER4i? For any k 2 N :

ERki= σk 2

k2 Γ1+

For k = 4, we have:

ER4i= σ4 22 Γ (3)

withΓ (3) = (3 1)! = 2! = 2

So, we have:ER4i= σ4 23 = 8σ4

Solution (contd):

The variance of R2i is equal to:

VR2i= E

R4i 4σ4 = 8σ4 4σ4 = 4σ4

As a consequence

Vbσ2 =

∑i=1

∑i=14σ4

=N4σ4

Solution (contd):

To sum up:Ebσ2 = σ2 (unbiased estimator)

limN!∞

Vbσ2 = lim

So, the estimator bσ2 is a (weakly) consistent estimator of σ2:

bσ2 p! σ2

Problem (contd)Question 3: Sometimes, the Rayleigh distribution is parametrized by σ(and not by σ2) with

R Rayleigh (σ)Propose a (weakly) consistent estimator for σ.

Solution:

Since σ > 0, a natural estimator for σ is dened by:

bσ =vuut 12N

∑i=1R2i =

pbσ2We shown that the estimator bσ2 is a (weakly) consistent estimator of σ2 :

bσ2 p! σ2

By applying the Continuous Mapping Theorem (CMP) for the continuousfunction g (x) =

px , we get immediately:

gbσ2 p! g

or bσ p! σ

Problem (contd)

Question 4: For any value of σ2, what is the nite sample (or exactsampling) distribution of the estimator bσ2 dened by

bσ2 = 12N

∑i=1R2i

Solution:

The estimator is dened by

bσ2 = 12N

∑i=1R2i

We know that R1,R2, ..,RN are i .i .d . random variables with a Rayleighdistribution.

Ri Rayleighσ2

Reminder: if X and Y are independent and normally distributed N0, σ2

random variables, the transformed variable R =

pX 2 + Y 2 has a Rayleigh

distribution.

The exact distribution of R2 = X 2 + Y 2 is unknown (it is not a χ2...),and as a consequence the nite sample distribution of bσ2 is unknown.

Problem (contd)Question 5: Write a Matlab code in order to approximate the true(unknown) nite sample distribution of bσ2 for a sample size N = 10, atrue value of σ2 = 16 and by using S = 1, 000 simulations.

(1) Plot an histogram of the 1,000 realisations of the estimator bσ2.(2) plot the Kernel estimator of the density fbσ2 (x) by using the Matlabbuilt-in function ksdensity.

Denition (Kernel density estimator)Let consider a sample X1, ..,XN , where X has a distribution characterizedby the pdf fX (x) , for x 2 R. A consistent (kernel) estimator of fX (x) forany x 2 R is given by:

bfX (x) = 1λN

∑i=1Kx xi

where K (.) denotes a kernel function and λ is bandwidth parameter.

bfX (x) p! fX (x) 8x 2 R

For more details (and a discussion on the optimal choice of λ), see:

Lecture notes "Econométrie Non Paramétrique", Hurlin (2008), MasterEconométrie and Statistique Appliquée, Université dOrléans.

Denition (Kernel function)

A kernel function K (u) satises the following properties:(i) K (u) 0(ii)

RK (u) du = 1

(iii) K (u) reaches its maximum for u = 0 and decreases with juj.(iv) K (u) is symmetric, i.e. K (u) = K (u) .

Some examples of Kernel functions

Normal : K (u) =1p2π

Triangular : K (u) = 1 juj u 2 [1, 1]

Quartic or BiWeight :K (u) =1516

2u 2 [1, 1]

Epanechnikov :K (u) =34

u 2 [1, 1]

Triweight :K (u) =3532

3u 2 [1, 1]

5 10 15 20 25 30 350

0 5 10 15 20 25 30 35 400

Problem (contd)

Question 6: For the special case where σ2 = 1, what is the nite sample(or exact sampling) distribution of the estimator bσ2 dened by

bσ2 = 12N

∑i=1R2i

Solution:

The estimator is dened by

bσ2 = 12N

∑i=1R2i

We know that R1,R2, ..,RN are i .i .d . random variables withRi Rayleigh (1)Reminder 1 : if Ri Rayleigh (1), then R =

pX 2 + Y 2 where X and Y

are independent and standard normally distributed N (0, 1) randomvariables.

Reminder 2: if X N (0, 1) , then X 2 χ2 (1)

Reminder 3: if X χ2 (v1) and Y χ2 (v2) , and X and Y areindependent, then X + Y χ2 (v1 + v2)

Solution (contd):

So, if X and Y are independent and standard normally distributedN (0, 1) random variables

Ri =pX 2 + Y 2 Rayleigh (1)

R2i = X2 + Y 2 χ2 (2)

The sum of independent chi-squared distributed random variable has achi-squared distribution.

∑i=1R2i χ2 (2N)

Solution (contd):

2Nbσ2 = N

∑i=1R2i χ2 (2N)

In the special case where σ2 = 1, the transformed variable 2Nbσ2 has anexact sampling (nite sample) distribution that corresponds to achi-squared distribution with 2N degrees of freedom.

Problem (contd)Question 7: Write a Matlab code in order to approximate the true(unknown) nite sample distribution of the transformed variable 2Nbσ2 fora sample size N = 10 in the special case where σ2 = 1 by usingS = 10, 000 simulations.

(1) Plot the Kernel estimator of the density f2Nbσ2 (x) by using the Matlabbuilt-in function ksdensity.

(2) Compare this estimated density function to the pdf of a chi-squareddistribution with 2N degrees of freedom.

0 10 20 30 40 50 60 700

0.07Estimated finite sample pdfTheoretical pdf of a chisquared

Problem (contd)

Question 8: What is the asymptotic distribution of the estimator bσ2 ?bσ2 = 1

∑i=1R2i

Solution:

We know that R21 ,R22 , ..,R

2N are i .i .d . variables with

ER2i= 2σ2 V

R2i= 4σ4

Step 1: By applying the Lindberg-Levy univariate Central LimitTheorem (CLT), we get:

∑i=1R2i 2σ2

0, 4σ4

Solution (contd):

Step 2: By denition, we have

bσ2 = 12N

∑i=1R2i = g

∑i=1R2i

∑i=1R2i 2σ2

0, 4σ4

with g (x) = x/2. So, g (.) is a continuous and continuously di¤erentiablefunction with g

2σ26= 0 and not involving N, then the delta method

implies

∑i=1R2i

2σ2! d! N

∂g (x)∂x

g2σ2=2σ2

2= σ2

∂g (x)∂x

∂x/2∂x

2σ2=12

Solution (contd):

∑i=1R2i

2σ2! d! N

The estimator bσ2 is asymptotically normally distributedpNbσ2 σ2

0, σ4

Problem (contd)

Question 9: What is the asymptotic variance of the estimator bσ2 ?bσ2 = 1

∑i=1R2i

Solution:

We know that:

pNbσ2 σ2

0, σ4

or equivalently bσ2 asy N

The asymptotic variance of bσ2 is equal to:

bσ2 = σ4

Problem (contd)Question 10: Write a Matlab code in order to approximate theasymptotic distribution of the transformed variable

pNbσ2 σ2

for a sample size N = 10, 000, a true value of σ2 = 16 by usingS = 10, 000 simulations.

(1) Plot the Kernel estimator of the density fZ (x) by using the Matlabbuilt-in function ksdensity.

(2) Compare this estimated density function to the pdf of a standardnormal distribution.

5 0 50

Estimated finite sample pdfTheoretical pdf of a standard normal

Problem (contd)

Question 11: What is the asymptotic distribution of the estimator bσ ofthe parameter σ dened by:

bσ =vuut 12N

∑i=1R2i

Solution:

Step 1 : We know that:

pNbσ2 σ2

0, σ4

Since bσ > 0, we have:

bσ =vuut 12N

∑i=1R2i =

pbσ2 = g bσ2where g (x) =

px is a continuous and continuously di¤erentiable function

with gσ26= 0 and that does not depend on N

Solution (contd):

Step 2: We have pNbσ2 σ2

0, σ4

The delta method for g (x) =

px implies

pNgbσ2 g σ2 d! N

∂g (x)∂x

gσ2= σ

∂g (x)∂x

Solution (contd):

Step 2 (contd):

pNgbσ2 g σ2 d! N

∂g (x)∂x

gσ2= σ

∂g (x)∂x

So, we have:pN (bσ σ)

d! N0,

Exercise 2

Problem (CAPM)The empirical analogue of the CAPM is given by:

rit rft| z excess return of security i for time t

= αi + βi (rmt rft )| z market excess return for time t

where εt is an i .i .d . error term. We assume that

erit = rit rft ermt = rmt rftE (εt ) = 0 V (εt ) = σ2 E ( εt jermt ) = 0

Problem (CAPM, contd)Consider the model erit = αi + βiermt + εt

Data: Microsoft, SP500 and Tbill (closing prices) from 11/1/1993 to04/03/2003

0.06 0.04 0.02 0.00 0.02 0.04 0.06 0.08

RSP500

500 1000 1500 2000

RSP500 RMSFT

Problem (CAPM, contd)We consider the CAPM model rewritten as follows

erit = x>t β+ εt t = 1, ..T

where xt = (1 ermt )> is 2 1 vector of random variables, β = (αi βi )> is

2 1 vector of parameters, and where the error term εt satisesE (εt ) = 0, V (εt ) = σ2 and E ( εt jermt ) = 0.

Problem (CAPM, contd)Question 1: show that the OLS estimator

bβ = T

∑t=1xtx>t

∑t=1xterit

satises pTbβ β0

0, σ2E1

x>t xt

Solution:

1 let us rewrite the OLS estimator as:

bβ = T

∑t=1xtx>t

∑t=1xterit

!= β0 +

∑t=1xtx>t

∑t=1xt εt

2 Normalize the vector bβ β0

pTbβ β0

∑t=1xtx>t

!1 pT1T

∑t=1xt εt

Solution (contd):

3. Using the WLLN and the CMP: 1T

∑t=1xtx>t

!1p! E1

4. Using the CLT:

∑t=1xt εt E (xt εt )

!d! N (0,V (xt εt ))

with E ( εt jermt ) = 0 and E ( εt j 1) = E (εt ) = 0 =) E (xt εt ) = 0and

V (xt εt ) = Ext εt εtx>t

Ext εt εtx>t

xtV ( εt j xt ) x>t

= σ2E

Solution (contd):

So, we have 1T

∑t=1xtx>t

!1p! E1

∑t=1xt εt

0, σ2E

Solution (contd):

By using the Slutskys theorem (for a convergence in distribution), wehave:

pTbβ β0

∑t=1xtx>t

!1 pT1T

∑t=1xtµt

!d! N (Π,Ω)

withΠ = E1

Ω = E1xtx>t

= σ2E1

Finally, we have:

pTbβ β0

0, σ2E1

Problem (CAPM, contd)Question 2: What is the asymptotic variance-covariance matrix of theOLS estimator bβ ?

bβ = T

∑t=1xtx>t

∑t=1xterit

Solution:

We shown thatpTbβ β0

0, σ2E1

or equivalently bβ asy N

The asymptotic variance-covariance matrix of bβ is equal to:

bβ = σ2

Remarks:

1 The asymptotic variance covariance matrix is a 2 2 symmetricmatrix:

bβ = σ2

0@ Vasy (bα) covbα, bβ

covbβ,bα Vasy

2 Since xt = (1 ermt )>, we have:Extx>t

1 ermtermt er2mt

1 E (ermt )

E (ermt ) Eer2mt

Problem (CAPM, contd)

Question 3: Let us consider a consistent estimator of σ2 dened by:

bσ2 = 1T 2

∑t=1bε2t = 1

∑t=1

erit x>t bβ2Propose a consistent estimator of the asymptotic variance Vasy

Solution:

We know that

bβ = σ2

Using the LLN, if xt is i .i .d ., we get:

∑t=1xtx>t

p! Extx>t

Using the CMP:

∑t=1xtx>t

!1p! E1

Solution (contd):

So, we have

bβ = σ2

∑t=1xtx>t

!1p! E1

bσ2 p! σ2

By using the Slutskys theorem:

∑t=1xtx>t

!1p! σ2

= Vasy

Solution (contd):

A consistent estimator of the asymptotic variance Vasy

bβ is dened bybVasy

bβ = bσ2T

∑t=1xtx>t

!1Or equivalently by

bβ = bσ2 T

∑t=1xtx>t

!1with bσ2 = 1

∑t=1bε2t = 1

∑t=1

erit x>t bβ2

Remark:

bβ = bσ2 T

∑t=1xtx>t

!1Since xt = (1 ermt )>:

∑t=1xtx>t =

T ∑T

t=1 ermt∑Tt=1 ermt ∑T

t=1 er2mt

Problem (CAPM, contd)Question 4: Using the excel le capm.xls, write a Matlab code toestimate the beta and the alpha for MSFT. Compare your results with thefollowing table of estimation results (Eviews).

Perfect....

Problem (CAPM, contd)Question 5: Using the excel le capm.xls, write a Matlab code

(1) to estimate the variance of the error term εt

(2) to estimate the asymptotic standard errors of the estimators bβ(3) Compare your results with the table of estimation results (Eviews).

Perfect too....

End of Exercices - Chapter 1

Christophe Hurlin (University of Orléans)

Exercises Chapter 1 Estimation Theory - univ-orleans.fr · Exercises Chapter 1 Estimation Theory...

Documents

Review Reading Skill Review Reading Skill Exercises Exercises

PATELLOFEMORAL EXERCISES STRETCHING EXERCISES

Homework Freshmen : Remaining Exercises Juniors: Remaining Exercises

Vocal Function Exercises Laryngeal Adduction Exercises

Advanced Econometrics II · Advanced Econometrics II School of Economics and Management - University of Geneva Christophe Hurlin, UniversitØ d OrlØans University of OrlØans

Minor Pentatonic Technical Technical Exercises Bass Clef Exercises

22b Module 22 -- Resource Estimation -- Exercises

Search Frictions, Credit Market Liquidity, and Net ... · seminar participants of the GDR Monnaie et Finance (OrlØans, 2009), the Annual Meeting of the SED (MontrØal, 2010), the

EXERCISES BOOKLET - University of Oxfordlogicmanual.philosophy.ox.ac.uk/exercises/exercises.pdf · EXERCISES BOOKLET forthe LogicManual óþÕß/óþÕŸ erearenochangestotheexercises

May-pole exercises : a collection of exercises compiled

Statistics Sampling Distributions and Point Estimation of Parameters Contents, figures, and exercises come from the textbook: Applied Statistics and Probability

1-3 Exercises - Spokane Public Schoolsswcontent.spokaneschools.org/cms/lib/WA01000970/Centricity/Domain/...17.∠RST if m∠RSP= ( 3x - 2) ... Estimation Use the following information

Exercises Pythonbasics Exercises v1.0

Gaussian estimation: Sequence and wavelet modelsjbradic/math281a/Johnstone.pdf12.5 Taming the shell bounds 319 Exercises 322 13 Sharp minimax estimation on ‘ pballs 324 13.1 Linear

Eye Exercises and Eye Relaxation Exercises

SOLUTIONS TO SELECTED EXERCISES A GUIDE TO ESTIMATION …

Chapter 2: Maximum Likelihood Estimation - · PDF fileChapter 2: Maximum Likelihood Estimation Advanced Econometrics - HEC Lausanne Christophe Hurlin University of OrlØans December

Creativity Exercises, Fantasy Developing Exercises (FAFEJ ...1 Creativity Exercises, Fantasy Developing Exercises (FAFEJ) and Inter-Disciplinary-Thinking (InDiGo). Miklós Erdély’s

Matlab Exercises - University of New South WalesMatlab Exercises 2 1 Matlab Exercises for Chapter 1 % Matlab exercises for Chapter 1 % The following codes provide some exercises or

OVERVIEW OF BRIEF EXERCISES, EXERCISES, PROBLEMS …unieducation.yolasite.com/resources/Chap005.pdf · 5.15 3, 6 *Supplemental Topic ... OVERVIEW OF BRIEF EXERCISES, EXERCISES, PROBLEMS