88
Exercises Chapter 4 Statistical Hypothesis Testing Advanced Econometrics - HEC Lausanne Christophe Hurlin University of OrlØans December 5, 2013 Christophe Hurlin (University of OrlØans) Advanced Econometrics - HEC Lausanne December 5, 2013 1 / 88

Exercises Chapter 4 Statistical Hypothesis Testing · Exercises Chapter 4 Statistical Hypothesis Testing Advanced Econometrics - HEC Lausanne Christophe Hurlin University of OrlØans

Embed Size (px)

Citation preview

Exercises Chapter 4

Statistical Hypothesis TestingAdvanced Econometrics - HEC Lausanne

Christophe Hurlin

University of Orléans

December 5, 2013

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 1 / 88

Exercise 1

Parametric tests and the Neyman Pearson lemma

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 2 / 88

ProblemWe consider two continuous independent random variables U and Wnormally distributed with N

�0, σ2

�. The transformed variable X de�ned

by:X =

pU2 +W 2

has a Rayleigh distribution with a parameter σ2 :

X � Rayleigh�σ2�

with a pdf fX�x ; σ2

�de�ned by:

fX�x ; σ2

�=xσ2exp

�� x2

2σ2

�8x 2 [0,+∞[

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 3 / 88

Problem (cont�d)

Question 1: we consider an i .i .d . sample fX1,X2, ..,XNg . Derive theMLE estimator of σ2

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 4 / 88

Solution

fX�x ; σ2

�=xσ2exp

�� x2

2σ2

�8x 2 [0,+∞[

The log-likelihood of the i .i .d . sample fx1, x2, .., xNg is

`N�σ2; x

�=

N

∑i=1ln fX

�xi ; σ2

�=

N

∑i=1ln (xi )�N ln

�σ2�� 12σ2

N

∑i=1x2i

The ML estimator bσ2 is de�ned as to be:bσ2 = argmax

σ2>0`N�σ2; x

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 5 / 88

Solution (cont�d)

bσ2 = argmaxσ2>0

N

∑i=1ln (xi )�N ln

�σ2�� 12σ2

N

∑i=1x2i

FOC (log-likelihood equations)

∂`N�σ2; x

�∂σ2

�����bσ2 = �Nbσ2 + 1

2bσ4N

∑i=1x2i = 0

So, the ML estimator of σ2 is

bσ2 = 12N

N

∑i=1X 2i

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 6 / 88

Solution (cont�d)

∂`N�σ2; x

�∂σ2

= � Nσ2+

12σ4

N

∑i=1x2i

SOC:

∂2`N�σ2; x

�∂σ4

�����bσ2 =Nbσ4 � 1bσ6

N

∑i=1x2i

=Nbσ4 � 2Nbσ

2

bσ6= � Nbσ4 < 0

since ∑Ni=1 x

2i = 2Nbσ2. So, we have a maximum. �

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 7 / 88

Problem (cont�d)

Question 2: what is the asymptotic distribution of the MLE estimator bσ2?

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 8 / 88

Solution

The average Fisher information matrix associated to the sample is:

IN�σ2�= Eσ2

∂2`N�σ2;X

�∂σ4

!

= Eσ2

� N

σ4+1

σ6

N

∑i=1X 2i

!

= � Nσ4+1

σ4

N

∑i=1

Eσ2

�X 2iσ2

�Since (X/σ)2 = (U/σ)2 + (W/σ)2 where U/σ and W/σ are twoindependent standard normal variables, then X 2/σ2 � χ2 (2) with

Eσ2

�X 2iσ2

�= 2

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 9 / 88

Solution (cont�d)

So, we have

IN�σ2�= � N

σ4+1

σ4

N

∑i=1

Eσ2

�X 2iσ2

�= � N

σ4+2Nσ4

=Nσ4

Since the sample is i .i .d ., the average Fisher information matrix is:

I�σ2�=1NIN�σ2�=1

σ4

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 10 / 88

Solution (cont�d)

The regularity conditions hold, and we have:

pN�bσ2 � σ2

�d! N

�0, I�1

�σ2��

Here pN�bσ2 � σ2

�d! N

�0, σ4

�where σ2 denotes the true value of the parameter. Or equivalently:

bσ2 asy� N�

σ2,σ4

N

��

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 11 / 88

Problem (cont�d)Question 3: consider the test

H0 : σ2 = σ20 H1 : σ2 = σ21

with σ21 > σ20. Determine the critical region of the UMP test of size α.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 12 / 88

Solution

Given the Neyman Pearson lemma, the rejection region is given by:

W =

(x1, .., xN j

LN�σ20; x1, ., xN

�LN (σ21; x1, ., xN )

< K

)

where K is a constant determined by the level of the test α. So, we have

`N�σ20; x

�� `N

�σ21; x

�< ln (K )

() ∑Ni=1 ln (xi )�N ln

�σ20�� 1

2σ20∑Ni=1 x

2i �∑N

i=1 ln (xi )

+N ln�σ21�+ 1

2σ21∑Ni=1 x

2i < ln (K )

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 13 / 88

Solution (cont�d)

N�ln�σ21�� ln

�σ20��+

�1

σ21� 1

σ20

�12

N

∑i=1x2i < ln (K )

()�1

σ21� 1

σ20

�12

N

∑i=1x2i < K1

with K1 = ln (K )�N�log�σ21�� log

�σ20��. or equivalently:�

σ20 � σ21σ20σ

21

�12

N

∑i=1x2i < K1

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 14 / 88

Solution (cont�d) �σ20 � σ21

σ20σ21

�12

N

∑i=1x2i < K1

Since σ21 > σ20, we have:12N

N

∑i=1x2i > A

where A = K1σ20σ21/��

σ20 � σ21�N�is a constant determined by α.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 15 / 88

Solution (cont�d)

The rejection region of the UMP test of size α

H0 : σ2 = σ20 H1 : σ2 = σ21

with σ21 > σ20 is:

W =nx : bσ2 (x) > Ao

where the critical value A is a constant determined by the size α andbσ2 (x) is the realisation of the ML estimator bσ2 (the test statistic):bσ2 = 1

2N

N

∑i=1X 2i

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 16 / 88

Solution (cont�d)

Given the de�nition of the size:

α = Pr (WjH0) = Pr�bσ2 > A���H0�

Under the null, for N large, we have:

bσ2 asy�H0N�

σ20,σ40N

�Then

1� α = Pr

bσ2 � σ20σ20/

pN<A� σ20σ20/

pN

�����H0!

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 17 / 88

Solution (cont�d)

1� α = Pr

bσ2 � σ20σ20/

pN<A� σ20σ20/

pN

�����H0!

Denote by Φ (.) the cdf of the standard normal distribution:

A = σ20 +σ20pN

Φ�1 (1� α)

The rejection region of the UMP test of size α

H0 : σ2 = σ20 H1 : σ2 = σ21

with σ21 > σ20 is:

W =

�x : bσ2 (x) > σ20 +

σ20pN

Φ�1 (1� α)

��

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 18 / 88

Problem (cont�d)Question 4: consider the test

H0 : σ2 = 2 H1 : σ2 > 2

For a sample of size N = 100, we have

N

∑i=1x2i = 470

What is the conclusion of the test for a size of 10%?

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 19 / 88

Solution

Consider the test

H0 : σ2 = σ20 H1 : σ2 = σ21

with σ21 > σ20. The rejection region of the UMP test of size α is given by:

W =

�x : bσ2 (x) > σ20 +

σ20pN

Φ�1 (1� α)

�This region does not depend on the value of σ21. So, it corresponds to therejection region of the one-sided UMP of size α :

H0 : σ2 = σ20 H1 : σ2 > σ20

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 20 / 88

Solution (cont�d)

H0 : σ2 = 2 H1 : σ2 > 2

W =

�x : bσ2 (x) > σ20 +

σ20pN

Φ�1 (1� α)

�NA: N = 100, α = 10% :

W =

�x : bσ2 (x) > 2+ 2

10Φ�1 (0.9)

�W =

nx : bσ2 (x) > 2.2563o

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 21 / 88

Solution (cont�d)

W =nx : bσ2 (x) > 2.2563o

For this sample (N = 100) we have ∑Ni=1 x

2i = 470, and as a consequence

bσ2 (x) = 12N

N

∑i=1x2i =

470200

= 2.35

For a signi�cance level of 10%, we reject the null H0 : σ2 = 2.�

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 22 / 88

Problem (cont�d)Question 5: determine the power of the one-sided UMP test of size α for:

H0 : σ2 = σ20 H1 : σ2 > σ20

Numerical application: N = 100, σ20 = 2 and α = 10%.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 23 / 88

Solution

The rejection region of the UMP of size α is:

W =nx : bσ2 (x) > Ao

with σ20 +Φ�1 (1� α) σ20/pN. By of the power, we have:

power = Pr (WjH1) = Pr�bσ2 > σ20 +

σ20pN

Φ�1 (1� α)

����H1�Under the alternative hypothesis, for N large, we have:

bσ2 asy�H1N�

σ2,σ4

N

�σ2 > σ20

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 24 / 88

Solution (cont�d)

Then, the power is equal to:

power = 1� Pr bσ2 � σ2

σ2/pN<A� σ2

σ2/pN

�����H1!= 1�Φ

�A� σ2

σ2/pN

Given the de�nition of the critical value A = σ20 +Φ�1 (1� α) σ20/pN,

we have:

power = 1�Φ�

σ20 � σ2

σ2/pN+

σ20σ2

Φ�1 (1� α)

�8σ2 > σ20 �

NA: σ20 = 2, N = 100 and α = 10%

power = 1�Φ�2� σ2

σ2/10+2

σ2Φ�1 (0.9)

�8σ2 > σ20

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 25 / 88

Solution (cont�d)

2 2.2 2.4 2.6 2.8 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

σ2

pow

er

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 26 / 88

Solution (cont�d)

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 27 / 88

Problem (cont�d)Question 6: consider the two-sided test

H0 : σ2 = σ20 H1 : σ2 6= σ20

What is the critical region of the test of size α?

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 28 / 88

Solution

Consider the one-sided tests:

Test A: H0 : σ2 = σ20 against H1 : σ2 < σ20

Test B: H0 : σ2 = σ20 against H1 : σ2 > σ20

The non-rejection regions of the UMP one-sided tests of size α/2 are:

WA =

�x : bσ2 (x) > σ20 +

σ20pN

Φ�1�α

2

��

WB =

�x : bσ2 (x) < σ20 +

σ20pN

Φ�1�1� α

2

��The non rejection region of the two-sided test corresponds to theintersection of these two regions:

W = W A \W B

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 29 / 88

Solution (cont�d)

So, the non rejection region of the two-sided test of size α is:

W =

�x : σ20 +

σ20pN

Φ�1�α

2

�< bσ2 (x) < σ20 +

σ20pN

Φ�1�1� α

2

��Since, Φ�1 (α/2) = �Φ�1 (1� α/2) , this region can be rewritten as:

W =

�x :���bσ2 (x)� σ20

��� > σ20pN

Φ�1�1� α

2

��The rejection region of the two-sided of size α is:

W =

�x :���bσ2 (x)� σ20

��� < σ20pN

Φ�1�1� α

2

���

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 30 / 88

Problem (cont�d)Question 7: determine the power of the two-sided test of size α for:

H0 : σ2 = σ20 H1 : σ2 6= σ20

Numerical application: N = 100, σ20 = 2 and α = 10%.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 31 / 88

Solution

The non rejection region of the two-sided test of size α is:

W =nx : A < bσ2 (x) < Bo

A = σ20 +σ20pN

Φ�1�α

2

�B = σ20 +

σ20pN

Φ�1�1� α

2

�By de�nition of the power:

power = Pr (WjH1) = 1� Pr�W��H1�

So, we have:

power = 1� Pr�bσ2 < B�+ Pr �bσ2 < A�

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 32 / 88

Solution (cont�d)

power = 1� Pr�bσ2 < B�+ Pr �bσ2 < A�

Under the alternative

bσ2 asy�H1N�

σ2,σ4

N

�σ2 6= σ20

So, we have

power = 1�Φ�B � σ2

σ2/pN

�+Φ

�A� σ2

σ2/pN

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 33 / 88

Solution (cont�d)

We have

power = 1�Φ�B � σ2

σ2/pN

�+Φ

�A� σ2

σ2/pN

�A = σ20 +

σ20pN

Φ�1�α

2

�B = σ20 +

σ20pN

Φ�1�1� α

2

�So 8σ2 6= σ20, the power function of the two sided test is de�ned by:

power = 1�Φ�

σ20 � σ2

σ2/pN+

σ20σ2

Φ�1�1� α

2

��+Φ

�σ20 � σ2

σ2/pN+

σ20σ2

Φ�1�α

2

���

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 34 / 88

Solution (cont�d)

NA: N = 100, α = 10% and σ20 = 2. 8σ2 6= 2

power = 1�Φ�2� σ2

σ2/10+2

σ2Φ�1 (0.95)

�+Φ

�2� σ2

σ2/10+2

σ2Φ�1 (0.05)

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 35 / 88

Solution (cont�d)

1 1.5 2 2.5 30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

σ2

pow

er

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 36 / 88

Solution (cont�d)

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 37 / 88

Problem (cont�d)Question 8: show that the two-sided test is unbiased and consistent.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 38 / 88

Solution

The power function is de�ned as to be:

P�σ2�= 1�Φ

�σ20 � σ2

σ2/pN+

σ20σ2

Φ�1�1� α

2

��+Φ

�σ20 � σ2

σ2/pN+

σ20σ2

Φ�1�α

2

��If σ2 < σ20, then:

limN!∞

P�σ2�= 1�Φ (+∞) +Φ (+∞) = 1� 1+ 1 = 1

If σ2 > σ20, then:

limN!∞

P�σ2�= 1�Φ (�∞) +Φ (�∞) = 1� 0+ 0 = 1

The test is consistent. �

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 39 / 88

Solution (cont�d)

The power function is de�ned as to be:

P�σ2�= 1�Φ

�σ20 � σ2

σ2/pN+

σ20σ2

Φ�1�1� α

2

��+Φ

�σ20 � σ2

σ2/pN+

σ20σ2

Φ�1�α

2

��This function reaches a minimum when σ2 tends to σ20.

limσ2!σ20

P�σ2�= 1�Φ

�Φ�1

�1� α

2

��+Φ

�Φ�1

�α

2

��= 1�

�1� α

2

�+

α

2= α

The test is unbiased. �

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 40 / 88

Subsection 4.2

The trilogy: LRT, Wald, and LM tests

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 41 / 88

Problem (Greene, 2007, page 531)We consider two random variables Y and X such that the pdf of theconditional distribution Y jX = x is given by

fY jX (y j x ; β) =1

β+ xexp

�� y

β+ x

�For convenience, let

βi =1

β+ xi

This exponential density is a restricted form of a more general gammadistribution,

fY jX (y j x ; β, ρ) =β

ρi

Γ (ρ)y ρ�1i exp (�yi βi )

The restriction is ρ = 1. We want to test the hypothesis

H0 : ρ = 1 versus H1 : ρ 6= 1Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 42 / 88

Reminder: the gamma function

The gamma function Γ (p) is de�ned as to be:

Γ (p) =∞Z0

tp�1 exp (�t) dt 8p > 0

The gamma function obeys the recursion

Γ (p) = (p � 1) Γ (p � 1)

Γ�12

�=p

π

So for integer values of p, we have

Γ (p) = (p � 1)!

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 43 / 88

Reminder: the gamma function (cont�d)

The derivatives of the gamma function are

∂kΓ (p)∂pk

=

∞Z0

(ln (t))k tp�1 exp (�t) dt

The �rst two derivatives of ln (Γ (p)) are denoted

∂ ln (Γ (p))∂p

=Γ0

Γ= Ψ (p)

∂2 ln (Γ (p))∂p2

=

�ΓΓ" � Γ

02�

Γ2= Ψ0 (p)

where Ψ (p) and Ψ0 (p) are the digamma and trigamma functions (seepolygamma function and function psy in Matlab).

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 44 / 88

Problem (cont�d)

Question 1: consider an i .i .d . sample fXi ,YigNi=1 and write itslog-likelihood under H1 (unconstrained model) and under H0 (constrainedmodel).

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 45 / 88

Solution

Under H1, with θ = (β : ρ)> , we have

fYi jXi (yi j xi ; θ) =β

ρi

Γ (ρ)y ρ�1i exp (�yi βi ) with βi =

1β+ xi

`N (y j x ; θ) =N

∑i=1ln fYi jXi (yi j xi ; θ)

The log-likelihood under H1 (unconstrained model) is:

`N (y j x ; θ) = ρN

∑i=1ln (βi )�N ln (Γ (ρ)) + (ρ� 1)

N

∑i=1ln (yi )�

N

∑i=1yi βi

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 46 / 88

Solution (cont�d)

Under H0 : ρ = 1, we have

fYi jXi (yi j xi ; β) = βi exp (�yi βi ) with βi =1

β+ xi

`N (y j x ; β) =N

∑i=1ln fYi jXi (yi j xi ; β)

The log-likelihood under H0 (constrained model) is:

`N (y j x ; β) =N

∑i=1ln (βi )�

N

∑i=1yi βi �

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 47 / 88

Problem (cont�d)Question 2: write the gradient vectors and the Hessian matricesassociated to the unconstrained log-likelihood (under H1) and to theconstrained log-likelihood (under H0).

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 48 / 88

Solution

Under H1 :

`N (y j x ; θ) = ρN

∑i=1ln (βi )�N ln (Γ (ρ)) + (ρ� 1)

N

∑i=1ln (yi )�

N

∑i=1yi βi

Remarks:

∂βi∂β

=∂ (1/ (β+ xi ))

∂β= � 1

(β+ xi )2 = �β2i

∂ ln (βi )∂β

=∂ (� ln (β+ xi ))

∂β= � 1

β+ xi= �βi

∂ ln (Γ (ρ))∂ρ

= Ψ (ρ)

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 49 / 88

Solution (cont�d)

`N (y j x ; θ) = ρN

∑i=1ln (βi )�N ln (Γ (ρ)) + (ρ� 1)

N

∑i=1ln (yi )�

N

∑i=1yi βi

The gradient vector under H1 is:

gN (y j x ; θ) =∂`N (y j x ; θ)

∂θ=

0@ ∂`N ( y jx ;θ)∂β

∂`N ( y jx ;θ)∂ρ

1Awith

∂`N (y j x ; θ)∂β

= �ρN

∑i=1

βi +N

∑i=1yi β

2i

∂`N (y j x ; θ)∂ρ

=N

∑i=1ln (βi )�NΨ (ρ) +

N

∑i=1ln (yi )

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 50 / 88

Solution (cont�d)

∂`N (y j x ; θ)∂β

= �ρN

∑i=1

βi +N

∑i=1yi β

2i

So, we have:∂2`N (y j x ; θ)

∂β2= ρ

N

∑i=1

β2i � 2N

∑i=1yi β

3i

∂2`N (y j x ; θ)∂β∂ρ

= �N

∑i=1

βi

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 51 / 88

Solution (cont�d)

∂`N (y j x ; θ)∂ρ

=N

∑i=1ln (βi )�NΨ (ρ) +

N

∑i=1ln (yi )

So, we have∂2`N (y j x ; θ)

∂ρ2= �NΨ0 (ρ)

∂2`N (y j x ; θ)∂ρ∂β

= �N

∑i=1

βi

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 52 / 88

Solution (cont�d)

The Hessian matrix associated to the log-likelihood under H1 is:

HN (y j x ; θ) =∂2`N (y j x ; θ)

∂θ∂θ>=

0B@ ∂2`N ( y jx ;θ)∂β2

∂2`N ( y jx ;θ)∂β∂ρ

∂2`N ( y jx ;θ)∂ρ∂β

∂2`N ( y jx ;θ)∂ρ2

1CAwith

HN (y j x ; θ) =

ρ ∑Ni=1 β2i � 2∑N

i=1 yi β3i �∑N

i=1 βi

�∑Ni=1 βi �NΨ0 (ρ)

!

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 53 / 88

Solution (cont�d)

Under H0 : ρ = 1, the gradient (scalar) is

gN (y j x ; β) =∂`N (y j x ; β)

∂β= �

N

∑i=1

βi +N

∑i=1yi β

2i

The Hessian (scalar) is:

HN (y j x ; β) =∂2`N (y j x ; β)

∂β2= ∑N

i=1 β2i � 2∑Ni=1 yi β

3i �

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 54 / 88

Problem (cont�d)Question 3: write the average Fisher information matrices under H1 andunder H0.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 55 / 88

Solution

Under H1 (unconstrained model), the Hessian (stochastic) is

Hi (Yi j xi ; θ) =

ρβ2i � 2Yi β3i �βi

�βi �Ψ0 (ρ)

!

The average Fisher information matrice can be de�ned (one of the threede�nitions) as:

I (θ) = EXEθ (�Hi (Yi j xi ; θ))

I (θ) = EX

�ρβ2i + 2Eθ (Yi ) β3i βi

βi Ψ0 (ρ)

!since βi = 1/ (β+ Xi ) depends on the random variable Xi .

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 56 / 88

Solution

I (θ) = EX

�ρβ2i + 2Eθ (Yi ) β3i βi

βi Ψ0 (ρ)

!Consider the score of the unit i . By de�nition, we have:

Eθ (si (Yi j xi ; θ)) =

�ρβi + Yi β2i

ln (βi )�Ψ (ρ) + ln (Yi )

!= 02�1

So, we haveEθ (Yi ) =

ρ

βi

where Eθ denotes the expectation with respect to the conditionaldistribution of Y given X = x .

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 57 / 88

Solution (cont�d)

I (θ) = EX

�ρβ2i + 2Eθ (Yi ) β3i βi

βi Ψ0 (ρ)

!

Eθ (Yi ) =ρ

βi

Under H1 (unconstrained model), the average Fisher information is de�nedas to be:

I (θ) = EX

ρβ2i βi

βi Ψ0 (ρ)

!

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 58 / 88

Solution (cont�d)

Under H0 (constrained model), we have:

Hi (Yi j xi ; β) = β2i � 2Yi β3i

Eβ (si (Yi j xi ; β)) = Eβ

��βi + Yi β

2i

�= 0

The average Fisher information number is de�ned by:

I (β) = EXEβ (�Hi (Yi j xi ; β))= EX

��β2i + 2Eβ (Yi ) β3i

�= EX

�β2i�

The average Fisher information number is equal to:

I (β) = EX�

β2i�

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 59 / 88

Problem (cont�d)

Question 4: denote bθH1 the ML estimator of θ = (β : ρ)> obtained underH1 and bθH0 = bβH0 the ML estimator of β obtained under H0 : ρ = 1.Determine the asymptotic distribution and the asymptotic variancecovariance matrix of bθH1 and bθH0 .

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 60 / 88

Solution

The regularity conditions hold. Under H1 (unconstrainded model) we have:

pN�bθH1 � θ1

�d! N

�0, I�1 (θ1)

�where θ1 denotes the true value of the parameters (under H1), orequivalently: bθH1 asy�H1 N

�θ1,

1NI�1 (θ1)

�with

I (θ1) = EX

ρβ2i βi

βi Ψ0 (ρ)

!

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 61 / 88

Solution (cont�d)

The regularity conditions hold. Under H0 (constrainded model) we have:

pN�bθH0 � θ0

�d! N

�0, I�1 (θ0)

�where θ0 = β0 denotes the true value of the parameter (under H0), orequivalently: bθH0 asy�H0 N

�θ0,

1NI�1 (θ0)

�with

I (θ0) = EX�

β2i�

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 62 / 88

Problem (cont�d)Question 5: Propose three alternative estimators of the average Fisherinformation matrices under H1 and under H0.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 63 / 88

Solution

Three alternative estimators of the average Fisher information matrix I (θ)can be used: bIA �bθ� = 1

N

N

∑i=1

bI i �bθ�bIB �bθ� = 1

N

N

∑i=1

∂`i (θ; yi j xi )

∂θ

����bθ ∂`i (θ; yi j xi )∂θ

����>bθ!

bI c �bθ� = 1N

N

∑i=1

�� ∂2`i (θ; yi j xi )

∂θ∂θ>

����bθ�

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 64 / 88

Solution (cont�d)

First estimator: actual Fisher information matrix

bIA �bθ� = 1N

N

∑i=1

bI i �bθ�Under H1:

bIA �bθ� = 1N

0@ bρ ∑Ni=1bβ2i ∑N

i=1bβi

∑Ni=1bβi NΨ0 (bρ)

1AUnder H0: bIA �bθ� = 1

N

N

∑i=1

bI i �bθ� = 1N

∑Ni=1bβ2i

where bβi = 1/�bβ+ xi� and where the estimators bβ and bρ are obtained

under H1 (unconstrained model) or H0 (constrained model) given the case.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 65 / 88

Solution (cont�d)

Second estimator: BHHH estimator

bIB �bθ� = 1N

N

∑i=1

∂`i (θ; yi j xi )

∂θ

����bθ ∂`i (θ; yi j xi )∂θ

����>bθ!

Under H1:

bIB �bθ� =1N

N

∑i=1

0@ �bρbβi + yibβ2iln�bβi��Ψ (bρ) + ln (yi )

1A���bρbβi + yibβ2i ln

�bβi��Ψ (bρ) + ln (yi ) �Under H0: bIB �bθ� = 1

N

N

∑i=1

��bβi + yibβ2i �2

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 66 / 88

Solution (cont�d)

Third estimator: Hessian

bIC �bθ� = 1N

N

∑i=1

�� ∂2`i (θ; yi j xi )

∂θ∂θ>

����bθ�

Under H1:

bIC �bθ� = 1N

N

∑i=1

0@ �bρbβ2i + 2yibβ3i bβibβi Ψ0 (bρ)1A

Under H0: bIC �bθ� = 1N

N

∑i=1

��bβ2i + 2yibβ3i � �

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 67 / 88

Problem (cont�d)

Question 6: Consider the dataset provided by Greene (2007) in the �leChapter4_Exercise2.xls. Write a Matlab code (1) to estimate theparameters of model under H1 (unconstrained model) by MLE, and (2)to compute three alternative estimates of the asymptotic variancecovariance matrix.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 68 / 88

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 69 / 88

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 70 / 88

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 71 / 88

Remarks

1 Asymptotically the three estimators of the asymptotic variancecovariance matrix are equivalent.

2 But, this exercise con�rms that these estimators can give verydi¤erent results for small samples

3 The striking di¤erence of the BHHH estimator is typical of its erraticperformance in small samples

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 72 / 88

Problem (cont�d)

Question 7: Write a Matlab code (1) to estimate the parameters ofmodel under H0 (constrained model) by MLE, and (2) to compute threealternative estimates of the asymptotic variance covariance matrix.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 73 / 88

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 74 / 88

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 75 / 88

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 76 / 88

Problem (cont�d)Question 8: test the hypothesis

H0 : ρ = 1 versus H1 : ρ 6= 1

with a likelihood ratio (LR) test for a signi�cance level of 5%.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 77 / 88

Solution

The likelihood ratio (LR) test-statistic is de�ned by:

LR = �2��`N�bθH0 ; y j x�� `N �bθH1 ; y j x��

In this sample, we have a realisation equal to

LR (y) = �2� (�88.4363+ 82.9160) = 11.0406

The critical region is

W =�y : LR (y) > χ20.95 (1) = 3.8415

where χ20.95 (1) is the critical value of the chi-squared distribution withp = 1 degrees of freedom. Conclusion: for a signi�cance level of 5%, wereject the null H0 : ρ = 1.�

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 78 / 88

Problem (cont�d)Question 9: test the hypothesis

H0 : ρ = 1 versus H1 : ρ 6= 1

with a Wald test for a signi�cance level of 5%.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 79 / 88

Solution

The null hypothesis H0 : ρ = 1 can be expressed as

H0 : c (θ) = 0

with c (θ) = ρ� 1. The Wald test-statistic is de�ned by:

Wald = c�bθH1�> � ∂c

∂θ>

�bθH1� bVasy

�bθH1� ∂c

∂θ>

�bθH1�>��1 c �bθH1�Here, we have

∂c

∂θ>

�bθH1� = � 0 1�

Then, we get

Wald (y) =�bρH1 � 1�2 �V�1

asy

�bρH1�

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 80 / 88

Solution (cont�d)

Wald (y) =�bρH1 � 1�2 �V�1

asy

�bρH1�Given the estimator chosen for the asymptotic variance, we get:

WaldA (y) =(3.1509� 1)2

0.5768= 8.0214

WaldB (y) =(3.1509� 1)2

1.5372= 3.0096

WaldC (y) =(3.1509� 1)2

0.6309= 7.3335

The critical region is

W =�y : Wald (y) > χ20.95 (1) = 3.8415

-

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 81 / 88

Solution (cont�d)

Conclusion:

1 For a signi�cance level of 5%, the Wald test-statistic based on theestimators A and C (actual Fisher matrix and Hessian) of theasymptotic variance covariance matrix lead to reject the nullH0 : ρ = 1.

2 For a signi�cance level of 5%, the Wald test-statistic based on theestimators B (BHHH estimator) of the asymptotic variance covariancematrix fails to reject the null H0 : ρ = 1.

3 In most of software, the Hessian (estimator C) is preferred and theWald test-statistics are computed with this estimator.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 82 / 88

Problem (cont�d)Question 10: test the hypothesis

H0 : ρ = 1 versus H1 : ρ 6= 1

with a Lagrange Multiplier test for a signi�cance level of 5%. Write aMatlab code to compute the three possibles values of the LM test-statistic.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 83 / 88

Solution

The Lagrange multiplier test is based on the restricted estimators. TheLM test-statistic is de�ned by:

LM = sN�bθH0 ; yi j xi�>bIN �bθH0��1 sN �bθH0 ; yi j xi�

or equivalently

LM = sN�bθH0 ; yi j xi�> N bI �bθH0��1 sN �bθH0 ; yi j xi�

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 84 / 88

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 85 / 88

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 86 / 88

Solution (cont�d)

So, given the estimator chosen for bI �bθH0�, we have:LMA (y) = 4.7825

LmB (y) = 15.6868

LMC (y) = 5.1162

The critical region is

W =�y : LM (y) > χ20.95 (1) = 3.8415

Conclusion: for a signi�cance level of 5%, we reject the null H0 : ρ = 1,whatever the choice of the estimator.

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 87 / 88

End of Exercices - Chapter 4

Christophe Hurlin (University of Orléans)

Christophe Hurlin (University of Orléans) Advanced Econometrics - HEC Lausanne December 5, 2013 88 / 88