Lecture 2: Linear random effects and theHausman test

Daniel le MaireDepartment of EconomicsUniversity of Copenhagen

September 2, 2014

Contents

1 Random effects model 1

2 Quasi-demeaning 3

3 The relationship between random effects and fixed effects 5

4 Variance estimation 6

5 The Hausman test 6

1 Random effects model

The basic unobserved effects model written for a randomly drawn cross sectionobservation

yit = xitβ + ci + uit t = 1, 2, ..., T

The random effects model is characterized by the following assumptionsRE.1:(a) E (uit|xi1, ...,xiT , ci) = 0 ⇒ E (ciuit) = 0 and E (xituit) = 0 for t =

1, 2, ..., T .(b) E (ci|xi1, ...,xiT ) = E (ci) = 0.

• RE.1(a) maintains the strict exogeneity assumption with respect to uit.

• RE.1(b) imposes that also ci is uncorrelated with the explanatory vari-ables. This is the feature that distinguishes it from the fixed effects model.

1

Define the unconditional variance matrix of vit : Ω ≡ E (viv′i), vi =

(vi1, vi2, ..., viT ). For being able to estimate the parameters we also need thestandard rank condition, that isRE.2: rank

(xi′Ω−1xi

)= K

RE.3:(a) E (uiu

′i|xi, ci) = σ2uIT

(b) E(c2i |xi

)= σ2c

• RE.3(a) says that uit is not autocorrelated.

• RE.3(b) says that the covariance between ci in any two periods is constant/homoscedastic.

• RE.3 along with RE.1 implies that the variance matrix is independent ofxi, i.e. E (viv

′i|xi) = E (viv

′i).

Thus, RE.1 and RE.3 implies that the covariance matrix for individual i isgiven by the T × T matrix

Ω = E (viv′i) =

σ2c + σ2u σ2c · · · σ2c

σ2c σ2c + σ2u · · ·...

.... . . σ2c

σ2c · · · σ2c σ2c + σ2u

= σ2cjT j′T + σ2uIT (1)

where jT is a T × 1 vector of ones. The covariance matrix is the same for allindividuals because the cross sectional units are sampled randomly.Note that

E (vitvis) = E ((ci + uit) (ci + uis)) = E(c2i)

= σ2c for t 6= s

that is that the joint error term is autocorrelated because of the individual effect.

• This indicates that applying OLS to the pooled sample does not providean effi cient estimate.

• The RE model puts structure on the error components and, thereby, ex-ploits the repeated observations for each cross section to gain effi ciency.

Equation (1) implies that β is not estimated effi ciently by OLS. Instead, itcan be estimated effi ciently by Generalized Least Squares (GLS).Assuming that estimates of σ2u and σ

2c are available so that Ω = σ2cjT j′T +

σ2uIT . The GLS estimator of β is then

βRE =

(∑i

(Xi′ Ω−1Xi

))−1(∑i

(Xi′ Ω−1yi

))(2)

By comparison, if vit = uit so that no individual effect exists, the covariancematrix would be Ω = σ2uIT and βRE from equation (2) would be identical tothe OLS estimator.

2

2 Quasi-demeaning

It turns out that βRE also can be estimated by running OLS on quasi-demeaneddata. To do this define a T × T matrix QT = IT − jT (j′T jT )

−1j′T = IT − PT .

Note that QT is the within transformation matrix. Hence, PT = IT − QT isthe matrix that calculate the within-group mean of a variable.Now, rewrite equation (1)

Ω = σ2cjT j′T + σ2uIT

= σ2cTPT + σ2u (QT + PT )

= σ2uQT +(σ2cT + σ2u

)PT

=σ2u

σ2cT + σ2u

(σ2cT + σ2u

)QT +

(σ2cT + σ2u

)PT

=(σ2cT + σ2u

)( σ2uσ2cT + σ2u

QT + PT

)=

(σ2cT + σ2u

)(ηQT + PT )

where η ≡ σ2uσ2cT+σ

2u. Next, define ST ≡ (ηQT + PT ) and note that S−1T =(

1ηQT + PT

).

We can verify this by showing that STS−1T = IT . This is so since:

• PT is idempotent such that PTPT =(jT(jT′ jT)−1

jT′)(

jT(jT′ jT)−1

jT′)

=

jT(jT′ jT)−1

jT′ = PT .

• Recall that Q3 =

1 0 00 1 00 0 1

− 1

313

13

13

13

13

13

13

13

=

23 − 13 − 13− 13

23 − 13

− 13 − 1323

.QT is idempotent, since pre-multiplying by QT implies subtracting themean of each column from each cell in the particular column and the meanof each column of QT is zero by construction, so in sum QT = QTQT .

• QTPT is a T × T matrix of zeros, since there is no "variation" in PT andsince pre-multiplying by QT implies subtracting the mean of each columnfrom each cell in the particular column.

Hence, STS−1T = QT + PT = IT . Using similar arguments we also have that

3

S−1/2T =

(√1ηQT + PT

). Using this, we have

S−1/2T =

(√1

ηQT + PT

)=

(√1

η(IT −PT ) + PT

)=

(√1

ηIT −

(√1

η−√η

η

)PT

)=

1√η

(IT − (1−√η) PT )

=1

1−(1−√η

) (IT − (1−√η) PT )

= (1− λ)−1

(IT − λPT )

where λ = 1−√η. Next, find

Ω−1/2 =(σ2cT + σ2u

)−1/2S−1/2T

=(σ2cT + σ2u

)−1/2(1− λ)

−1(IT − λPT )

=(σ2cT + σ2u

)−1/2(( σ2uσ2cT + σ2u

)1/2)−1(IT − λPT )

=1

σu(IT − λPT )

=1

σuCT

where CT = (IT − λPT ). Assume that an estimate of λ is available, so that

CT =(IT − λPT

)and insert into equation (2)

βRE =

(∑i

(Xi′ Ω−1Xi

))−1(∑i

(Xi′ Ω−1yi

))

=

(∑i

(Xi′ CT

′CTXi

))−1(∑i

(Xi′ CT

′CTyi

))

=(

X′X)−1

X′y (3)

where Xi is the T ×K matrix of explanatory variables and yi the T × 1 vectorcontaining the dependent variable for individual i. X is now the NT×K matrixof stacked of CTXi while yi is the NT × 1 matrix of stacked CTyi.

• Equation (3) says that all there is to do, is to pre-multiply Xi and yi byCT and estimate by OLS on the pooled sample.

4

• The only thing needed to implement βRE from equation (3) are estimates

of σ2u and σ2c , so that λ = 1−

√n = 1−

√σ2u

σ2u+T σ2ccan be calculated.

• Within and between group residuals can be used to obtain estimates ofσ2u and σ

2c , respectively:

σ2u =1

NT −N −K

(Qy −QXβFE

)′ (Qy −QXβFE

)=

u′uNT −N −K

and

σ2c =1

NT −K

(Py −PXβBE

)′ (Py −PXβBE

)− 1

Tσ2u

where the between group estimator βBE = (X′P′PX)−1

X′P′Py.

• Alternatively, recall that vit = ci+uit. A consistent estimate of σ2v can beobtained from the pooled OLS and knowing one of the other components,that is either σ2c or σ

2u, the complete error structure is known.

3 The relationship between random effects andfixed effects

• The within transformation takes out the levels and leaves no variation inthe level across cross sections but only preserves the time variation withineach cross section.

• The between transformation preserves the levels but takes out the timevariation. The variation in the between-transformed series is thus acrossindividuals.

• The RE estimator exploits both the within and between variation in thedata.

• βRE from equation (3) shows that the RE estimator is obtained by quasi-demeaning the data: Rather than removing the entire level of the series, asthe fixed effects estimator does, it removes a fraction of the time-average.

Consider λ = 1−√η = 1−

√σ2u

σ2u+T σ2c. Notice that

• λ→ 1 for T →∞, and the RE estimator will be close to the FE estimator,since CT → (IT −PT ).

• for λ = 0 the RE estimator is identical to the pooled OLS estimator.

• when σ2u gets large relative to σ2c (for fixed T ) the RE estimator is closeto the pooled OLS estimator.

5

4 Variance estimation

An estimate of the variance-covariance matrix of βRE is obtained by

Avar(βRE

)= σ2u

(X′X

)−1Let vi = yi − xiβRE for i = 1, ..., N . The robust variance estimator is

Avar(βRE

)=

(N∑i=1

xi′Ω−1xi

)−1( N∑i=1

xi′Ω−1 vivi

′ Ω−1xi

)(N∑i=1

xi′Ω−1xi

)−1

5 The Hausman test

We now turn to the test the assumption of the RE model, that E (x′c) = 0(RE.1 (b)), is plausible when tested against the assumption of the FE model,that E (x′c) 6= 0. Assuming E (x′c) = 0 is crucial for consistency of the REestimator.

• Under assumption RE.1-RE.3 the RE and FE estimator are both con-sistent, although the FE estimator is ineffi cient, because it estimates Nintercept terms, i.e. N-2 parameters (the individual intercepts - constantterm - one additional variance estimate) more than the RE model.

• On the other hand if E (x′c) 6= 0 then the FE gives consistent estimates,but the RE estimator does not.

• This can be used to form a χ2-statistic under the null hypothesis thatE (x′c) = 0

The Hausman test statistic is

H =(βFE − βRE

)′ [var

(βFE

)− var

(βRE

)]−1 (βFE − βRE

)∼ χ2 (M)

where βRE is the (M × 1) vector of random effects estimates excluding theparameter estimates for time-invariant variables.

• Note that both models require that E (x′u) = 0 and that the Hausmanstatistic is only valid under the null of RE.1-RE.3.

• If we are only interested in one parameter βj the Hausman test is

H =(βj

FE − βj

RE

)′ [se(βj

FE

)2− se

(βj

RE

)2]− 12a∼ N (0, 1)

6