Verbeek AGuidetoModernEconometrics and Microeconometrics

Basics Heteroskedasticity Endogenous regressors Maximum likelihood Limited dependent variables Panel data

Microeconometrics

Based on the textbooksVerbeek: A Guide to Modern Econometrics

and Cameron and Trivedi: Microeconometrics

Robert M. [email protected]

University of Viennaand

Institute for Advanced Studies Vienna

January 28, 2014

Microeconometrics University of Vienna and Institute for Advanced Studies Vienna


Outline

Basics

Heteroskedasticity

Endogenous regressors

Maximum likelihood

Limited dependent variablesBinary choice modelsMultiresponse modelsCount dataTobit modelsDuration models

Panel data











Binary choice models


Linear regression only works as long as it is reasonable to view thedependent variable as distributed Gaussian or similar (unimodal,continuous, support is R). It fails when the dependent variable isbinary (0-1, buys a laptop-does not buy, employed-unemployedetc.).Two suggestions:

◮ P(yi = 1|xi ) = G (xi , β), known G is a link function with itsimage in [0, 1];

◮ y∗i = x ′iβ + ui for the latent (hidden, unobserved) variable y∗,and y = 1 if y∗ exceeds a certain value.

Both suggestions lead to the same models.




Popular link functions

Often, the link function is seen as reflecting ‘utility’ or ‘willingnessto pay’. Thus, assume G : R → [0, 1] to be monotonic, like adistribution function G (x , β) = F (x ′β):

◮ F (w) = Φ(w), the normal cdf, defines the probit model;

◮ F (w) = L(w) = ew

1+ew, the logistic cdf, defines the logit

model;

◮ F (w) =

0, w ≤ 0;w , 0 ≤ w ≤ 1;1, w > 1.

defines the linear probability model.




Density of the logistic distribution

−4 −2 0 2 4

0.00.1

0.20.3

0.4

x

f(x)

The density of the logistic distribution (magenta) is less peakedaround 0 and has heavier tails than the standard normal density(black).




Marginal effects

In linear regression, βk reflect the marginal effect of a change inxk . In binary choice models, this issue is more complex, as themarginal effects (φ denotes the normal density)

∂Φ(x ′iβ)

∂xik= φ(x ′iβ)βk ;

∂L(x ′i β)

∂xik=

exp(x ′iβ)

{1 + exp(x ′iβ)}2βk ,

depend on the values of the covariates. They are often computedat sample averages.




The odds ratio

For pi = P(yi = 1|xi ), the term

logpi

1− pi

is the log odds ratio. At the point of undecidedness, the log oddsratio is 0. In the logit model, it is a linear function of thecovariates:

logpi

1− pi= x ′iβ,

such that βk is the marginal reaction of the log odds ratio to achange in xk .




Underlying latent model

Consider the second interpretation

y∗i = x ′iβ + εi

with yi = 1 if y∗i > 0 and yi = 0 otherwise. y∗ is an unobservedlatent variable (imagine utility etc.). Note

P(yi = 1) = P(y∗i > 0) = P(−εi ≤ x ′iβ) = F (x ′iβ),

with F the distribution function of the errors ε. Normal (logistic)distribution for ε implies the probit (logit) model.




The likelihood of a binary choice model

The standard logit/probit model is fully parametric. The likelihoodis

L(β; y ,X ) =

N∏

i=1

P(yi = 1|xi ;β)yiP(yi = 0|xi ;β)1−yi ,

which yields for the log-likelihood

logL(β; y ,X ) =

N∑

i=1

yi log F (x′iβ) +

N∑

i=1

(1− yi ) log(1− F (x ′iβ)).




The maximum-likelihood estimator

Maximizing the likelihood L in β yields the ML estimator β. Thereis no closed form. Because of continuity, the ML estimator can beobtained numerically by solving

∂ logL(β)∂β

=

N∑

i=1

yi − F (x ′iβ)

F (x ′iβ)(1 − F (x ′iβ))f (x ′iβ)xi = 0,

with f = F ′, the pdf of the distribution.




Interpretation of ML as an orthogonality condition

ConsiderN∑

i=1

yi − F (x ′i β)

F (x ′i β)(1 − F (x ′i β))f (x ′i β)xi = 0,

i.e. the columns of the covariates matrix X are orthogonal toεGi , i = 1, . . . ,N, the generalized residuals. Note that

εGi =

f (x ′iβ)

F (x ′iβ), yi = 1,

−f (x ′iβ)

1−F (x ′iβ), yi = 0.

This orthogonality corresponds to the orthogonality of regressorsand residuals in OLS estimation.




A property for logit estimation

If F is the logistic cdf, the generalized residual becomes

εGi = yi −exp(x ′i β)

1 + exp(x ′i β)= yi − pi ,

i.e. the traditional residual. If one of the regressors is a constant,this implies that

N∑

i=1

pi = N1,

with N1 the number of all observations with yi = 1 in the sample.The average of all predicted probabilities equals the share ofyi = 1. This is not exactly true for the probit version, however.




Goodness of fit for binary choice

How should we measure goodness of fit in binary choice models?R2 cannot work. A suggestion is to use the likelihood for theestimated model L1 and for a model with constant (‘intercept’)only L0. For example, the pseudo R2

R2pseudo = 1− logL1

logL0

is often used. Alternatively, one may consider the LR statistic

LR = 2(logL1 − logL0),

which is distributed χ2(K ) under H0 : β1 = . . . = βK = 0.




The classification table

If we accept the implications

x ′i β > 0 ⇒ pi > 0.5 ⇒ yi = 1, x ′i β ≤ 0 ⇒ pi ≤ 0.5 ⇒ yi = 0,

we can summarize correct and incorrect predictions in aclassification table:

yi0 1 total

yi 0 n00 n01 N0

1 n10 n11 N1

total n0 n1 N




An R2 constructed from the classification table

Let njk denote the number of cases where yi = j but yi = k , forj , k = 0, 1. Then,

wr1 =n01 + n10

N

is the proportion of incorrect predictions by the model. A trivialbenchmark would predict all yi to be 1 when N1/N > 0.5 and tobe 0 when N1/N < 0.5 and mis-classify all zeros (wr0 = N0/N)resp. ones (wr0 = N1/N). The goodness-of-fit measure

R2p = 1− wr1

wr0

is sometimes shown with the classification tables.




The ROC curve

If correct prediction of 1 is called sensitivity and correct predictionof 0 is called specificity, their frequencies will change along withthe level of the critical point for p. If the cut-off point is 0,everything is classified as 1. If it is 1, everything is classified as 0.The curve depicting sensitivity and (one minus) specificity for allpossible cut-offs is called receiver operating characteristic curveor ROC curve.

A ROC curve close to the diagonal is seen as a bad ROC curve. AROC curve increasing steeply around the origin and staying closeto horizontal until (1,1) is considered an ideal ROC curve. Someresearchers calculate the area under the ROC curve as agoodness-of-fit criterion.




Specification tests in binary choice models

The quadratic form in LM tests is often interpreted as NR2 from aregression of transformed residuals on covariates. Here, it is easierto view it as NR2 from regressing the constant ‘1’ on covariates:

◮ An LM test for omitted variables can be constructed byregressing 1 on products of generalized residuals with theoriginal covariates and J potential other regressors εGi x

′i and

εGi z′i . The statistic NR2 is distributed χ2(J) under the null;

◮ A heteroskedasticity test regresses 1 on εGi x′i and on

εGi (x′i β)z

′i , if z summarizes J potential sources for

heteroskedasticity. The statistic NR2 is distributed χ2(J)under the null.

This is simple for logit (rather than probit), as εGi = yi − pi .



Multiresponse models

Ordered and unordered response

If the dependent variable assumes more than two values, a majordistinction is between ordered and unordered response models:

◮ With ordered response, one can imagine a latent variableworking behind the scenes (having two or more computersy = 2 reflects a stronger interest in computers than havingexactly one computer y = 1, which again reflects moreinterest in computers than having no such device y = 0);

◮ With unordered response, there is no such natural ordering(vacations on the beach y = B do not express a strongerinterest in tourism than vacations in the mountains y = M orin a foreign city y = C ).




Ordered logit model

Consider observations on covariates xi and on a dependent variableyi ∈ {1, 2, . . . ,M}. An unobserved variable y∗i depends on thecovariates linearly,

y∗i = x ′iβ + εi ,

and yi assumes specific values according to intervals for theunobserved y∗,

yi = j if γj−1 < y∗i ≤ γj ,

with formally γ0 = −∞ and γM = ∞. If εi follows the logisticdistribution, this model is the ordered logit model.




Ordered probit model

Most technical advantages of logit relative to probit are lost inordered response, so it may be more attractive to assume εi in

y∗i = x ′iβ + εi ,

to be normally distributed. This defines the ordered probit model,with

P(yi = j |xi ) = P(γj−1 < y∗i ≤ γj) = Φ(γj − x ′iβ)− Φ(γj−1 − x ′iβ),

with Φ denoting the cdf of the normal distribution. Variance andintercept are not identified. It is customary to restrict σ = 1 andto set the intercept at 0.




Unordered multiresponse modelsVarious models can be considered:

◮ The conditional logit model applies if common reactioncoefficients β are targeted and covariates on all M alternativesare available (prices for using bicycle, train, car in modal-splitresearch);

◮ The multinomial logit model applies if only individualcharacteristics are available and reaction coefficients βj for allalternatives j = 1, . . . ,M are targeted (modal split dependenton commuting time);

◮ The nested logit model is sometimes preferred withhierarchical decisions (bus or car, and if bus then choicebetween two bus lines);

◮ Multinomial probit models are technically complex and arerarely used.




Conditional logit

Presume individual i chooses alternative j (j = 1, . . . ,M) if thatalternative maximizes her utility. If the utility depends oncovariates linearly in the sense of

Uij = x ′ijβ + εij

and if the distribution of the iid ε is a type I extreme-valuedistribution (standard Gumbel F (x) = exp{− exp(−x)}), one canshow that, after standardizing the first choice option x ′i1β = 0,

P(yi = j) =exp(x ′ijβ)

1 + exp(x ′i2β) + exp(x ′i3β) + . . . + exp(x ′iMβ),

which yields the conditional logit model.




Density of the Gumbel distribution

−4 −2 0 2 4

0.00.1

0.20.3

0.4

x

f(x)

The density of the Gumbel distribution (magenta) is slightlyasymmetric and has extremely light tails even compared to thestandard normal density (black).




Multinomial logit

If all covariates are individual characteristics constant across choiceoptions, coefficient vectors may change along j

(‘alternative-specific’ reaction):

P(yi = j) =exp(x ′iβj )

1 + exp(x ′iβ2) + exp(x ′iβ3) + . . . + exp(x ′iβM)

defines the multinomial logit model. The two models‘multinomial logit’ and ‘conditional logit’ may also be mixed:‘mixed logit’.




Independence of irrelevant alternatives

Multinomial and conditional logit models use iid ε errors acrossalternatives. The choice between j1 and j2 is independent of thepreference for j 6= j1, j2. This may not hold in practice (forexample, the well-known blue-bus versus red-bus: people whoprefer the red bus to the car may also prefer the blue bus). Theassumption is called independence of irrelevant alternatives

(IIA).

An answer to the IIA problem is the nested logit model, whichpermits sequences of hierarchic decisions: bus versus car, and if carthen rental car versus privately owned car.

IIA is sometimes tested by a Hausman test, with the robustestimate represented by separate binary logits and theefficient/inconsistent estimate by multinomial logit.



Count data

Count data: the idea

Count data models concern cases where the dependent variable isinteger and typically too small for the convenient continuousapproximation. Count data differ from multiple response as theyare clearly numeric (such as hospital arrivals, doctor visits, patentsetc.).

The standard model of count data analysis is the Poissonregression, which plays a role similar to the OLS regression forcontinuous variables.



Count data

The Poisson distribution

The Poisson distribution is a statistical distribution on thenon-negative integers that is governed by one parameter λ. It isdefined by

P(x = k) =exp(−λ)λk

k!. k = 0, 1, 2, . . . ,

with the characteristic properties

Ex = λ

andVx = λ.



Count data

Visualization of the Poisson distribution

0 5 10 15 20

0.00

0.05

0.10

0.15

0.20

0.25

x

p(x)

Probability mass function for the Poisson distribution with λ = 2(black) and λ = 3 (red).



Count data

Poisson regression

If the Poisson parameter is assumed to depend linearly on thecovariates via an exponential link function,

P(yi = k |xi ) =exp(− exp(x ′iβ))(exp(x

′iβ))

k

k!, k ∈ N, i = 1, . . . ,N,

this defines the Poisson regression model. Coefficients β can beestimated by maximum likelihood. The statistical properties arereasonably robust to modifications of assumptions, such that ML isoften used in the sense of quasi–ML.

By construction, the marginal reaction ∂E(yi |xi )/∂xik is not βkbut exp(x ′iβ)βk .



Count data

Maximizing the likelihood in Poisson regression

Consider the likelihood for the Poisson regression model,

logL(β) =N∑

i=1

− exp(x ′iβ) + yi (x′iβ)− log yi !,

which, after taking derivatives w.r.t. β and thus forming scores,yields the ‘orthogonality condition’ for generalized residuals εG

N∑

i=1

{yi − exp(x ′i β)}xi =N∑

i=1

εGi xi = 0,

which is reminiscent of the orthogonality condition for OLS.



Count data

Asymptotic variance of the Poisson ML

Under the usual regularity conditions, the asymptotic variancematrix of the Poisson ML estimator is given by

V(βML) = I (β)−1,

with I (β) determined from the product of scores or from theirderivatives. In misspecified models, with quasi–ML interpretation,the formula

V(βML) = I (β)−1J(β)I (β)−1

can be used, with I determined from second derivatives and J

directly from scores. The latter formula is popular, as the Poissonassumption is often violated.



Count data

Overdispersion

The Poisson distribution has its mean equal to its variance, whichis called equidispersion. With actual data, it often occurs that thevariance is larger (overdispersion), rarely that the variance issmaller (underdispersion) than the mean.

Alternative models, such as negative binomial regression, aresometimes used and allow for overdispersion. Unfortunately, thesemodels are sensitive to the negative binomial distributionalassumption, which may not hold either.



Tobit models

Tobit models: the idea

Tobit handles mixed situations, where the dependent variable issometimes a regular continuous variable, with all covariatesobserved, and the relation typically linear, and sometimes different,i.e. 0 although the underlying function value may not be 0 orsimply unobserved, and covariates observed or unobserved.

In the standard tobit model (after Tobin), all covariates areobserved even for the y = 0 cases, and y = 0 can be interpreted asreflecting y∗ < 0 for the latent variable that otherwise coincideswith y .

In the most important extension, Tobit-II or ‘Heckman model’,the main reaction function differs from the equation thatdetermines whether y∗ is observed.



Tobit models

The standard tobit model

A latent variable y∗ is assumed to depend on covariates x linearly,i.e.

y∗i = x ′iβ + εi , i = 1, 2, . . . ,N.

Covariates xi are observed for all i . Errors εi are distributed normalN (0, σ2). The dependent variable y is subject to left censoring

according to

yi =

{

y∗i , ify∗i > 0,0, ify∗i ≤ 0.

A very similar model is obtained if right censoring is assumed.



Tobit models

Expected value of dependent variable in standard tobit

It is derived easily that

E(yi |yi > 0) = x ′iβ + σφ(x ′iβ/σ)

Φ(x ′iβ/σ),

i.e. a regular linear term plus a nonlinear adjustment term. Thisequation implies

E(yi ) = x ′iβΦ(x′iβ/σ) + σφ(x ′i β/σ),

whence∂E(yi )

∂xik= βkΦ(x

′iβ/σ)



Tobit models

Estimating the standard tobit modelIn the standard tobit model, estimation is conducted bymaximizing the likelihood

logL(β, σ2) =∑

i∈I0

logP(yi = 0) +∑

i∈I1

log φ(yi ),

with I0 and I1 defined as the sets of observations with yi = 0 andyi > 0, respectively. Inserting the expressions for normaldistribution, this implies

logL(β, σ2) =∑

i∈I0

log{1− Φ(x ′iβ/σ)}

+∑

i∈I1

log[1√2πσ2

exp{−(yi − x ′iβ)2

2σ2}].



Tobit models

Properties of the tobit ML estimatorThe maximum-likelihood estimator cannot be written in a closedform. For correct specification, it is consistent and asymptoticallyefficient.

Like in other models that generalize regression, there is anorthogonality condition for generalized residuals

N∑

i=1

εGi xi = 0,

where the generalized residuals are defined by

εGi =

yi−x ′i βσ , yi > 0,

− φ(x ′iβ/σ)

1−Φ(x ′iβ/σ)

, yi = 0.



Tobit models

The tobit-II model: the idea

In the standard tobit model, the random sampling assumption isretained. Some observations are censored, but even for these thecovariates are observed. Censoring follows the same functionalform as the main regression relationship.

Tobit-II models generalize these assumptions:

◮ An observation rule that typically differs from the mainregression determines whether the dependent variable isobserved. Often, different covariates appear in this rule;

◮ The dependent variable is interpreted as unobserved ratherthan zero for the irregular cases.

The tobit-II model (according to some sources due to Heckman)is the simplest model that addresses sample selection.



Tobit models

The tobit-II model

Assume a linear theoretical relationship

y∗i = x ′1,iβ1 + ε1,i ,

and an observation rule (selection equation)

h∗i = x ′2,iβ2 + ε2,i ,

with vectors x1 and x2 often sharing some variables. Observedvariables are x1 , x2, and y , with

yi =

{

y∗i (hi = 1) if h∗i > 0,unobserved(hi = 0) if h∗i ≤ 0.

The covariates x1, x2 are observed over the entire sample.



Tobit models

Tobit-II: assumptions on errors

The unobserved errors are assumed to be independent acrossobservations and normally distributed:

(

ε1,iε2,i

)

∼ N((

00

)

,

(

σ21 σ12

σ12 1

))

Note that the variance of ε2 is set at 1, as the sample selectionportion is a probit model with unidentified variance.



Tobit models

An important property for tobit-II

For the expectation of y given that it is observed, it can be shownthat

E(yi |hi = 1) = x ′1,iβ1 + σ12φ(x ′2,iβ2)

Φ(x ′2,iβ2),

i.e. the sum of a regular linear component and a term due tosample selection. The pdf-cdf ratio φ(.)/Φ(.) is generally calledthe inverse Mills ratio. In the present context, the term

λi =φ(x ′2,iβ2)

Φ(x ′2,iβ2)

is called Heckman’s lambda. If error terms across equations areuncorrelated, β1 can be estimated consistently by OLS. Otherwise,OLS is subject to a sample selection bias.



Tobit models

Estimation in the tobit-II model

The tobit-II parameters β1, β2, σ21 , σ12 can be estimated by

maximizing the likelihood. The tobit-II likelihood is essentially acombination of the probit and the tobit likelihoods.

Alternatively, the equation

yi = x ′1,iβ1 + σ12λi + ηi

can be viewed as a regression that needs a preliminary estimate ofHeckman’s lambda λi . The required estimate of β2 is obtained viaprobit estimation of the selection equation. This estimator is calledHeckman’s two-step estimator.



Duration models

Duration models: main concepts

Duration models are used in survival analysis: given the fact that aperson (light bulb) has already survived until T = t, what will bethe probability of death (burning out)? The classical economicapplication is to labor data, where unemployed persons ‘die’ whenthey find an employment.

Apart from the cumulated distribution function F (t) = P(T ≤ t),duration models also use the counterpart, the survivor function

S(t) = 1− F (t) = P(T > t).

The main focus is now on the probability of death, given survivaluntil t.



Duration models

The hazard function

The hazard function describes the local intensity of death givensurvival until t:

λ(t) = limh↓0

P(t ≤ T < t + h|T ≥ t)

h

It follows that

λ(t) =f (t)

1− F (t)=

f (t)

S(t)= −∂ log S(t)

∂t

and that

F (s) = 1− exp

(

−∫ s

0λ(t)dt

)

.

Thus, there is a one-one relationship between hazard and c.d.f.



Duration models

Customary hazard specifications

Constant hazard λ(t) ≡ λ implies

S(t) = exp(−λt), F (t) = 1− exp(−λt),

the exponential distribution. There is no duration dependence:the risk of death does not change with age.

The Weibull distribution corresponds to the specification

λ(t) = γαtα−1

with parameters α > 0, γ > 0. The hazard rate is monotonicallyincreasing (decreasing) for α > 1 (α < 1).



Duration models

Proportional hazards

In the proportional hazards models, the hazard is a product ofthe baseline hazard (e.g., Weibull or exponential-constant) and anexponential term that depends on covariates:

λ(t, xi ) = λ0(t) exp(x′iβ)

Thus, the logarithm of the hazard depends linearly on thecovariates, with its marginal reaction to changes in xk expressed byβk .



Duration models

Lifetimes are often right-censored

Typically, some individuals in the sample are still alive(unemployed), hence their lifetimes are unavailable (rightcensored). We only know that ti > ci for a censoring time point ci .Likelihood estimation must account for this feature.

Estimation is conducted by maximizing the likelihood

logL(θ) =N∑

i=1

di log f (ti |xi ; θ) + (1− di ) log S(ci |xi ; θ),

with the first time accounting for dead (employed) individuals andthe second term accounting for alive (still unemployed) individuals.




Documents

Verbeek AGuidetoModernEconometrics and Microeconometrics