˜Binary Choice !LPM !logit !logistic regresionpioneer.netserv.chula.ac.th/~ppongsa/2945310/IntroEcono14.pdf · (c) Pongsa Pornchaiwiseskul, Faculty of Economics, Chulalongkorn University

(c) Pongsa Pornchaiwiseskul, Faculty of Economics,

Chulalongkorn University

1



2

� Binary Choice!LPM!logit!logistic regresion!probit

� Multiple Choice!Multinomial Logit



3



4



5



6

Question: What determines thechoice selection?

Model to determine the probability of an event under a given condition (value of independent variables)

Pr(choice#j)=Fj(X1,X2,1,X

K)

where X3s are determinants for theprobability.



7

Note that

1)

2) function Fj() must return a value

between 0 and 1

∑ =j

j 1)#Pr(choice



8

Example

JOIN=1 if the observation will join

the government-run health

insurance program

= 0, otherwise



9

JA=1 if the observation will join Plan A= 0, otherwise

JB=1 if the observation will join Plan B= 0, otherwise

JC=1 if the observation will join Plan C= 0, otherwise

Note that JA+JB+JC=1 always.



10

General Structure

Note that

),...,,()1Pr( 21 KXXXFJOIN ==

),...,,(1)0Pr( 21 KXXXFJOIN −==

1),...,,(0 21 ≤≤ KXXXF



11

Define

Assumption of LPM

Linearity of F(.)

Note that there is no error term

KK XXXP βββ +++= L2211

)1Pr( == JOINP



12

Formulation of LPM

E(JOIN)=(1)P+(0)(1-P)=P

==> JOIN=P+v

where v is an error term. E(v)=0

=> OLS is valid but not the best. Why?

)1(2211

---- vXXXJOIN KK ++++= βββ L



13

Note that V(ν) = V(JOIN) but

V(JOIN) =(1-P)2P+(0-P) 2(1-P)= P(1-P)

==> V(ν) is not constant. It depends on the independent variables (X3s)

==> Violation of a CLRM assumption or ν is heteroscedastic



14

Define

where

)1(

1

PPw

−=

)2(**

*

22

*

11

*

---- νβ

ββ

+++

+=

KK X

XXJOIN

L

νν w

KkwXX

wJOINJOIN

kk

=

==

=

*

*

*

,...,1

for



15

Note that

==> OLS is BLUE for Model (2)

1

)1()1(

1

)()(2*

=

−−

=

=

PPPP

VwV νν



16

Estimation of LPM

Step 1 run OLS for unweighted model (1)

==> JOIN = Xββββ

Note that JOIN is the estimate for P

==> OLS is BLUE for Model (2)



17

Step 2 compute the weight

Step 3 compute

Step 4 estimate the weighted model (2)using OLS

)1(

1

JOINJOIN

w

−=

**

2

*

1

*,...,,, KXXXJOIN



18

Step 5 re-compute JOIN using the

new set of ββββ.

Note that LPM does not assure that

or 1),...,,(0

10

21 ≤≤

≤≤

KXXXF

JOIN



19

Correction

If Xββββ<0, set JOIN=0

If Xββββ>1, set JOIN=1



20

P

Xβ

1

1



21

� Less expensive in computer time. No

non-linear equations

� is the effect of X on the

probability. In general, the explanatory

variables should be unitless or are

expressed in percentage

k

kX

Pβ=

∂∂



22

Assumption of Logit

F() is a logistic function

Note that always.KK

Z

XXXZ

eP

βββ +++=+

= −

L2211

1

1

1)(0 ≤≤ ZF

No error term



23

1.0

0.5

Z

Ze−+1

1

0



24

Note that OLS does not applyML Estimation of Logit model

or

Note that Y=JOIN

∑

∏

=

=

−

−−+=

−=

n

i

iiii

n

i

Y

i

Y

i

PYPYL

PPL ii

1

1

)1(

)]1ln()1()ln([lnmax

)1()(max

β

β



25

Note that

First-order conditions

For k=1,1,K

0]1

)1([

]1

[ln

1

1

=+

−−

+=

∂∂

∑

∑

=

=−

−

n

i

Z

iki

n

iZ

Z

iki

k

iZ

i

i

i

e

eYX

e

eYX

L

β

Ze

P+

=−1

11



26

Solving FOC for ML estimates.

Second-order Conditions

yields Variance-covariance matrix of

∑

∑

=

−

=

−+−−

+−=

∂∂∂

n

i

Z

ikiji

n

iZ

Z

ikiji

kj

iZ

i

i

i

e

eYXX

e

eYXX

L

12

12

2

])1(

)1([

])1(

[ln

ββ

β̂



27

Variance-Covariance Matrix for

Note that it is not the estimated VC matrix.Do Z-test or Chi-square test instead of t-

test or F-test on parameters

12

ln)ˆ(

−

∂∂∂

−=kj

LV

βββ

β̂



28

Interpretation

sign of βk==>direction of the effect of X

k

on the probability to JOIN.

kk

Z

kiZ

i

e

e

X

Pββ }{

)1(2

+=+

=∂∂

−

−



29

No R2 for a logit model since there is no

error term.

Define

It is a measure for goodness-of-fit.

JOIN>0.5 ==> predict that JOIN=1

JOIN<0.5 ==> predict that JOIN=0

(n) size samplen predictiocorrect#

=− 2Rpseudo



30

Assumption of Logistic Regression

F(.) is a logistic function but the observation(experiment) for eachgiven set of independent variables (X) will be repeated several times.Only the proportion of JOIN=1 can be observed.



31

From Logit Model

Note that P is the expected proportion of population JOINing given X3s

KK XXXP

Pβββ +++=

−

L22111

ln



32

Define

Ri=observed proportion of observation with the same value of X

ithat JOIN.

Derived Model

iKiKii

i

i XXXR

Rνβββ ++++=

−L

22111

ln

Why? )1(

1)(

iii

iRRN

V−

=ν



33

Define

Estimation

where

***

2

*

1

*

21 iKiiiXXXR Ki νβββ ++++=

)1( iii RRNw −=

iii

kiiki

i

ii

w

KkXwX

R

RwR i

νν =

==

−=

*

*

*

,...,1

1ln

for



34

=> OLS is BLUE

Interpretation of the parameters same as those for logit model as the

underlying function is also logistic



35

Assumption of Probit

F() is a cumulative distribution function of a standard normal.

Note that always.

KK XXXZ

ZP

βββ +++=

Φ=

L2211

)(

1)(0 ≤Φ≤ Z

No error term



36

1.0

0.5

Z

Ze−+1

1

0

)(ZΦ



37

Assumption of Multinomial Logit

Define PAi= Pr(JA

i=1)

PBi= Pr(JB

i=1)

PCi= Pr(JC

i=1)

Choose the choice of plan C as the reference.



38

iZA

i

i ePC

PA=

iZB

i

i ePC

PB=

where

where

KiKiii XXXZA ααα +++= L2211

KiKiii XXXZB βββ +++= L2211



39

ii

ii

ii

ZBZAi

ZBZA

i

ii

ZBZA

i

ii

eePC

eePC

PBPA

eePC

PBPA

++=

++=+

+

+=+

1

1

11



40

ii

i

ii

i

ZBZA

ZB

i

ZBZA

ZA

i

ee

ePB

ee

ePA

++=

++=

1

1



41

ML Estimation of Multinomial Logit model

Solving FOC yields

)]1ln()1(

)ln()ln([lnmax

)1()()(max

1

1

)1(

iiii

n

i

iiii

n

i

JBJA

ii

JB

i

JA

i

PBPAJBJA

PBJBPAJAL

PBPAPBPAL iiii

−−−−+

+=

−−=

∑

∏

=

=

−−

or β

β

β,α ˆˆ



42

Interpretation

kZAZA

ZA

ki

i

ii

i

ee

e

X

PAα

++=

∂∂

1

( )k

ZB

k

ZA

ZAZA

ZA

ii

ii

i

eeee

eβα +

++−

2)1(

kZAZA

ZA

ZAZA

ZA

ii

i

ii

i

ee

e

ee

eα

++−

++=

11

1

kZAZA

ZB

ZAZA

ZA

ii

i

ii

i

ee

e

ee

eβ

++++−

11



43

Interpretation

own-effect cross-effect

sign of αk==>direction of the own-effect

of Xkon the probability to JOIN A.

sign of βk==>direction of the cross-effect

of Xkon the probability to JOIN A.

kiikii

ki

i PBPAPAPAX

PAβα −−=

∂∂

)1(

+ +



44

� Nested Logit /Serial Logit

� Ordered Logit

� Generalized Extreme-Value (GEV)



45

Models for Limited Dependent

Varaibles

� Censored Regression

� Tobit Models

Documents

˜Binary Choice !LPM !logit !logistic regresionpioneer.netserv.chula.ac.th/~ppongsa/2945310/IntroEcono14.pdf · (c) Pongsa Pornchaiwiseskul, Faculty of Economics, Chulalongkorn University