Supremecourt

Reproduce the paper “Dynamic Ideal Point Estimationvia Markov Chain Monte Carlo for the US Supreme

Court 1953-1999” by Andrew Martin and Kevin Quinn

Wei Wang

April 14, 2011

Wei Wang () Reproduce the paper “Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the US Supreme Court 1953-1999” by Andrew Martin and Kevin QuinnApril 14, 2011 1 / 20

The same graph

Figure: Ideology of Justices


The Secret Inner World of Supreme Court Justices

For decades, political scientists and legal scholars have been trying tomodel the latent preferences of Supreme Court Justices.

The most widely-used model is the so-called Item Response Model

The main contribution of Martin and Quinn’s paper is to propose adynamic model to capture the temporal evolutions of the ideal points.

Let’s first go through the Static Item Response Model.


The Secret Inner World of Supreme Court Justices

For decades, political scientists and legal scholars have been trying tomodel the latent preferences of Supreme Court Justices.

The most widely-used model is the so-called Item Response Model

The main contribution of Martin and Quinn’s paper is to propose adynamic model to capture the temporal evolutions of the ideal points.

Let’s first go through the Static Item Response Model.


Ideal Point and Utility Specification

Let K denote all the cases, and Jk denotes the set of justices on thecase k. θj is the uni-dimensional ideal point position for justice j,while x(r)k is the location of the case k under an affirmative vote and

x(a)k is the location of the case k under a resersal.

We define utility of resersing case k for justice j to be

u(r)k,j = −|θj − x

(r)k |2 + ξ

(r)k,j

where ξ(r)k,j is some random disturbance.

Similarly we can define u(a)k,j .


Ideal Point and Utility Specification

Follwoing a pure attitudinal manner, we assume the justice makedecision entirely based on her utilities under status quo and thealternative scenario.

zk,j = u(r)k,j − u

(a)k,j

= −∣∣θj − x(r)k

∣∣2 + ξ(r)k,j +∣∣θj − x(a)k

∣∣2 − ξ(a)k,j

=(x(a)k x

(a)k − x

(r)k x

(r)k

)+ 2

(x(r)k − x

(a)k

)θj +

(ξ(r)k,j − ξ

(a)k,j

)= αk + βkθj + εk,j


Item Response Model

Formally, the model is given by

vk,j =

{1 if zk,j > 0

0 if zk,j 6 0

wherezk,j = αk + βkθj + εk,j, εk,j

i.i.d∼ N(0, 1)

And because we are Bayesian, we also need priors(αk

βk

)∼ N(b0,B0), θj ∼ N(0, τ2)

Actually, the great thing is that we don’t need to give aconservative/liberal rating on each case.


Item Response Model

Formally, the model is given by

vk,j =

{1 if zk,j > 0

0 if zk,j 6 0

wherezk,j = αk + βkθj + εk,j, εk,j

i.i.d∼ N(0, 1)

And because we are Bayesian, we also need priors(αk

βk

)∼ N(b0,B0), θj ∼ N(0, τ2)

Actually, the great thing is that we don’t need to give aconservative/liberal rating on each case.


Inference

The inference of this static model is standard using Gibbs Sampler1 Sampling the Latent Utilities zk,j, given vk,j,αk,βk, θj2 Sampling the Case Parameteres αk,βk, given zk,j, θj3 Sampling the Item Response θj, given zk,j,αk,βk


From Static to Dynamic

Apparently, static model works, but is not particularly attractive, sincedata from Supreme Court each year are very small in size by natrueand most researchers are dealing with longitudinal data spanningacross several decades.

There is compelling common-sensical evidence to believe thatideologies of the Justices do change over years. (Justice Stevens)

So how to jump from the Static World to the Dynamic World?













So Martin and Quinn added subscript t into the model, whichbecomes

vt,k,j =

{1 if zt,k,j > 0

0 if zt,k,j 6 0

wherezt,k,j = αk + βkθt,j + εt,k,j, εt,k,j

i.i.d∼ N(0, 1)

And of course, they did more than just that.

The ideal points θt,j are modeled as a random walk.

θt,j = θt−1,j + δt,j, δt,ji.i.d∼ N(0,∆t,j)



So Martin and Quinn added subscript t into the model, whichbecomes

vt,k,j =

{1 if zt,k,j > 0

0 if zt,k,j 6 0

wherezt,k,j = αk + βkθt,j + εt,k,j, εt,k,j

i.i.d∼ N(0, 1)

And of course, they did more than just that.

The ideal points θt,j are modeled as a random walk.

θt,j = θt−1,j + δt,j, δt,ji.i.d∼ N(0,∆t,j)


Inference

The first two steps of Gibbs Sampler are still the same.

For the sampling of the Ideal Points given all the Latent Utilities andCase Parameters, the model can be rewritten as

zt,k,j − αk = βkθt,j + εt,k,j

θt,j = θt−1,j + δt,j

This is a standard state-space process and Martin and Quinnborrowed results from Baysian Dynamic Linear Model literature andapplied a “Forward Filtering Backward Sampling” approach to solvethe sampling problem.


So we have...

time

Ideo

logy

−6−4−2

024

−6−4−2

024

−6−4−2

024

−6−4−2

024

Black

Douglas

Oconnor

Stevens

1960 1970 1980 1990

Blackmun

Ginsberg

Powell

Stwart

1960 1970 1980 1990

Brennan

Harlan

Rehnquist

Thomas

1960 1970 1980 1990

Burger

Kennedy

Scalia

Warren

1960 1970 1980 1990

Clark

Marshall

Souter

White

1960 1970 1980 1990

Figure: Reproducation of Figure 1 from the target paper, results from 1953-1999


Notes on Reproducibility

The authors maintain a pretty good standard of reproducibility. Theyprovided the exact C++ code and data sets to replicate the graph intheir paper.

But I ran into an unexpected problem. The C++ code is 10 years oldand its compilation depends on an ancient version of GCC(2.x.x),which is incompatible with every modern computer...

Fortunately, they incorporated their model into R package MCMCpack

with a function named MCMCdynamicIRT1d(). Afer massaging thedata set from their website, and running the slow R function for 8hours, I reproduced the evolution graph in the original paper.

Reproducibility of code is perhaps a trickier issue than we thought.


Assumptions open to challenge

Several assumptions can be questioned or improved1 Uni-dimensional ideal point space2 t distributed error terms for more robustness

However, it might involve substantially more complicated inferencialprocess, if not intractable.

In fact, based on the current model there might be one moreinteresting question to look at.


Assumptions open to challenge

Several assumptions can be questioned or improved1 Uni-dimensional ideal point space2 t distributed error terms for more robustness

However, it might involve substantially more complicated inferencialprocess, if not intractable.

In fact, based on the current model there might be one moreinteresting question to look at.


Certiorari Granting Process

Each year, US Supreme Court receives thousands of petitions forcertiorari, but only less than 100 certs are granted each year.

From Wikipedia:During the Justices’ regular conference, the Justicesdiscuss the petitions, and grant certiorari in less than fivepercent of the cases filed...... Before each conference, theChief Justice prepares a list of those petitions he believeshave sufficient merit to warrant discussion. Any otherJustice may also add a case to the ”discuss list”; cases notdesignated for discussion by any Justice are automaticallydenied review.



Each year, US Supreme Court receives thousands of petitions forcertiorari, but only less than 100 certs are granted each year.

From Wikipedia:During the Justices’ regular conference, the Justicesdiscuss the petitions, and grant certiorari in less than fivepercent of the cases filed...... Before each conference, theChief Justice prepares a list of those petitions he believeshave sufficient merit to warrant discussion. Any otherJustice may also add a case to the ”discuss list”; cases notdesignated for discussion by any Justice are automaticallydenied review.



From Wikipedia:The votes of four Justices at Conference will suffice to grantcertiorari and place the case on the court’s calendar.The grant or denial of certiorari petitions by the Court areusually issued as one-sentence orders without explanation.


Selection biased, surely

Also, historically, more than 60% of cases considered by SupremeCourt are reversed, and in the last decades, the rate is around mid70%.

It is obvious that Justices use the selection process to choose thecases with which they can make their judicial claims.


Selection biased, surely

Also, historically, more than 60% of cases considered by SupremeCourt are reversed, and in the last decades, the rate is around mid70%.

It is obvious that Justices use the selection process to choose thecases with which they can make their judicial claims.


A Possible Pattern

Then it is tempting to assume that in a relatively liberal court, moreconservative cases might be selected so that their reversals canserve for liberal causes, and vice versa.

Rememeber the “liberal/conservative”ness of each case is measuredby case parameters βk.

We might want to look at correlation between average βk with medianJustice ideology over the years.


time

Ideo

logy

Mea

sure

−0.8

−0.6

−0.4

−0.2

0.0

0.2

0.4

0.6

1960 1970 1980 1990

Figure: Beta vs. Mean of Justices Ideology


time

Ideo

logy

Mea

sure

−0.5

0.0

0.5

1.0

1960 1970 1980 1990

Figure: Beta vs. Median of Justices Ideology


beta.by.year

mea

n.ju

stic

e.by

.yea

r

−0.8

−0.6

−0.4

−0.2

0.0

0.2

0.4

0.6

●●

●●

●

● ●

●●

●

●

●

● ●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●●

●

●

●

● ●●

●

●

●

●

●

●

●●

●● ●

−0.2 0.0 0.2 0.4beta.by.year

med

ian.

just

ice.

by.y

ear

−0.5

0.0

0.5

1.0

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●●

●

●

●

●

●

● ●●

●

●●

●

●

−0.2 0.0 0.2 0.4

Figure: Scatter Plots


Documents

Supremecourt