Adaptive Centering with Random Effects in Studies of Time-Varying Treatments

1

Adaptive Centering with Adaptive Centering with Random Effects in Studies of Random Effects in Studies of Time-Varying TreatmentsTime-Varying Treatments

Stephen W. RaudenbushStephen W. Raudenbush

University of ChicagoUniversity of Chicago

December 11, 2006 December 11, 2006

2

Adaptive Centering with Random Effects in Studies of Time-Varying Treatments

by Stephen W. RaudenbushUniversity of Chicago

Abstract

Of widespread interest in education are observational studies in which children are exposed to interventions as they pass through classrooms and schools. The interventions might include instructional approaches, levels of teacher qualifications, or school organization. As in all observational studies, the non-randomized assignment of treatments poses challenges to valid causal inference. An attractive feature of panel studies with time-varying treatments, however, is that the design makes it possible to remove the influence of unobserved time-invariant confounders in assessing the impact of treatments. The removal of such confounding is typically achieved by including fixed effects of children and/or schools. In this paper, I introduce an alternative procedure: adaptive centering of treatment variables with random effects. I demonstrate how this alternative procedure can be specified to replicate the popular fixed effects approach in any dimension. I then argue that this alternative approach offers a number of important advantages: appropriately incorporating clustering in standard errors, modeling heterogeneity of treatment effects, improved estimation of unit-specific effects, and computational simplicity.

3

ClaimsClaims1.1. Adaptive centering with random effects can replicate the Adaptive centering with random effects can replicate the

fixed effects analysis of time-varying treatments in any fixed effects analysis of time-varying treatments in any dimension of clustering.dimension of clustering.

2.2. Adaptive centering with random effects has several Adaptive centering with random effects has several advantagesadvantages

a.a. Incorporating multiple sources of uncertaintyIncorporating multiple sources of uncertaintyb.b. Modeling heterogeneityModeling heterogeneityc.c. Modeling multi-level treatmentsModeling multi-level treatmentsd.d. Improved estimates of unit-specific effectsImproved estimates of unit-specific effectse.e. Computational simplicity Computational simplicity

4

Table 1. Outcome data for 20 hypothetical kids by 9 teachers nested with 3 schoolsTable 1. Outcome data for 20 hypothetical kids by 9 teachers nested with 3 schools Teacher 1 2 3 4 5 6 7 8 9

x -1 0 1 -1 0 1 -1 0 1

w Child

0 1 -2.4102 2.4628 6.2245

1 2 3.6396 4.1441 11.0898

1 3 2.1827 10.1339 12.3134

0 4 -3170 3.6596 4.8397

0 5 -.0727 1.6280 6.0525

0 6 -2.7852 1.4795 10.0131

0 7 .2350 6.0839 7.5142

0 8 -.8803 3.5167 9.7337

0 9 -1.5147 5.8636 10.2860

0 10 2.6814 7.6954 10.0192

1 11 4.4966 9.5578 11.1152

1 12 4.7195 8.2204 14.6855

1 13 4.3609 12.6474 16.8547

1 14 4.7778 11.9663 18.3998

1 15 8.5264 12.9066 18.6272

1 16 8.6820 11.8265 17.0661

1 17 9.5595 13.8078 16.3071

1 18 5.6075 12.7943 21.075

1 19 8.9094 13.5301 20.049

0 20 6.3465 7.3268 11.5147

5

Table 2 Correlations Table 2 Correlations

ww xx school school idid

child idchild id teacher teacher idid

yy

w = child w = child covariate covariate

-.23-.23 .00.00 .43.43 .06.06 5858

x = teacher x = teacher covariate covariate

-.23-.23 .34.34 -.48-.48 .57.57 .14.14

school id school id .00.00 .34.34 .00.00 .97.97 .67.67

child idchild id .43.43 -.48-.48 .00.00 -.13-.13 5858

teacher idteacher id -.06-.06 .57.57 .97.97 -.13-.13 .62.62

y .58 .14 .67 .58 .62

6

1. “True model”1. “True model”

)1,0(5.0252

)()2(

N

childidschoolidwxy tijkikijtijk ti

Estimates of Fixed EffectsEstimates of Fixed Effects

Predictors β Std. Err. T p

(Constant)(Constant) -.415-.415 .302.302 -1.375-1.375 .175.175

xx 2.1712.171 .200.200 10.86610.866 .000.000

ww 4.7994.799 .278.278 17.29417.294 .000.000

schooled-2schooled-2 3.9703.970 .166.166 23.91223.912 .000.000

child idchild id .539.539 .027.027 20.00120.001

934.ˆ 2

7

Methods of Estimation

• OLS – no control• Child random effects • Child fixed effects: • Child random effects, within-child centering• Child and school random effects• Child and school fixed effects• Child and school random effects, two-way

centering– Without teacher random effects– With teacher random effects*

8

OLS : No ControlOLS : No Control

),0(~, 2 Nxy tijktijkjtijk ti


PredictorPredictor ββ Std. Err.Std. Err. T T p p

(Constant)(Constant) 7.9637.963 .748.748 10.63810.638 .000.000

X 1.001 .966 1.036 .305

37.33ˆ 2

9

Child random effects “as if randomized”Child random effects “as if randomized”

),0(~

),,0(~,2

2

Nu

Nuxy

i

tijktijkijtijk ti


ParameterParameter EstimateEstimate Std. Err.Std. Err. dfdf tt Sig.Sig.

InterceptIntercept 7.7374437.737443 1.2745011.274501 16.06716.067 6.0716.071 .000.000

xx 4.3815804.381580 .822129.822129 47.76847.768 5.3305.330 .000.000

ParameterParameter EstimateEstimate

σσ22 12.98512212.985122

ττ2228.09857828.098578

10

One-Dimensional Control: OLS Fixed Child EffectsOne-Dimensional Control: OLS Fixed Child Effects

fixedifor u

N

uxy

i

tijk

tijkijtijk ti

19,...,1

),,0(~ 2

ParameterParameter EstimateEstimate Std. ErrorStd. Error tt Sig.Sig.

InterceptIntercept 13.89408713.894087 2.2170452.217045 6.2676.267 .000.000

xx 5.4980955.498095 .865904.865904 6.3506.350 .000.000

[childid=1.00][childid=1.00] --17.29984117.299841 3.3660293.366029 -5.140-5.140 .000.000

[childid=2.00][childid=2.00] --11.26835311.268353 3.2270333.227033 -3.492-3.492 .001.001

[childid=3.00][childid=3.00] -9.349477-9.349477 3.2270333.227033 -2.897-2.897 .006.006

[childid=4.00][childid=4.00] --14.83204514.832045 3.2270333.227033 -4.596-4.596 .000.000

[childid=5.00][childid=5.00] --11.35816911.358169 3.0134343.013434 -3.769-3.769 .001.001

[childid=6.00][childid=6.00] --12.82553812.825538 3.1086903.108690 -4.126-4.126 .000.000

[childid=7.00][childid=7.00] --11.11573211.115732 3.1086903.108690 -3.576-3.576 .001.001

[childid=8.00][childid=8.00] -9.770723-9.770723 3.0134343.013434 -3.242-3.242 .002.002

[childid=9.00][childid=9.00] -9.015820-9.015820 3.0134343.013434 -2.992-2.992 .005.005

[childid=10.00][childid=10.00] --12.59349112.593491 3.3660293.366029 -3.741-3.741 .001.001

[childid=11.00][childid=11.00] -.006149-.006149 2.8863462.886346 -.002-.002 .998.998

[childid=12.00][childid=12.00] -1.020260-1.020260 2.9007422.900742 -.352-.352 .727.727

[childid=13.00][childid=13.00] -.773729-.773729 2.9435072.943507 -.263-.263 .794.794

[childid=14.00][childid=14.00] -2.179455-2.179455 3.0134343.013434 -.723-.723 .474.474

[childid=15.00][childid=15.00] -2.373398-2.373398 3.1086903.108690 -.763-.763 .450.450

[childid=16.00][childid=16.00] .463474.463474 2.9435072.943507 .157.157 .876.876

[childid=17.00][childid=17.00] -.669300-.669300 3.0134343.013434 -.222-.222 .825.825

[childid=18.00][childid=18.00] 1.0975821.097582 2.9435072.943507 .373.373 .711.711

[childid=19.00][childid=19.00] .268870.268870 3.0134343.013434 .089.089 .929.929

[childid=20.00][childid=20.00] 0(a)0(a) 00 .. ..

Estimates of Covariance Estimates of Covariance ParametersParameters

Parameter Estimate

σ2 12.496491

11

One-Dimensional Control:One-Dimensional Control:Child random effects with person-mean centered xChild random effects with person-mean centered x

Note this gives the same coefficient, standard error, and residual Note this gives the same coefficient, standard error, and residual variance estimate as the student fixed effects model. variance estimate as the student fixed effects model.

ParameterParameter EstimateEstimate Std. Err.Std. Err. dfdf tt Sig.Sig.

InterceptIntercept 8.0295498.029549 .927088.927088 1919 8.6618.661 .000.000

5.4980955.498095 .865904.865904 3939 6.3506.350 .000.000

Estimates of Fixed Effects Estimates of Fixed Effects


σσ 22 12.49649112.496491

ττ 22 13.02435313.024353

),0(~),,0(~

,)(22

NNu

uxxy

tijki

tijkiijtijk ti

Estimate of Covariance ParametersEstimate of Covariance Parameters

)( it xxik

12

Table 3. Treatment ReceivedTable 3. Treatment Received Teacher 1 2 3 4 5 6 7 8 9

x -1 0 1 -1 0 1 -1 0 1

Child

1 1 1 1 1

2 1 0 1 .6667

3 0 1 1 .6667

4 1 1 0 .6667

5 0 0 0 0

6 0 0 1 .3333

7 0 1 0 .3333

8 -1 -1 1 .3333

9 -1 0 1 0

10 1 1 1 1

11 -1 -1 -1 -.3333

12 -1 -1 0 -.6667

13 -1 1 0 0

14 -1 0 1 .3333

15 0 0 1 .3333

16 0 -1 0 -.3333

17 0 0 0 0

18 -1 -1 1 -.3333

19 -1 0 1 0

20 -1 -1 -1 -.3333

-0.25 0 0.45

ix

kx

13

Random child and school effects with x Random child and school effects with x “as if randomized”“as if randomized”

),0(~),,0(~),,0(~

,222

NNsNu

suxy

tijkki

tijkkijtijk ti

Estimates of Fixed EffectsParameterParameter EstimateEstimate Std. Err. Std. Err. dfdf tt Sig.Sig.

Intercept 7.864998 2.493818 3.034 3.154 .050

x 2.468256 .285074 38.494 8.658 .000

Estimates of Covariance Estimates of Covariance ParametersParameters


σσ22 1.0006171.000617

ττ22 23.75986923.759869

ψψ22 15.04229215.042292

14

Two dimensional controls: OLS fixed child and school effectsTwo dimensional controls: OLS fixed child and school effects

fixed2,1

fixed19,...,1,

),,0(~

,2

k

i

tijk

tijkkitijtijk

s

iu

N

suxy

ParameterParameter EstimateEstimate Std. ErrorStd. Error dfdf TT Sig.Sig.

Intercept 14.642231 .630345 37 23.229 .000

X 2.573106 .287937 37 8.936 .000

[childid=1.00] -11.449864 .998365 37 -11.469 .000

[childid=2.00] -6.393372 .946257 37 -6.756 .000

[childid=3.00] -4.474496 .946257 37 -4.729 .000

[childid=4.00] -9.957064 .946257 37 -10.523 .000

[childid=5.00] -8.433180 .864876 37 -9.751 .000

[childid=6.00] -8.925554 .901385 37 -9.902 .000

[childid=7.00] -7.215747 .901385 37 -8.005 .000

[childid=8.00] -6.845734 .864876 37 -7.915 .000

[childid=9.00] -6.090831 .864876 37 -7.042 .000

[childid=10.00] -6.743514 .998365 37 -6.755 .000

[childid=11.00] -.006149 .815539 37 -.008 .994

[childid=12.00] -.045263 .821167 37 -.055 .956

[childid=13.00] 1.176263 .837825 37 1.404 .169

[childid=14.00] .745534 .864876 37 .862 .394

[childid=15.00] 1.526586 .901385 37 1.694 .099

[childid=16.00] 2.413467 .837825 37 2.881 .007

[childid=17.00] 2.255688 .864876 37 2.608 .013

[childid=18.00] 3.047574 .837825 37 3.637 .001

[childid=19.00] 3.193858 .864876 37 3.693 .001

[childid=20.00] 0(a) 0 . . .

[schoolid=1.00] -7.679293 .367143 37 -20.916 .000

[schoolid=2.00] -3.340106 .347120 37 -9.622 .000

[schoolid=3.00] 0(a) 0 . . .

Estimates of Covariance ParametersEstimates of Covariance Parameters

2Parameter Estimate

σ2 .997655

15

Two-Dimensional Controls: Random child and school Two-Dimensional Controls: Random child and school effects with interaction-contrast centeringeffects with interaction-contrast centering

),0(~

),,0(~

),0(~

,)(

2

2

2

Ns

Nu

N

suxxxxy

k

i

tijk

tijkkikijtijk ti


ParameterParameter EstimateEstimate Std. Err. Std. Err. tt Sig.Sig.

InterceptIntercept 8.0294638.029463 2.8515202.851520 2.8162.816 .083.083

2.5731062.573106 .287937.287937 8.9368.936 .000.000xxxx kijti

Estimates of Covariance ParametersParameter Estimate

σ2 .997655

τ2 16.857298

ψ2 21.815022

16

Two-Dimensional Controls:Two-Dimensional Controls:fixed school effects, random kid effects, fixed school effects, random kid effects, person-mean centered x.person-mean centered x.

fixed2,1for

),,0(~

),0(~,)(2

2

ks

Nu

Nsuxxy

k

i

tijktijkkiijtijk ti

ParameterParameter EstimateEstimate Std. Err.Std. Err. dfdf TT Sig.Sig.

InterceptIntercept 11.70268211.702682 .951278.951278 21.03221.032 12.30212.302 .000.000

2.5731062.573106 .287937.287937 37.00037.000 8.9368.936 .000.000

[schoolid=1.00][schoolid=1.00] -7.679293-7.679293 .367143.367143 37.00037.000 -20.916-20.916 .000.000

[schoolid=2.00][schoolid=2.00] -3.340106-3.340106 .347120.347120 37.00037.000 -9.622-9.622 .000.000

[schoolid=3.00][schoolid=3.00] 0(a)0(a) 00 .. .. ..


ij xxti


σ2 0.997655

τ2 16.857298

Estimates of Covariance ParametersEstimates of Covariance Parameters

17

ClaimsClaims

1.1. For studying time-varying treatments, For studying time-varying treatments, adaptive centering with random effects adaptive centering with random effects replicates fixed effects analysis in any replicates fixed effects analysis in any dimensiondimension

2.2. Adaptive centering with random effects is Adaptive centering with random effects is generally the preferable approachgenerally the preferable approach

18

a. A natural way to incorporate a. A natural way to incorporate uncertainty as a function of clusteringuncertainty as a function of clustering

Note we are incorporating uncertainty Note we are incorporating uncertainty associated with classrooms, which cannot be associated with classrooms, which cannot be done using fixed effects if the treatmentdone using fixed effects if the treatment

is at that level.is at that level.

19

Two-dimensional controls (kids and schools)random effects of kids, teachers within schools, schoolsinteraction contrast for treatment

),0(~

),0(~

),,0(~

),,0(~

,)(

2)(

2

2

2

)(

Nc

Ns

Nu

N

csuxxxxy

kj

k

i

tijk

tijkkjkikijtijk ti

Parameter Estimate Std. Err. t Sig.

Intercept 8.029410 2.436174 3.296 0.170

2.573422 0.284396 9.049 .000


xxxx kijti


σ2 0.97125

τ2 16.73688

2 0.00073

ψ2 15.24546

20

b. A natural framework for modeling heterogeneity

* Heterogeneity is interesting;

* A failure to incorporate heterogeneity leads to biased standard errors.

),0(~

),0(~

,0

0~

,0

0~

))((

2

2)(

1110

0100

1

0

1110

0100

1

0

11)(000

N

Nc

Ns

s

Nu

u

xxxxsucsuy

tijk

kj

k

k

ik

i

tijkkijkikkjkitijk ti

21

c. We can easily study multilevel c. We can easily study multilevel treatment and their interactiontreatment and their interaction

),0(~

),0(~

,0

0~

,0

0~

)(*

))((

2

2)(

1110

0100

1

0

1110

0100

1

0

10

11)(000

N

Nc

Ns

s

Nu

u

xxxxww

xxxxsucsuy

tijk

kj

k

k

ik

i

tijkkijkk

kijkikkjkitijk

ti

ti

22

d. Improved estimates of unit-specific effects

• Fixed Effects Approach via OLS

fixeds

fixediu

N

suxy

k

i

tijk

tijkkijtijk ti

2,1

19,...,1,

),,0(~

,2

23

Random Effects ApproachEmpirical BayesStep 1: Estimate

fixedk for s

Nu

N

suxxy

k

i

tijk

tijkkiijtijk ti

2,1

),,0(~

),0(~

,)(

2

2

24

Random Effects Approach

• Step 2: Compute

)ˆ,ˆ,ˆ,ˆ,|(ˆ

),,0(~),,0(~

,ˆˆ

22

22

adjadjkadji

EBi

adjadjiadj

adjtijk

adjtijk

adji

adjkjtijk

adjtijk

syuEu

NuN

usxyyti

25

Results

• Correlation

• Mean Squared Error

• Relative Efficiency

993.),( OLSEB uur

3311.20/)ˆ()(

3647.20/)ˆ()(

20

1

2

20

1

2

ii

EBi

EB

ii

OLSi

OLS

uuuMSE

uuuMSE

91.3647./3311.)ˆ:ˆ( EBOLS uuEfficiency Relative

26

Role of reliability

• Reliability of OLS Fixed Effects

• In large samples,efficiency of OLS relative to EB is approximately equal to the reliability (Raudenbush, 1988, Journal of Educational Statistics).

99.3/92.24.24

24.24

/)ˆ(

)(22

2

TuVar

uVar

adjadj

adj

OLSi

27

e. Computational Easee. Computational Ease

We don’t need dummy variables to We don’t need dummy variables to represent kids, teachers, or schools.represent kids, teachers, or schools.

Documents

Adaptive Centering with Random Effects in Studies of Time-Varying Treatments