Do Stimulant Medications Improve

Educational and Behavioral Outcomes

for Children with ADHD?

Janet Currie, Mark Stabile, Lauren Jones

Motivation

• Over past 20 years, mental health disabilities have overtaken physical disabilities as the leading cause of activity limitations in children.

• U.S. Surgeon General’s report on Mental health:

– 20% of children in the U.S. suffer some impairment from a mental health disorder

– 11% have significant functional impairment

– 5% extreme functional impairments

• ADHD is 3 times more likely than asthma to be contributing to childhood disability in the US.

Kids with ADHD: outcomes and treatment

• Recent literature indicates that children with ADHD have:

– Lower standardized test scores

– More likely to be placed in special education

– More likely to be delinquent (Currie and Stabile, 2006, 2007)

• Untreated children disrupt learning for their classmates (Aizer,

2009)

• In the US about 8% of children diagnosed with ADHD. More than

half on stimulant medication such as Ritalin (CDCP, 2005)

• Millions of children worldwide taking stimulants such as Ritalin

for ADHD

• (Ritalin is the brand name for Methylphenidate, a central nervous

system stimulant that helps with hyperactivity and impulse

control)

Controversy over treatment.

• Despite or perhaps because of the number of children taking

meds, treatment for ADHD is somewhat controversial.

• National Institute of Mental Health (US) recommends

treatment with stimulants and says it is safe under medical

supervision (NIMH 2012)

• However, there remain concerns about safety and side effects

including cardiovascular problems, decreased appetite,

insomnia, headache, dizziness, and mood changes (Nissen,

2006, Schachter, 2001, NIMH, 2012).

• Significant coverage in the NY Times over the past year on

treatment and side effects of treatment for ADHD.

• E.g. In 2006 US Food and Drug Administration voted to

recommend a warning label for cardiovascular risk

Controversy cont.

• Part of the reason for the controversy around treatment is that

is quite difficult to get evidence on the longer term benefits.

• Drugs often prescribed with the goal of helping children do

better in school. But we don’t know much about whether they

do.

• Most drug trials only follow children for a short period of time

• Families choose whether to seek treatment and how long to

stay on – not random.

• If drugs do not actually lead to scholastic benefits in the long

run, then the case for subjecting kids to even a small risk of

side effects is weakened.

Our contributions:

• Assess the short and long term treatment for ADHD with stimulant medication using longitudinal data and a policy experiment which expanded access to coverage for prescription drugs.

• Large sample of children followed from 1994 to 2008

• All children assessed for ADHD so that we do not have to deal with selection into diagnosis

• Exploit exogenous variation in the availability of drugs due to the policy experiment

• Able to use child fixed effects to control for unobservable characteristics of the child.

Findings in Brief

• We find strong evidence that insurance expansion in Quebec

resulted in an increase in the use of Ritalin relative to the rest

of Canada.

• We find little evidence of reported improvements in behavior.

• We find negative effects on some schooling and behavior

measures in both the medium and long run.

What do we know about Stimulants and Outcomes?

• Controlled studies suggest medication improves short

term memory and performance over a few months.

Assumed these will translate into academic gains.

• US National Institute of Mental Health Multimodal

Treatment Study (MTA): random selection of ADHD

kids into 1) behavioral therapy, stimulants plus

behavioral therapy, stimulants alone, usual

community care. No significant difference in

behavior between the four groups.

• Note: all children were treated, is not comparing

“community care” vs. “no care.”

• Studies that track kids who select into treatment

(Zoega, 2009, Barberesi et al 2007) suggest treated

children do better when treatment starts earlier (but

potential endogeneity of treatment start dates).

• Dalsgaard, Nielsen, Simonsen (2012) show positive

effects on ER contact, poisoning, injuries.

• ADHD with treatment on health (Barberesi et al

2013) – higher rate of suicide.

• We know much less about the effects on education.

Data

• National Longitudinal Survey of Children and Youth

• Surveyed children ages 0-11 and their families

beginning in 1994.

• For short term analysis: use full cross sectional data

for all children ages 2-15 where possible.

• Includes ADHD score for all children as defined

above

• Includes indicator variable for taking stimulants

(Ritalin or other similar medication).

• For longer term analysis: use longitudinal sample of

children ages 2-11 in 1994. Track children through to

2008

The Measurement of Mental Health

• Measures taken from Qs asked to parents about symptoms.

• Qs similar to those asked as a first step in a diagnosis of a mental health condition.

• Asked of all children, regardless of whether they have a diagnosis or not.

• Provides a linear measure for children ranging from no symptoms to mild to severe, instead of a 0/1 measure.

Mental Health Var E.g.

• Hyperactivity score:

– HOW OFTEN WOULD YOU SAY THAT “name”:

• Can't sit still, is restless or hyperactive?

• Is distractible, has trouble sticking to any activity?

• Can't concentrate, can't pay attention for long?

• Cannot settle to anything for more than a few moments?

• Is inattentive?

Distribution of Hyperactivity Scores:

0

5

10

15

20

25

Score 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

Outcomes of Interest – Short Term

• Behavior, medium term:

– Behavioral scores

– Relationship with parents

Based on questions to parents.

Behavior Scores

• Anxiety-Emotional Disorder Score. Score ranges from 0-

12. Higher is worse.

• Subset of anxiety related to happiness: How often does

seem:

1. unhappy, sad or depressed

2. not as happy as other kids

3. unable to enjoy him/herself

• How well does child get along with parents over the past

six months? Child gets along “quite well” or “very well”

gets a 1; 0 otherwise

Outcomes of Interest – Short Term

• Education, medium term:

– Math test score (based on 15 questions from

Canadian Achievement Test. Administered in

schools.)

– Parents asked whether child repeated a grade in

past 2 years.

Outcomes – Long Term

• Behavior Longer term:

– Ever diagnosed with a mental/emotional disorder

– Depression score

• Education, Longer term:

– Graduates from high school

– Ever attends post-secondary schooling

Quebec’s Insurance Expansion

• January 1, 1997 Quebec implemented its prescription drug

insurance program. This is on top of regular public health

insurance.

• All residents of Quebec, by law, required to have drug

insurance

• Can have either private coverage through employer or group or

public (new)

• Public plan available for seniors, welfare recipients, and

people without access to a private employer plan.

• Public plan premiums and deductibles are scaled according to

income. Maximum premium in 1997 was $750 with 25% co-

insurance.

Quebec

Rest of Canada

Wave 1 1994

Wave 2 1996

Wave 3 1998

Wave 4 2000

Wave 5 2002

Wave 6 2004

Wave 7 2006

Wave 8 2008

Medium term

outcomes sample

Age 2-11

Age 2-13

Age 4-15

Age 6-16

Age 8-16

Age 10-16

Age 12-16

Age 14-16

Policy Change

Date

Changes in Insurance Coverage in Quebec versus the Rest

of Canada:

Reported Drug Insurance Coverage

Pre-Reform Post -Reform

Year 1996 1998 2002 2003

Quebec 55% 84% 86% 89%

Rest of Canada 65% 72% 74% 76%

Source: Authors’ calculations from NPHS and CCHS

Empirical Strategy

We first estimate a difference-in-difference for the effects of the policy on the

use of stimulants:

Where X includes family income, pmk immigrant, pmk sex, child sex, birth

order, family size, two parents, mother’s age at the birth, whether mom had a

teen birth, and child-age dummies. Includes Year dummies and Province

dummies.

Policy variable is lagged by one period to allow time for take up.

Some versions of the model include a child fixed effect, so that we are

comparing the same children over time.

Empirical Strategy

Next estimate a DDD model, focusing on those children most likely to be

affected by an increase in stimulant use, those with higher pre-policy ADHD

scores:

Prefer DDD as DD may reflect province specific time trends.

Empirical Strategy Cont.

• For longer term outcomes follow child from 1994 to 2008.

• Observe exposure to stimulants between ages 4 and 15.

• Measure whether child is ever on stimulants between 4 and 15.

• DDD term is the number of years in Quebec between ages 4

and 15 post 1996 * 1996ADHD score.

Sample 1 - Outcomes observed before age 16

Variable Quebec Rest of Canada

ADHD score in 1996 2.83 (2.42) 2.65 (2.31)

Stimulants, waves 1 and 2 0.016 0.014

Stimulants, waves 3 and up 0.049 0.023

Medium-term Outcomes

Unhappiness Score (6 pt) 0.47 (0.80) 0.53 (0.89)

Parent relationship 0.93 (0.26) 0.85 (0.36)

Standardized Math Score 10.02 (4.49) 8.27 (4.37)

Repeat a grade since last interview 0.07 0.02

Child and Family Characteristics

Child is male 0.51 0.51

Child is first born 0.55 0.50

Permanent Household income $58,958(33087) $64,518(36938)

Two-parent household 0.82 0.84

Family size 3.99 (0.93) 4.12 (0.93)

Mother age at birth 27.98 (4.78) 28.14 (5.13)

Mother high school grad 0.85 0.90

Mother is working 0.71 0.75

PMK is an immigrant 0.04 0.09

Number of children in sample 1 2,478 10,471

Number of obs. In sample 1 10,622 44,617

Sample 2 - Outcomes observed after age 16

Variable

Lives in Quebec

(cycle 1)

Rest of Canada

(cycle 1)

ADHD score in 1996 2.80(2.44) 2.54(2.32)

Ever Stimulants 0.09 0.05

# Surveys used Stimulants, given ever used 2.21(1.26) 2.06(1.26)

Post-1997 Years used Stim., given ever used 1.91(1.30) 1.56(1.22)

Long-term outcomes

Self-assessed depression score (36 pt) 5.84(4.73) 6.53(4.99)

High school grad 0.85 0.90

Some post-secondary 0.77 0.65

Child and Family Characteristics

Child is male 0.50 0.50

Child is first born 0.53 0.46

Permanent household income $58,711 (34,333) $64,669 (37,075)

Two-parent household in 1994 0.89 0.88

Family size in 1994 3.93 (0.89) 4.06 (0.90)

Mother age at birth 27.77 (4.59) 27.96 (5.09)

Mother high school grad in 1994 0.82 0.87

PMK is an immigrant 0.07 0.05

Number of children in sample 2 1,654 6,989

Standard errors of continuous variables in parentheses.

Table 3: Effect of Law Change on Stimulant Use

Outcome: Uses Stimulants

(1) (2) (3) (4)

DD - FE DD - No FE DDD - FE DDD - No FE

After 1997 -0.0072** -0.0092** -0.0172** -0.0133**

(0.0027) (0.0036) (0.0031) (0.0040)

Quebec -0.0118 0.0154** -0.0256 0.0101**

(0.0209) (0.0027) (0.0243) (0.0039)

After 1997 * Quebec 0.0247** 0.0287** 0.0123** 0.0159**

(0.0046) (0.0062) (0.0030) (0.0052)

1996 ADHD Score - - - 0.0105**

(0.0006)

After 1997*1996 ADHD Score - - 0.0039** 0.0014

(0.0008) (0.0010)

Quebec*1996 ADHD Score - - 0.0051 0.0004

(0.0083) (0.0009)

Aft. 1997*Que.*ADHD Sc. - - 0.0043** 0.0048**

(0.0021) (0.0018)

N 55,239 55,239 55,239 55,239

Age Range 2-15 2-15 2-15 2-15

Table 3: Effect of Law Changes on Stimulant Use, Con’t

Outcome: Ever Used Stimulants

(5) (6)

DD DDD

U16 Survey years after 1997 0.0003 0.0078**

(Elig Yrs) (0.0036) (0.0033)

Quebec in 1994 -0.0120 0.0181

(0.0121) (0.0253)

Elig Yrs * Quebec 0.0196** 0.0073

(0.0034) (0.0085)

1996 ADHD Score - 0.0403**

(0.0047)

Elig Yrs*1996 ADHD Score - -0.0038**

(0.0012)

Quebec*1996 ADHD Score - -0.0201**

(0.0098)

EligYrs*Que.94*96 ADHD Sc. - 0.0056*

(0.0032)

N 8,643 8,643

Table 4: Child Fixed Effects Estimates of Exposure to the Policy on

Contemporaneous Outcomes

Dependant Variable: Did Not Repeat Grade Math score Unhappiness Relationship w

Parents

(1) (2) (3) (4) (5) (6) (7) (8)

After 1997 0.0247** 0.0207** -0.3179** -0.2836** -0.2846** -0.1977** 0.0227** -0.0015

(0.0063) (0.0061) (0.1072) (0.1129) (0.0176) (0.0285) (0.0094) (0.0109)

Quebec 0.0578** 0.0225 0.2730 -0.1511 -0.0023 0.2054 0.0480 -0.0665

(0.0282) (0.0388) (0.2410) (0.2828) (0.1133) (0.1280) (0.1157) (0.1330)

After 1997 * Quebec -0.0581** -0.0228** -0.1883 -0.0694 0.1232** 0.0769 -0.0353** -0.0014

(0.0068) (0.0076) (0.1927) (0.1929) (0.0152) (0.0475) (0.0073) (0.0099)

After 1997*1996 ADHD

Score - 0.0016 - -0.0141** - -0.0326** - 0.0088**

(0.0009) (0.0056) (0.0076) (0.0021)

Quebec*1996 ADHD Score - 0.0117 - 0.1369** - -0.0894** - 0.0789**

(0.0141) (0.0557) (0.0366) (0.0348)

Aft. 1997*Que.*ADHD Sc. - -0.0128** - -0.0403** - 0.0172 - -0.0124**

(0.0016) (0.0062) (0.0145) (0.0033)

N 44,968 44,968 32,515 32,515 36,458 36,458 22,554 22,554

Age Range 4-15 4-15 5-15 5-15 2-11 2-11 4-9 4-9

Magnitudes

• Coefficient on DDD term can be compared to

baseline magnitudes:

• Change in not repeating grades = (-.0128/.93)

= 1.4% decline

• Change in math score = (-.0403/10) = -.004 decline

• Change in Unhappiness = (.0172/.47) = 4% increase

• Change in Relationship w Parents = (-.0124/.93)

= 1.3% decline

Table 5: Effects of Exposure to the Policy on Long-Term

Outcomes

Dependant Variable: Depression Score High School grad Some Post-sec

(1) (2) (3) (4) (5) (6)

U17 Survey years after 1996 0.3696** 0.4226** -0.0028 -0.0066 -0.0005 -0.0096

(Elig Yrs) (0.0793) (0.1149) (0.0047) (0.0061) (0.0096) (0.0122)

Quebec in 1994 0.6756 1.6972** -0.2191** -0.2430** -0.1265** -0.2068**

(0.3817) (0.4795) (0.0307) (0.0448) (0.0377) (0.0662)

Elig Yrs * Quebec -0.0876 -0.3073** 0.0498** 0.0675** 0.0666** 0.0868**

(0.0883) (0.1413) (0.0097) (0.0164) (0.0142) (0.0219)

1996 ADHD Score - 0.3414** - -0.0206** - -0.0447**

(0.0923) (0.0062) (0.0089)

Elig Yrs*1996 ADHD Score - -0.0288 - 0.0019 - 0.0046

(0.0274) (0.0018) (0.0026)

Quebec*1996 ADHD Score - -0.4340** - 0.0125 - 0.0382**

(0.1242) (0.0133) (0.0163)

EligYrs*Que.94*96 ADHD Sc. - 0.0867** - -0.0067 - -0.0084

(0.0402) (0.0043) (0.0048)

N 6,493 6,493 4,676 4,676 4,676 4,676

Magnitudes

Comparing DDD coefficient to baseline yields:

- Depression: (.0867/5.84) = 1.5% increase

- High School Graduation (-.0067/.85) = .8% decrease

- Some Post-Secondary (-.0084/.77) = 1.1% decrease

Table 6: Effects on Stimulant Use by Gender

Outcome: Uses Stimulants

(1) (2)

Boys Girls

After 1997 -0.0255** -0.0077**

(0.0052) (0.0022)

Quebec 0.0070 -0.0651

(0.0325) (0.0391)

After 1997 * Quebec 0.0299** -0.0031

(0.0058) (0.0042)

After 1997*1996 ADHD Sc. 0.0049** 0.0022**

(0.0012) (0.0009)

Quebec*1996 ADHD Score 0.0009 0.0154

(0.0117) (0.0105)

Aft. 1997*Que.*ADHD Sc. 0.0004 0.0078**

(0.0025) (0.0021)

N 27,971 27,268

Age Range 2-15 2-15

Table 6: Effects on Stimulant Use by Gender

Outcome: Ever Used Stimulants

(3) (4)

Boys Girls

U16 Survey years after 1997 0.0107** 0.0032

(Elig Yrs) (0.0052) (0.0038)

Quebec in 1994 -0.0499 0.0580**

(0.0513) (0.0182)

Elig Yrs * Quebec 0.0254 -0.0043

(0.0160) (0.0040)

1996 ADHD Score 0.0525** 0.0228**

(0.0061) (0.0071)

Elig Yrs*1996 ADHD Score -0.0050** -0.0018

(0.0015) (0.0016)

Quebec*1996 ADHD Score -0.0057 -0.0293**

(0.0178) (0.0092)

EligYrs*Que.94*96 ADHD Sc. 0.0021 0.0076**

(0.0056) (0.0021)

Sample Size 4,333 4,310

Age Range 0-9 in 1994 0-9 in 1994

Table 7: Child Fixed Effects Estimate of Exposure to Policy

on Contemporaneous Outcomes, by Gender

Dependant Variable: Unhappiness

Rel. w.

Parents

Did Not Rep.

Gr. Math Sc.

Boys (1) (2) (3) (4)

After 1997 * Quebec 0.1300* -0.0106 -0.0439** -0.1299

(0.0666) (0.0136) (0.0091) (0.1712)

Aft. 1997*Que.*ADHD Sc. -0.0047 -0.0120** -0.0142** -0.0257*

(0.0203) (0.0057) (0.0018) (0.0134)

N 18,484 11,457 22,719 16,191

Girls

After 1997 * Quebec 0.0258 0.0090 -0.0124 -0.0063

(0.0344) (0.0152) (0.0128) (0.2027)

Aft. 1997*Que.*ADHD Sc. 0.0430** -0.0127** -0.0058* -0.0588**

(0.0120) (0.0060) (0.0031) (0.0210)

N 19,974 11,097 22,249 16,324

Age Range 2-11 4-9 4-15 5-15

Table 8: Effects of Exposure to the Policy on Long-Term

Outcomes by Gender

Dependant Variable: Depression Score High School grad Some Post-sec

Boys (1) (2) (3) (4) (5) (6)

Elig Yrs * Quebec -0.0223 0.1049 0.0591** 0.0836** 0.0735** 0.0744**

(0.0841) (0.1932) (0.0171) (0.0242) (0.0187) (0.0271)

EligYrs*Que.94*96 ADHD Sc. - -0.0317 - -0.0091* - -0.0017

(0.0630) (0.0047) (0.0058)

N 3,213 3,280 2,259 2,259 2,259 2,259

Girls

Elig Yrs * Quebec -0.1428 -0.6068** 0.0439** 0.0549** 0.0619** 0.0942**

(0.1450) (0.2221) (0.0137) (0.0210) (0.0195) (0.0237)

EligYrs*Que.94*96 ADHD Sc. - 0.2025** - -0.0041 - -0.0150**

(0.0541) (0.0049) (0.0050)

N 3,280 3,280 2,417 2,417 2,417 2,417

Robustness checks:

• Look at ventilator use – maybe effects are due to some other

drug use?

• Exclude kids with chronic conditions.

• Exclude kids who may have been affected by the roll out of

universal preschool.

• Use a broader measure of psychological problems.

Placebo test

• Exclude Quebec.

• Imagine policy experiments in each province in turn, and in

each year from 1995 to 2005.

• Estimate the model for each of these “experiments.”

• Look at the distribution of estimates and where our estimate

lies in this distribution.

Why might stimulant use have negative effects?

• This natural experiment reflects “community care” which may

not be optimal care. E.g. among those who report using

stimulants, children use them for about 30% of the survey

years we observe them.

• Stigma or increased probability of placement in special

education?

• Medication may substitute for other types of therapy or

educational responses that would be more effective. E.g. if

children are less disruptive, them may be less likely to receive

needed attention. Ritalin improves focus, but does not help

children to decide what to focus on.

Summary

• Policy change in one Canadian province that

improved access to prescription drugs resulted in an

increase in the use of Ritalin.

• We examine the effects of this increase on behavioral

outcomes, child relationships, short-term educational

outcomes and some longer term educational

outcomes.

• We find little evidence of improvements in child

outcomes, and fairly consistent evidence of small

negative effects. Effects on emotional outcomes are

worse for girls.