Long-term effect of Michigan’s state-funded preschool...

Long-term effect of Michigan’s state-funded preschool program

Tomoko Wakabayashi

Oakland University

Lawrence J. Schweinhart

HighScope Educational Research Foundation

Zongping Xiang

SRCD--Baltimore, MDMarch 21, 2019

Michigan Great Start Readiness Program (GSRP)

•Established: 1985

•The Premise: High-quality preschool can mitigate risk factors and support the school readiness and subsequent achievement for children with demographic factors that predispose them to poorer school and life outcomes.•Evaluation: 1995-(HighScope 1995-2017)

•Recent Expansion: 2013-2015

What are the program elements and implementation context?

NIEER Benchmark (2017)

What is the research design and key findings on impact?

GSRP Evaluation Study SamplesStudy Year N

Longitudinal Evaluation 1995-2011 Treatment=338

Comparison=258

GSRP Lansing Regression

Discontinuity Design (RDD)

2011-2014 Treatment=317

Comparison=333

Preschool Variation Study 2004-2011 GSRP=190

Head Start=143

Tuition-based=70

Pre-Post Design

• Urban

• Rural

• West

2011-2017

2012-2017

2014-2017

Cohort 2=483

Cohort 3=463

Cohort 2=467

Cohort 1=525

Cohort 2=598

GSRP Longitudinal High School Graduation Study

Schweinhart, L. J., Xiang, Z., Daniel-Echols, M., Browning, K. & Wakabayashi, T. (2012, March). Michigan Great Start Readiness Program evaluation 2012: High school graduation and grade retention findings. Technical report prepared for Michigan Department of Education. https://www.michigan.gov/documents/mde/GSRP_Evaluation_397470_7.pdf

A study of 595 low-income children entering kindergarten in 1996

•In six Michigan school districts – Detroit, Grand Rapids, Grayling, Kalamazoo, Muskegon, and Port Huron.

•338 children attended GSRP.

•257 children did not attend a preschool program but had family incomes under $30,300, low enough to qualify for GSRP.

Similar Group BackgroundsCharacteristic GSRP No GSRP

Age at kindergarten entry 5.3 5.3

% female 51% 51%

Fathers in home 62% 61%

Persons in household 4.5 4.7

Mothers’ years of schooling 12.1 12.0

Fathers’ years of schooling 12.1 11.7

Average annual income $17,882 $18,022

High School Graduation on Time

57%59%

56%

43%

37%

47%

0%

10%

20%

30%

40%

50%

60%

70%

All Non-White White

GSRP

No GSRP

****

Grade Retention by Grade 12 by GSRP Status

37%

49%

0%

10%

20%

30%

40%

50%

60%

Repeated a grade

GSRP

No GSRP

**

Graduation Timing by Grade Retention

80%

6%

3%

28%

17%

67%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

Never retained Ever retained

Graduated on Time

Graduated a Year Later

Has not graduated

How can the findings be used to improve quality in the field at large?

Michigan GSRP Expansion

FY2014 $65Million budget

FY2015 $65Million budget

TOTAL: $130 Million expansion

GSRP enrollment increased by 14,891 children annually.

GSRP Benefit-Cost Analysis

Wakabayashi, T., Claxton, J., & Hardin, B. A. (2018, February). Examining costs and benefits of Michigan Great Start Readiness Program expansion 2013-2016: Phase 2 report. Technical report prepared for Michigan Department of Education.

Wakabayashi, T., Hardin, B. A., Claxton, J., & Grace, E. (2017, September). Examining costs and benefits of Michigan Great Start Readiness Program expansion 2013-2016: Phase 1 report. Technical report prepared for Michigan Department of Education.

Legislative decision to expand GSRP was based on:

The evidence that attending GSRP programs can effectively increase eligible children’s school readiness and later school success;

The assumption that more funding will increase GSRP access, i.e., more four-year-olds eligible for GSRP will enroll in the program; and

The expansion will not compromise the essential components that contribute to the effectiveness of the GSRP model; hence more children will reap the benefit of GSRP.

GSRP Total Enrollment FY 2013-2016

State-wide GSRP PQA Mean Scores FY2013-2016

PQA Scale 2012–13 2013-14 2014-15 2015-16

Classroom Level (Form A) N=1042 N=1537 N=2034 N=2245

Total Score for Form A 4.20 4.29 4.30 4.35

I. Learning environment 4.15 4.20 4.23 4.28

II. Daily routine 4.15 4.31 4.33 4.37

III. Adult-child interaction 4.23 4.27 4.23 4.27

IV. Curriculum planning and assessment 4.37 4.49 4.54 4.59

Center Level (Form B) N=392 N=333 N=480 N=659

Total Score for Form B 4.35 4.32 4.41 4.43

V. Parent involvement and family services 4.50 4.38 4.48 4.49

VI. Staff qualifications and development 4.02 4.07 4.18 4.24

VII. Program management 4.47 4.48 4.55 4.55

Benefit-Cost Analysis

Total ADDITIONAL

cost (FY2014-2015)

(C)

$195,104,004 $95,015,650 $54,629,121

Yield (number

children NEWLY

served during the

expansion year)

20,265 9,869 5,674

HS Graduates

over HS Dropouts

(Newly Served)

Female only

(48.7% of Total)

Black only

(28% of Total)

Total benefit $458,410,330 $311,237,798 $219,547,725

Net Benefit (B-C) $263,306,326 $216,222,148 $164,918,604

Benefit/Cost Ratio 2.35 3.28 4.02

Legislative changes in GSRP program elements and implementation

• Administrative/financial oversight

• Community-based organizations

• Income eligibility

• Transportation funds

Issues raised➢Regional differences

➢Teacher salaries and benefits

➢Rent/facility usage and maintenance

$23,020 $21,635 $31,488

$20,609

$40,788

$59,290 $56,800 $62,720

$- $10,000 $20,000 $30,000 $40,000 $50,000 $60,000 $70,000

*

Questions?

Tomoko Wakabayashi Twakabayashi@Oakland.edu

Oakland University

@twakabayashi264

For more information about GSRP:Michigan.gov/gsrp

Thank you!

EXTRA SLIDES--ONLY USE IF NEEDED DURING Q & A--

GSRP Risk Factors

1. Low income;

◦ 90% of enrollment below 250% Federal Poverty Level;

◦ 10% may attend GSRP w/ sliding scale tuition based on income (from 2013)

2. Diagnosed disability/developmental delay;

3. Severe or challenging behavior;

4. Primary home language other than English;

5. Parent(s) with low educational attainment;

6. Abuse/neglect of child or parent;

7. Environmental risk.

Kindergarten Teacher Ratings of Retained and Non-Retained Students

74%

80%

61%

80%

80%

90%

22%

33%

16%

39%

61%

75%

0% 20% 40% 60% 80% 100%

Literacy

Math

GOOD STUDENT

Retains learning

Completes assignments

SOCIAL

Gets along with children

Gets along with adults

Not Retained

Retained

GSRP Evaluation 2012-2017 Study Sites

Kent ISD (2014 – 2017)

•Pre-Post Risk Factor Analysis (2 cohorts) with focus on Spanish-English Dual Language Learners

Thumb Area (Huron, Tuscola & Sanilac; 2012- 2017)• Pre-Post Risk Factor Analysis (2

cohorts)

Lansing School District (2011- 2017)

•Regression Discontinuity Study (2011-2014)

•Pre-Post Risk Factor Analysis (2 cohorts)

GSRP Evaluation 2012-2017GSRP has significant impact on increasing children’s early literacy and math skills (GEE-RDD*).

*Group Equivalency Enhanced Regression Discontinuity Design—a supplementary data collection and analysis method to enhance the equivalency between treatment and control that we added to a traditional RDD.

Wakabayashi, T., & Xiang, Z. (2014, October). Michigan Great Start Readiness Program evaluation: Lansing school district 2011-2014—Group Equivalency Enhanced Regression Discontinuity Design. Technical report prepared for Michigan Department of Education.

Regression Discontinuity Design (RDD)

GEE-RDD SampleTreatment Comparison

N=333 N=317Female 167 53% 162 52%

Male 166 47% 148 48%Risk Factors

Extremely low-income 228 72% 222 72%Primary language other than English 50 16% 43 14%

Low parental education 66 21% 81 26%Ethnicity/Race

White 120 38% 96 31%

Black/African American 65 21% 86 28%Asian 7 2% 10 3%Hispanic 47 15% 65 21%Other 93 24% 60 17%

Group Equivalency Enhanced RDD Sample

Type of Participants Included in the Estimation

NEstimated Treatment

Effect at CutoffTreatment (K-entry)

Comparison (Pre-K entry) PPVT Math

Letter-Word

Traditional RDD (All participants tested)

Adjusted for SES & demographic status 303 363 2.89 1.57** 4.20**

Group Equivalency RDD (Controlled for differential attrition)Participants with & without sufficient treatment

Adjusted for SES & demographic status 333 317 4.50+ 1.73** 4.55**Participants with sufficient treatment only

Adjusted for SES & demographic status 316 310 4.86+ 1.79** 4.47**

+ p<.10; ***p<.01

Estimated Treatment Effects (in raw score

GSRP Evaluation 2012-2017

➢ Significant achievement gap exists between higher-risk and lower-risk children at GSRP entry (Pre).

➢ GSRP helps reduce the achievement gap between higher-risk children and lower-risk children (Pre-Post).

Wakabayashi, T., & Xiang, Z. (2012, October). Michigan Great Start Readiness Program evaluation: Lansing school district 2011-2012. Technical report prepared for Michigan Department of Education.

Wakabayashi, T., & Xiang, Z. (2015, October). Michigan Great Start Readiness Program evaluation: GSRP programs in Huron, Tuscola, and Sanilac Counties (the Thumb area) 2012–2013. Technical report prepared for Michigan Department of Education.

Wakabayashi, T., Xiang, Z., & Kantor, P. (2013, January). Preschool Variation Study: Follow-up through fifth grade (2004-2011). Technical report prepared for Michigan Department of Education.

Vulnerability Category Status N PPVTLetter Word

Spelling Math

Extremely low family incomeYes 271 98.58 96.42 93.46 101.71

No 92 109.03 102.11 97.39 106.36

Diagnosed disability/developmental delayYes 114 102.90 97.57 94.48 103.37

No 249 100.46 98.06 94.47 102.65

Severe or challenging behaviorYes 7 105.86 103.29 93.00 108.57

No 356 101.14 97.79 94.50 102.76

Primary home language other than EnglishYes 59 96.00 96.54 95.19 102.32

No 304 102.24 98.17 94.34 102.98

Parent(s) with low educational attainment

Yes 77 98.51 93.77 92.24 99.81

No 286 101.96 98.98 95.08 103.73

Abuse/neglect of child or parentYes 19 106.32 101.37 95.65 104.05

No 344 100.95 97.71 94.40 102.81

Environmental riskYes 303 101.00 97.84 94.08 102.80

No 60 102.40 98.20 96.36 103.28

Minority (non-White)Yes 229 98.02 96.47 93.91 100.59

No 114 106.72 100.26 95.41 106.71

Mean Program Entry and Gain Scores by Vulnerability Index (Urban)

Gain Scores

0 risk vs. 1-3 risks: p < .05

112.1

105.21

97.96

93.43

80

85

90

95

100

105

110

115

n=40 n=113 n=166 n=44

0 1 2 3

Vulnerability index

Stan

dar

d S

core

s

1.07

4.6 4.36 4.09

0123456

n=40 n=113 n=166 n=44

0 1 2 3

Vulnerability index

Mean Program Entry and Gain Scores by Vulnerability Index (Rural)

Gain Scores

Low vs. High risks: p < .05

106.83 105.76

101.74

80

85

90

95

100

105

110

115

n = 89 n = 182 n = 47

Low Medium High

Vulnerability Index

Stan

dar

d S

core

s

1.932.59

5.32

0123456

n = 89 n = 182 n = 47

Low Medium High

Vulnerability Index

Preschool Variation Study

Original versus Longitudinal follow-up Sample Size

Preschool group

N

Original

N

Follow-up

%

Retention

GSRP 190 160 84

Head Start 143 116 81

Tuition-Based 70 58 83

Total 403 334 83

PPVT and WJ Math Scores at Program Entry by Vulnerability Level

Vulnerability Category nPPVT WJ Math Problem

mean p mean p

Number of economic risks 4 15 91.8

<.001

94.1

<.0013 75 91.5 98.32 59 97.9 100.91 88 98.8 101.10 158 105.3 105.9

Mother’s educationLower than high school 24 92.9

<.00193.6

<.001High school diploma 184 96.3 99.62-4 years of college 116 100.6 103.9Bachelor degree or higher 71 108.9 109.1

Minority statusNon white 99 91.8 <.001 97.0 <.001White 300 102.1 103.9

Overall vulnerability level3 risks 70 88.7

<.00195.3

<.0012 risks 117 97.8 100.51 risk 107 101.4 102.70 risk 105 106.7 108.1

Preschool Variation Study—Preschool Entry Scores

102.2

94.8

103.2

94.5

103.1

98.8

104.9

98.7

889092949698

100102104106

LowerVulnerability

HigherVulnerability

LowerVulnerability

HigherVulnerability

GSRP Head Start

Stan

dar

d S

core

s

Program Entry

PPVT

Math

Preschool Variation Study—Preschool Gain Scores

1.9***

5.9***

1.6 1.41.9

3.5

1.71

01234567

LowerVulnerability

HigherVulnerability

LowerVulnerability

HigherVulnerability

GSRP Head Start

Stan

dar

d S

core

s

Gain Scores

PPVT

Math

***p<.001

Preschool Variation Study—MEAP Scores

67.1 67.9 66.2

87.277.8

83.878.1 68.5

74.3

55.2**52.1* 48.5**

0102030405060708090

100

Grade 3 Grade 4 Grade 5 Grade 3 Grade 4 Grade 5

GSRP Head Start

% P

rofi

cie

nt

Reading

LowerVulnerability

HigherVulnerability

*p<.05; **p<.01

Preschool Variation Study—MEAP Scores

73.7 71.663.7

89.7 91.781.182.2 80.8

72.9

58** 59.2**

43.5**

0102030405060708090

100

Grade 3 Grade 4 Grade 5 Grade 3 Grade 4 Grade 5

GSRP Head Start

% P

rofi

cie

nt

Mathematics

LowerVulnerability

HigherVulnerability

**p<.01