Performance Based Incentives for Learning in the Mexican Classroom Brian Fuller, MPA, Foundation Escalera Victor Steenbergen, MPA Candidate, London School

Performance Based Incentives forLearning in the Mexican Classroom

Brian Fuller, MPA, Foundation Escalera Victor Steenbergen, MPA Candidate, London School of Economics

Agenda

1. Star Program in Context2. Theoretical Overview3. Evaluation Strategy4. General Treatment Effects on Test Scores5. Indirect Treatment Effects (Teachers and Parents)

The Star Program• Incentives program aimed at marginalized, rural middle

school students (ages 12-15) in Chiapas, Mexico.

• RCT with 147 schools participating (76 treatment, 71 control)

• In total 7852 students (4011 treatment, 3841 control)

• Chiapas, Mexicoo 37% of workers earn less than monthly minimum wageo Ranks last nationally for computer ownership (7.2%), landlines (12.6%) o 49% rural residents

• Indigenous Peopleso 39% cannot read or writeo 83% work in agriculture o Avg. income is 32% of avg non-indigenous

• Telesecundaria Middle Schoolso Distance learning expands coverage -> many first-time learnerso 95% and 89% in lowest two categories in reading and math

• (21% and 19%, OECD)

The Star Program

A. Most Improved Student (1 Female, 1 Male)o Increases in general GPA

o Targets middle and low-achieving students

B. A+ Lottery Winners (1 Female, 1 Male)o Individual subject grades = tickets

o Targets high achieving students

The Star ProgramThree times a year, each grade selected:

•First and second grade students: o material incentives (books, MP3-players, laptops)o increases likelihood students keeping their prices

•Third grade students:o financial incentives (scholarships up to M$2000).o grants might involve parents in children’s learning.

•All winners:o Certificates of achievemento Public recognition at ceremony

The Star ProgramAwards


Agenda

• Incentive programs work when students lack sufficient motivation to learn (Fryer, 2010)o They do not ‘enjoy’ learning mucho Their concerns are short-term driveno They lack knowledge of future benefits to education.

• Primary impact: financial reward for learning might incentivise student effort to win the prize.

Theoretical Overview

• Student performance requires complementary support from parents and teachers (Fryer, 2010)

• Secondary impact: incentive program might enhance parent and teacher involvement (Kremer et al, 2004).o Parents: family also benefit from the incentive program (e.g. scholarship)o Teachers: community informally sanctions absent teachers and rewards winners.o Teachers: increased student motivation leads to greater teacher effort.

Theoretical Overview

Increase student effort to win prizeIncrease student

effort to win prize

INCENTIVE PROGRAMINCENTIVE PROGRAM

Student Extrinsic MotivationStudent Extrinsic Motivation

OVERALL STUDENT

ACHIEVEMENT

OVERALL STUDENT

ACHIEVEMENT

Parental InvolvementParental Involvement

Teacher EffortTeacher Effort

Improve teaching and parental support

Improve teaching and parental support

Conceptual Framework


Agenda

• Three groups of control variableso Student characteristicso Teacher and director variableso School and regional level characteristics

• Randomization check using two-tailed t-tests• OLS regression with treatment status as dependent variable

• The trial was well randomized; no significant differences for variables hypothesized most closely associated with student achievement.

Evaluation Strategy

• General treatment effects: Generalized Least Squares regression with random errors clustered at the school level.

• Effects across student ability: Quantile regression at different points in conditional distribution of test scores, clustered on school level using bootstrapping (Chen, Wei and Parzen, 2003).

• Indirect treatment effects: Ordered logit regression with clustered standard errors on school level.

Evaluation Strategy


Agenda

• General impact between 0.217 and 0.259 standard deviations.• Impacts are generally significant at the 5% level.• Slightly higher impact for males than females.

General Treatment Effects

Treatment Effects per Grade

• All impacts are statistically significant at conventional levels.• Impact size between 0.200 and 0.346 standard deviations.• Larger impact for reading than mathematics.

Treatment Effects Grade 3

Impact across Student Ability• Do incentive awards exclusive benefit previously high-performing

students? (Leuven et al, 2010).


Agenda

Treatment Control ImpactTeacher was absent 3 days or less in the last 4 weeks 90.3% 75.6% 14.68***

Treatment Control ImpactTeacher always or almost always asks students if they understand topic

89.0% 78.3% 10.65*

Teacher attendance

Teacher effort

Treatment Effects on Teachers

Treatment Control ImpactParents have attended a school meeting this year 93.5% 88.0% 5.57*

Parental Involvement

Treatment Effects on Parents

1. Incentive program had a sizeable, significant and robust impact on student performance, raising overall test scores by 0.237 standard deviations.

• Comparable impact to other education programs.

2. Overall impact for mathematics > reading

3. Overall impact for males > females

Conclusion

4. Impact for financial incentives > material incentives• Relevance of prize

5. Impact for low, average and high-performing students• Successful mix of incentives (‘most improved’ and ‘high grades’)

6. Additional treatment effects for teacher effort and parental support.

Conclusion

Brian [email protected]

Questions

Documents

Performance Based Incentives for Learning in the Mexican Classroom Brian Fuller, MPA, Foundation Escalera Victor Steenbergen, MPA Candidate, London School