Teacher Incentives

8/11/2019 Teacher Incentives

http://slidepdf.com/reader/full/teacher-incentives 1/49

Teacher Incentives

Paul Glewwe, Nauman Ilias, and Michael Kremer

Abstract

Advocates of teacher incentive programs argue that they can strengthen weak incentives, whileopponents argue they lead to teaching to the test!" #e find evidence that e$isting teacher

incentives in Kenya are indeed weak, with teachers a%sent &'( of the time! #e then report on arandomi)ed evaluation of a program that provided primary school teachers in rural Kenya withincentives %ased on students* test scores! +tudents in program schools had higher test scores,significantly so on at least some e$ams, during the time the program was in place! Ane$amination of the channels through which this effect took place, however, provides littleevidence of more teacher effort aimed at increasing longrun learning! -eacher attendance did notimprove, homework assignment did not increase, and pedagogy did not change! -here is,however, evidence that teachers increased effort to raise shortrun test scores %y conducting moretest preparation sessions! #hile students in treatment schools scored higher than theircounterparts in comparison schools during the life of the program, they did not retain these gainsafter the end of the program, consistent with the hypothesis that teachers focused on

manipulating shortrun scores! In order to discourage dropouts, students who did not test wereassigned low scores! Program schools had the same dropout rate as comparison schools, %ut ahigher percentage of students in program schools took the test!

.epartment of Applied /conomics, 0niversity of Minnesota! /mail1 pglewwe2apec!umn!edu

3ompetition /conomics, Inc!, #ashington, .3! /mail1 nilias2competitioneconomics!com

.epartment of /conomics, 4arvard 0niversity5 -he 6rookings Institution5 3enter for Glo%al .evelopment

N6/7! /mail1 mkremer2fas!harvard!edu

#e would like to thank 7achel Glennerster, /d Kaplan, 8anina Matus)eski, and 3ourtney 0m%erger for very helpfulcomments and assistance! #e are especially grateful to +ylvie Moulin and 7o%ert Namunyu for outstanding work inthe field and to /mily 9ster for outstanding research assistance in the 0!+! #e thank the #orld 6ank and theMacArthur :oundation for financial support!

'

mailto:[email protected]

mailto:[email protected]



1. Introduction

-eacher incentive programs have en;oyed growing popularity! In the 0nited +tates, a

num%er of teacher incentive programs have %een introduced in the past decade, generally

offering annual merit pay on the order of <'( to ='( of an average teacher>s monthly salary

?American :ederation of -eachers, &'''@!< 0nder the No 3hild eft 6ehind ?N36@ act, passed

in &''<, poorly performing schools face sanctions across the 0nited +tates! Israel has provided

incentives to teachers %ased on students> scores ?avy &''&a, %@ and a #orld 6ank program in

Me$ico will provide performance incentives to primary school teachers!

Advocates of incentive pay for teachers note that teachers currently face weak incentives,

with pay determined almost entirely %y educational attainment, training, and e$perience, rather

than performance ?4ar%ison and 4anushek, <BB&5 4anushek et al!, <BBC5 4anushek, <BBD5

ockheed and Eerspoor, <BB<@, and argue that linking teachers> pay to students> performance

would increase teacher effort!

9pponents of test score%ased incentives argue that since teachers* tasks are multi

dimensional and only some aspects are measured %y test scores, linking compensation to test

scores could cause teachers to sacrifice promoting curiosity and creative thinking in order to

teach skills tested on standardi)ed e$ams ?4olmstrom and Milgrom, <BB<5 4annaway, <BB&@!

In many developing countries, incentives for teachers are even weaker than in developed

countries! -hus, for e$ample, in our data teachers are a%sent from school &'( of the time and

a%sent from their classrooms even more freFuently! #ork in progress suggests that a%sence rates

among primaryschool teachers are &D( in 0ganda, &( in India, <D( in /cuador and <( in

Peru! ?3haudhury et al!, &''@!

< /$amples include programs in 7hode Island in <BBB, .enver in <BBB&''', .ouglas 3ounty, 3olorado %eginning

in <BB= and Iowa %eginning in &''< ?9lsen, <BBB5 /ducation 3ommission of the +tates, &'''@! A <BBB program in3alifornia offered a onetime award of H&,''' to teachers in underperforming schools whose students showedsu%stantial gains ?9lsen, <BBB@!

<



In environments with very weak incentives, it could %e argued that the key pro%lem is to

get teachers to show up at all! Given that most teaching in many developing countries is %y rote,

the risk of reducing efforts to stimulate creativity may seem remote! 9n the other hand, if

incentive systems are very weak, schools could potentially respond to test score%ased incentives

in more pernicious ways than teaching to the test! :or e$ample, they could deli%erately force

students to repeat grades or even drop out in order to raise average scores on the e$am!

-his paper e$amines the issue of teacher incentives in Kenya, where some local school

committees strengthen teacher incentives %y providing %onuses to teachers whose students

perform well on e$ams! #e report on a randomi)ed evaluation of a program along these lines

that provided incentives to teachers in ' rural schools %ased on the average test score of

students already enrolled at the start of the program! +tudents who did not take the test were

assigned very low scores so as to discourage dropouts! /ach year the program provided pri)es

valued at up to =( of typical monthly salary to teachers in grades = to C %ased on the

performance of the school as a whole on the Kenyan government*s districtwide e$ams! -his

ratio of pri)e to salary was similar to that used in typical 0!+! incentive programs!

.uring the life of the program, students in treatment schools were more likely to take

e$ams, and scored higher, at least on some e$ams! An e$amination of the channels through

which this effect took place, however, provides little evidence of more teacher effort aimed at

preventing dropouts or increasing longrun learning! .ropout rates did not fall, teacher

attendance did not improve, homework assignment did not increase, and pedagogy did not

change! -here is, however, evidence that teachers increased efforts to increase the num%er of

pupils taking tests in the short run and to raise shortrun test scores! 3onditional on %eing

enrolled, students in treatment schools were more likely to take tests, and teachers in treatment

schools were more likely to conduct test preparation sessions! #hile students in treatment

schools scored higher than their counterparts in comparison schools during the two years that the

&



program operated, they did not retain these gains after the end of the program, consistent with a

model in which teachers focused on manipulating shortrun scores!

-here is evidence that teachers learned how to ad;ust to the system over time! -est

preparation sessions increased from the first to the second year of the program, as did the gap

%etween treatment and comparison schools in e$am participation rates and overall test scores!

1.1 Related Literature

A num%er of earlier papers e$amine the impact of linking teacher pay to students* test

scores! avy ?&''&a@ finds that an Israeli program providing teachers individual cash pri)es for

increases in student test scores on a highschool matriculation e$am increased high school

matriculation e$am rates from =&( to =!(! At D'( to ''( of the average monthly salary, the

pri)es given in this case were much larger than those in most teacher incentive programs in the

0!+! avy ?&''&%@ finds that rewarding Israeli teachers %ased on school average performance

?rather than individual performance@ increased test scores and participation in matriculation

e$ams, %ut not the percentage of students receiving matriculation certificates!

8aco% ?&''&@ e$plores a 3hicago program in which students with low test scores were not

promoted to the ne$t grade and schools and teachers were put on pro%ation! 4e finds that the

program increased student achievement, although the improvement was larger in skill sets used

on the highstakes e$am! +ome schools manipulated scores %y putting more students in special

education classes! :iglio and #inicki ?&''&@ show that school districts in Eirginia increase the

num%er of calories in school lunches on days when highstakes tests are administered, thus

artificially inflating test scores! Koret) ?&''&@ finds that a teacher incentives program in

Kentucky had significant positive impacts ?'! to '!D standard deviations@ on the test used to



determine rewards for teachers %ut much smaller effects ?'!< to '!& standard deviations@ on

another test that was not tied to the rewards!

-his paper differs from earlier work in several ways! :irst, since %oth advocates and

opponents of teacher incentive programs agree that incentives can increase test scores, %ut

disagree a%out whether these higher test scores would %e due to increased overall teacher effort

or more teaching to the test, we measure not only how teacher incentives affect test scores, %ut

also how they affect different types of teacher effort! In particular, we e$amine teacher %ehavior

in the classroom and scores not only on e$ams to which incentives were linked, %ut also on other

e$ams given %oth contemporaneously with the program and after its conclusion! +econd, since

teacher incentive programs are likely to %e introduced in areas where teacher performance is

worse than e$pected, and since the introduction of teacher incentives may %e correlated with

other factors affecting teacher performance, it is difficult to econometrically identify the effect of

such teacher incentive programs! #e address this pro%lem %y e$amining a conte$t with random

assignment of schools to treatment and comparison groups! -hird, we e$amine teacher incentives

in a developing country conte$t!&

:inally, %y collecting panel data on teacher a%sence, we are

a%le to show that teacher a%sence is widespread, suggesting e$isting incentives are weak in the

conte$t we e$amine!

-he remainder of the paper is organi)ed as follows! +ection & sketches a simple

4olmstromMilgrom style model in which linking %onuses to test scores could potentially either

increase teaching effort or divert effort towards teaching to the test! +ection discusses primary

education in Kenya and argues that the high rate of teacher a%sence suggests current incentives

are inadeFuate! +ection = descri%es the teacher incentives program that we e$amined and the

process %y which schools were selected for the program! +ection reports the impact of the

& Kingdon and -eal ?&''&@ show that in private schools in India, teacher pay and student achievement are linked, %ut

they do not demonstrate specifically that linking pay to performance has any effect!

=



program on teacher outcomes, while +ection D reports the impact on student outcomes! +ection J

discusses how teacher %ehavior changed in response to incentives over time, and +ection C

concludes!

2. A Model of Productive and Signaling Effort

4olmstrom and Milgrom ?<BB<@ consider a model in which linking pay to o%serva%le

signals of performance can potentially lead employees to focus on tasks for which output is

easily measured and divert effort away from tasks for which output is difficult to measure! -hey

motivate their analysis using two main e$amples! In the first, linking teacher pay to test scores

may cause teachers to teach to the test rather than encourage creativity! In the second, employees

who are responsi%le %oth for producing output and for maintaining the value of an underlying

asset, such as a piece of eFuipment or a firm>s reputation, may neglect the long run value of the

asset if they are provided with strong incentives to focus on current output!

#e consider a model that com%ines elements of %oth 4olmstrom and Milgrom>s

motivating e$amples, and can %e considered a special case of their general model! -eachers can

e$ert two types of effort1 efforts to promote longrun learning and signaling effort," which

improves scores in the shortrun %ut has little effect on longterm learning! /mployers o%serve

only test scores! In particular, we assume test scores depend %oth on underlying learning

?produced %y teaching effort over time@ and contemporaneous signaling effort! +uppose that

,@?!!!@,,? &< t t t t t t seee LT ε γ ++= −− where t T denotes test scores during period t , L denotes

student learning, t e denotes teaching effort on longrun learning during period t , s denotes

signaling effort, and ε is a random shock! -hus, teaching effort produces longrun improvements

in students* understanding, while signaling effort produces only shortrun effects on test scores!



?-eaching effort can thus %e seen as uno%serva%le effort to maintain asset value in 4olmstrom

and Milgrom>s framework!@

Assume that teachers* utility is given %y @,? seC M U −= where M is teacher pay and C

is a utility cost that depends on %oth teaching and signaling effort! In this specification, e and s

can %e either su%stitutes or complements! :or e$ample, they could %e su%stitutes if there is a

fi$ed amount of time in the day that must %e allocated %etween them! 9n the other hand, they

could %e complements if there is a fi$ed cost to teachers of attending school at all!

+uppose teacher pay is BT M +=α ! If '= B , so pay is independent of performance,

teachers will choose effort in teaching and signaling such that the marginal product of each is

eFual to )ero! As noted %y 4olmstrom and Milgrom ?<BB<@, C 1 ?','@ and C 2 ?','@ may %e

negative, so some effort may %e e$erted even if 6 '! -eachers may care a%out their students, or

en;oy e$erting some effort even in the a%sence of performance incentives!

If the government or an NG9 makes a surprise announcement that pay will %e linked to

test scores for a single year, teachers will change %oth teaching and signaling effort to satisfy the

first order conditions implied %y the a%ove eFuations! +pecifically, teachers will e$ert teaching

and signaling effort such that1 @,? see

C B

e

L

∂

∂=

∂

∂ and @,? se

s

C B

s ∂

∂=

∂

∂γ ! If e and s are

complements in the utility function, or if utility is additively separa%le, then %oth types of effort

will increase! If they are su%stitutes in the utility function then incentives may increase one type

of effort at the e$pense of the other! -hus in this model, incentives could potentially either

increase or decrease teaching effort!

3learly, there is a continuum %etween e$erting effort on promoting longterm learning and trying to manipulate

shortrun test scores! -he e$treme of manipulating shortrun scores would %e actually cheating at the time of the test5less e$treme versions would include going over Fuestions from previous years> e$ams, and teaching testtakingstrategies such as guessing on multiple choice Fuestions! #ithin the category of promoting learning, teachers couldfocus narrowly on the curriculum to %e tested or could promote learning more %roadly! 9ne could imaginegenerali)ing this model to allow teachers to choose from a menu of activities, with varying components of true andsignaling effort, %ut results would presuma%ly %e similar!

D



In the model, if it were possi%le to cheaply and accurately monitor individual

performance on %oth tasks as part of an incentive program, then a wage contract could induce

teaching effort without inducing signaling effort! 4owever, while distinguishing teaching and

signaling effort would %e e$pensive and inaccurate at the individual level, particularly if tied to

an incentive program, there are potential ways to distinguish them empirically at the aggregate

level, at least if results are not tied to compensation! :irst, outside o%servers may %e a%le to

o%serve teachers> activities directly! :or e$ample, in Kenya, some schools conduct what are

known locally as preps"Le$tra test preparation or coaching outside of normal class time, often

during school vacations! 9ne could potentially interpret preps as including a higher rate of

signaling to teaching effort than ordinary classroom attendance! +econd, improved learning

should have a longrun effect on test scores, whereas under the model signaling has only a short

run effect!= -hus a finding that test score gains do not persist is consistent with the hypothesis

that the program led only to e$tra coaching specific to the test at hand! It is more difficult to

reconcile this result with the hypothesis of increased longrun learning! A third potential way to

distinguish efforts to increase longrun learning from test preparation activities is to check if test

scores improved primarily in su%;ects prone to memori)ation!

Note that under the model, parents and local communities may not o%;ect to teachers>

investing in shortrun test preparation, since students* prospects for further education and la%or

market success depend on test scores as well as underlying learning! -est preparation, however,

is assumed to %e socially wasteful, in that it reFuires teacher effort %ut does not improve the

underlying learning that affects total output in society!

3. ac!ground

= In practice, some types of signaling may have a longrun effect on test scores! :or e$ample, helping students cheat

will only increase scores in the short run, %ut teaching students to guess on multiple choice e$ams or %etter allocatetheir time could raise scores on other tests and in the long run!

J



-his section provides some %ackground on primary school teaching in Kenya and

provides evidence that teacher a%sence is widespread in the area of the study!

-eacher hiring, firing, and transfer decisions in Kenya are made centrally %y the Ministry

of /ducation! 4iring is %ased primarily on academic Fualifications!

+alaries are set through

collective %argaining %etween the government and the politically powerful Kenyan National

0nion of -eachers ?KN0-@! In <BBJ, the starting salary for teachers was Ksh ,<J ?HCC@ per

month, and a typical teacher in our sample earned appro$imately Ksh J,''' ?H<<B@ per month!D

-aking into account generous %enefits, total teacher compensation was appro$imately H&,''' a

year, or more than five times annual G.P per capita!J

-eachers> salaries depend primarily on education and e$perience! -here is little

opportunity for performance%ased promotion or increases to salary! -eachers have strong civil

service and union protection and are difficult to fire! In some cases teachers who have performed

very %adly are transferred to less desira%le locations, while the government may look more

favora%ly on reFuests for transfers to more desira%le postings or to home areas from teachers

who perform well!

Although incentives provided to teachers %y their employer are weak, every school is

supposed to have a parent committee, and these committees sometimes provide gifts for teachers

when schools perform well on the national e$ams! +imilarly, communities sometimes refuse to

allow e$ceptionally %ad teachers to enter the school, thus putting pressure on the Ministry of

/ducation to arrange a transfer for the teacher! 4owever, only a minority of school committees

provide supplemental %onuses, and school committees typically only attempt to influence the

national authorities in e$treme situations!

Primary school teachers in Kenya typically have completed two years of teacher training %eyond secondary school!

A small num%er of teachers were hired under an older system in which primary teachers had only a J th gradeeducation and two years of teacher training!D -his is assuming an e$change rate of C!J shillings per dollar, the <BBJ dollarshilling e$change rate!J Authors* calculations %ased on value of housing allowance and other %enefits! -his is calculated from salary scales

and represents a salary for a teacher with average education and e$perience in a sample of schools in the area!

C





A%sences seem fairly %roadly distri%uted among the population of teachers rather than

primarily accounted for %y a su%set of teachers with very high a%sence rates! -o see this, it is

helpful to consider the empirical distri%ution of a%sences, as well as two ways of cali%rating the

underlying distri%ution of a%sences across teachers that correct for the additional dispersion in

a%sence rates across teachers that is created %y sampling error!

:igure < and -a%le < show the percentage of teachers who were a%sent )ero times out of

eight visits, one time out of eight visits, twice out of eight visits, and so on! 4owever, it is

important to note that with only a few visits to each school, the dispersion of a%sence rates in the

empirical distri%ution will e$aggerate the underlying dispersion of pro%a%ilities of attendance

among teachers! :or e$ample, suppose there were only two visits, and that one Fuarter of

teachers were a%sent on %oth visits, half were a%sent once, and a Fuarter were present during

%oth visits! Note that this hypothetical data would %e consistent with the hypothesis that all

teachers attend half the time %ut is inconsistent with the hypothesis that a Fuarter of the teachers

have a )ero pro%a%ility of a%sence, half attend half of the time, and one Fuarter are always

present! In general, due to sampling error, the variance of empirically o%served a%sence rates

across teachers will %e greater than the underlying variance of pro%a%ilities of a%sence! As the

num%er of visits to each school increases, the empirical distri%ution of o%served teacher a%sence

rates converges to the underlying distri%ution!

-o correct for this pro%lem and assess the e$tent to which teacher a%sences are

concentrated, we cali%rate two more structural models of a%sences!C -he first is a nonparametric

model in which we assume the population of teachers consists of groups, with proportions α<

through α and pro%a%ilities of attendance p< through p! -here are nine independent unknowns in

this model1 α< α= and p< p! -he model is therefore identified with eight visits, since this

C #e are grateful to /mily 9ster for outstanding work cali%rating these two models!

<'



gives nine possi%le outcomes ?ranging from never attending to attending at all visits@! #e

o%serve a proportion $' of teachers who are never present, $< who are present once, and so on!

#e solve a system of eFuations in which the e$pected proportion of teachers present for 8C

visits ?%ased on α< α= and p<p@ is set eFual to the actual proportion $8! -he resulting group

si)e and attendance pro%a%ilities are graphed in :igure < and detailed in -a%le <! -he median

teacher in this model is a%sent a%out <B( of the time!

-he second model imposes more structure, %ut allows for a continuous distri%ution of the

pro%a%ility of attendance! :or this model we assume that each teacher>s pro%a%ility of attending

p that is drawn from a %eta distri%ution, << @<?

@?@?@? −− −

Γ Γ +Γ β α

β α β α p p ! #ith eight visits, the

pro%a%ility of o%serving a teacher with pro%a%ility of attendance p at school 8 times is %inomial1

J J p p

J p J v

−−

== C@<?C

@OPr? !

Ma$imum likelihood analysis yields the parameters of the %eta distri%ution most

consistent with the data o%served! -he likelihood function for 8 attendances is1

<<



dp p p

J

p p J J

−

−

ΓΓ

+Γ −−−

∫

C<

'

<< @<?C

@<?

@?@?

@? β α

β α

β α ,

or eFuivalently,

∫ −−+−+ −−Γ Γ

+Γ <

'

<C< @<?@PC?P

PC

@?@?

@?dp p p

J J

J J β α

β α

β α

@?@<?@<?@?@?

@?@?@<?@?

N J N J

J N J N

++Γ +−Γ +Γ Γ Γ

−+Γ +Γ +Γ +Γ

β α β α

β α β α !

-he %eta distri%ution of attendance pro%a%ilities implied %y the M estimates of α and β

is graphed in :igure < and detailed in -a%le <! -he median teacher is a%sent a%out <=( of the

time in this cali%ration!

6oth models yield similar results! Although a few teachers are rarely present, the ma;ority

of a%sences appear to %e due to those who attend %etween '( and C'( of the time! In addition,

in %oth models ?although more so in the %eta distri%ution model@ a large minority of a%sence is

actually due to those who attend more than C'( of the time, as can %e seen in -a%le <!

-he widespread nature of a%sences suggests that teachers who are a%sent freFuently may

not %e violating a social norm! -his is not, however, an implicit contract in which the

government pays teachers only for parttime work! As noted a%ove, including %enefits, Kenyan

teachers are paid up to times average per capita income! -here is su%stantial unemployment

among people who would %e Fualified to %ecome teachers and Fueuing for teaching ;o%s! -he

government imposed a hiring free)e in <BBC so several cohorts of graduates of teacher training

colleges are unemployed! +ome are working for free in the hope of eventually o%taining a paying

;o%!

<&



:urther evidence of weak incentives is that while teachers are a%sent from school a%out

&'( of the time, they are a%sent from their classroom much more often! In <BBC, the time of

teacher arrival was not recorded during classroom visits to o%serve pedagogy, %ut in <BBB, when

it was, =( of the time teachers never arrived in the classroom!

7ecalling the model in +ection &, the fact that we have information on %oth teacher

attendance and preps provides us the opportunity to run a simple regression of test scores on

teacher attendance and test preparations sessions! -o the e$tent that one is willing to interpret

this causally, this regression suggests that the marginal product of test preparation sessions,

which can plausi%ly %e interpreted as signaling effort, may %e much higher than that of teacher

attendance, a plausi%le measure of teacher effort!B -he evidence, %ased on visits conducted in

<BBC, suggests that teachers who attend school &' percentage points more of the time have

students who score '!'<< standard deviations higher ?standard error '!'<&@!<' #e do not have

information on e$actly how long teachers spent on coaching, %ut only know whether they taught

during each of the three vacation periods over the year or outside of normal school hours during

each of the three terms! 4owever, those who report coaching in one additional time period have

pupils who score '!'== standard deviations higher ?standard error '!''CJ@! It is somewhat

difficult to draw conclusions a%out the relative marginal productivity of coaching and teaching

given that we do not have data on how many days of coaching took place! 6ased on discussions

with teachers a%out the freFuency of coaching, however, it seems likely that, if interpreted

causally, our point estimates would imply that the marginal test score effect of a day of coaching

is an order of magnitude or more greater than that of a day of school attendance!

9f course, it is not clear that these correlations should %e interpreted causally!

Nonetheless, the hypothesis that test preparation activities can raise test scores is consistent with

B .ata on teacher attendance and test scores are descri%ed %elow!

<' Note, however, that these estimates will %e su%;ect to attenuation %ias since teacher attendance is estimated %ased

on random visits and thus measured with su%stantial error!

<



evidence from the 0!+! on the effects of commercial test preparation! -hese studies often show

gains of '!< to '!= standard deviations on admissions tests, even though most 0!+! admission

tests are supposed to measure aptitude, rather than achievement, and thus to %e difficult to study

for! /$tra coaching raised scores on an achievement test %y '!& standard deviations ?6angert

.rowns et al.,<BC@! In an environment in which teacher pay is not linked to test scores, it seems

plausi%le that teachers might make only limited efforts at test preparation, leaving the marginal

test score product of test preparation su%stantially greater than that of teacher attendance!

-he hypothesis that the marginal test score impact of a day of preps is larger than a day of

teaching is reasona%le if one assumes that incentives are initially weak! .enote e0 and s0 as the

levels of e and s when 6'! It seems reasona%le that teachers> utility from putting in teaching

effort may %e greater than their utility from signaling ?although there may %e some %enefit from

signaling if it enhances the teacher>s reputation with students and parents@! If this is true, then

@,?@,? '''' se s

see

L

∂

∂<

∂

∂ γ !

". Progra# $escri%tion

As noted a%ove, some school committees in Kenya provide %onuses to teachers whose

students perform well in e$ams! -his paper evaluates a program conducted %y International

3hristeli;k +teunfonds ?I3+@, a .utch NG9, in 6usia and -eso districts of #estern Kenya! -he

program offered schools the opportunity to provide gifts to teachers if students performed well! It

provided pri)es to teachers in grades = to C %ased on the performance of the school as a whole on

the district e$ams in each year! All teachers who taught these grades were eligi%le for the pri)e!<<

I3+ awarded pri)es in two categories1 Q-opscoring schoolsQ and QMostimproved

schools!Q +chools could not win in more than one category! Improvements were calculated

<< -eachers of lower grades were not a part of the competition, %ecause there were no district e$ams for those

classes! -hey received a lantern as a token pri)e, whether or not they %elonged to a winning school

<=



relative to performance in the %aseline year! +ince the results of the district e$ams were not

availa%le for <BBJ, the scores for <BBD were used as the %ase to measure improvements!

?4enceforth, we will refer to the last preprogram year for which we have data as year ',<& the

first ?<BBC@ and second ?<BBB@ years of the program as years < and & respectively, and the post

program year ?&'''@ as year !@ In each category, three first pri)es were awarded, three second

pri)es were awarded, three third pri)es were awarded, and three fourth pri)es were awarded!

-hus, overall, &= out of the ' schools participating in the program received pri)es of some type,

and teachers in most schools should have felt that they had a chance of winning a pri)e! +ince

6usia and -eso districts had separate district e$ams, pri)es were offered separately in each

district in proportion to the num%er of schools in the district!

/ducation e$perts generally are more sympathetic to school%ased incentives than to

individual%ased incentives since they feel these are more conducive to cooperation among

teachers ?7ichards and +heu, <BB&5 4anushek, <BBD@! In order to encourage cooperation among

teachers within schools and to avoid creating incentives for teachers to sa%otage each other*s

work, I3+ pri)es were %ased on the performance of all of the grade = to C pupils in the school,

with each su%;ect weighted eFually, rather than on a teacher%yteacher %asis! -hus, every teacher

in grades = to C in the winning schools received the same pri)e! +etting pri)es at the school level

could potentially allow freeriding within the teaching staff! 4owever, teachers are in a relatively

good position to monitor each other*s performance, particularly on easily o%served aspects such

as attendance! Moreover, since teachers can o%serve each other*s work at high freFuency, they

were in a repeated game with each other! +ince the typical school in the sample had only &''

students and <& teachers, a%out half of whom taught in the upper grades, teachers should have

%een a%le to enforce cooperation within the school!

<& -his may %e either <BBD or <BBJ depending on the type of data!

<



In order to discourage schools from forcing weaker students to repeat, drop out, or not

take the e$am, students who did not take the e$am were assigned low scores! Multiple choice

e$ams were used in all su%;ects other than /nglish! +tudents who did not take these e$ams were

assigned a score of <, whereas students who simply guessed would have o%tained a score of &

on average! 9n the /nglish essay e$am component, students who did not take the test were

assigned a score of )ero! In order to discourage schools from recruiting strong students to take

the e$ams, only students enrolled in school as of :e%ruary <BBC were included in the

computation of the school mean score!

Pri)es ranged in value from &< to =( of typical teacher monthly salaries! < -his is

compara%le to merit pay programs in the 0nited +tates and to the program avy e$amined in

Israel! :or e$ample, the <BBB= .allas merit pay program, which was also %ased on school

wide performance, awarded H<''' annual %onuses, which were B( of an average monthly

salary of -e$as teachers that year, and presuma%ly a somewhat lower percentage of salaries for

teachers in .allas ?3lotfelter and add, <BBD5 American :ederation of -eachers, &'''@!

+imilarly, a <BBB 7hode Island program awarded H<,''' annual %onuses, worth a%out &( of

monthly salary ?9lsen, <BBB5 American :ederation of -eachers, &'''@! 9ther programs in

3olorado award %etween <'( and '( of monthly salary in merit%ased annual %onuses

?/ducation 3ommission of the +tates, &''<@! In Israel, the annual %onuses awarded ranged from

H&' to H<''', which amounts to C( to ( of the average teacher>s monthly salary!<=

+chools were offered the opportunity to participate in this program in :e%ruary <BBC and

all accepted! ?-he academic year in Kenya runs from 8anuary to Novem%er!@ -he pri)es were

awarded during a ceremony held in Novem%er of each year, and all the schools in the program

< /ach winning school also received a %riefcase for the headmaster, a wall clock, a time keeping clock, and a

%ell!

<= -he average teacher salary in Israel is H','''! Assuming that a teacher works <' months per year implies a

monthly salary of H'''!

<D



were invited to attend! In order to discourage teachers from arranging transfers into treatment

schools in order to %e eligi%le for the program, eligi%ility was restricted to teachers who were

employed in the school as of March <BBC! As discussed %elow, teacher entry and e$it rates did

not differ significantly %etween treatment and comparison schools!

#hen the program was originally announced in :e%ruary <BBC, it was scheduled to run

for a single year and teachers were informed of this! ater the program was e$tended for an

additional year! 6ecause the NG9 had conducted other programs in the area, we think most

teachers found the commitment to provide the pri)es credi%le! 4owever, it is possi%le that

teachers did not react fully to the program until after they had actually seen the pri)es awarded

during the first year! -he awards ceremony presuma%ly increased the salience of the program! It

is not clear whether teachers e$pected the program to continue longer than the NG9 had

promised, since the NG9 tries to %e conservative in announcing %enefits it will provide to avoid

creating overly high e$pectations!

-he short duration of the program and its surprise introduction allowed several elements

to %e included that might not otherwise have %een possi%le in a permanent program! 4alf the

pri)es were %ased on improvements in performance! -eachers in many lower performing schools

may have felt these were the only pri)es for which they could realistically compete! -he

incentives created %y these pri)es for improvement were presuma%ly larger than they would have

%een in a permanent program in which teachers who increased their scores in one year would

find it harder to win the pri)e in su%seFuent years! Moreover, the short duration of the program

made it possi%le to %ase pri)es on the test scores of all students originally enrolled in the school,

which allowed the program to discourage repetition and dropouts! 0nder a permanent program,

schools might have incentives to restrict admissions to school to students who they %elieve

would score well on e$ams! +imilarly, the program was restricted to the original teachers at this

school! It thus reduced the opportunity for teachers to seek assignment to schools with high

<J



scoring pupils due to nonteacher factors such as parental characteristics! All these factors imply

that the incentives provided %y the program were stronger than could have %een provided with

the same e$penditure under a permanent program! 9n the other hand, teachers might have had

stronger incentives to promote longrun learning if they e$pected the program to continue

indefinitely!

9verall, the conte$t seems particularly favora%le for a teacher incentives program1 the

level of teacher a%sence suggests that teacher effort was an issue in schools5 there was relatively

little scope for diverting teacher attention away from creativity and towards teaching to the test5

and the short duration of the program made possi%le a design that did not encourage

manipulation of the student %ody or the set of teachers in the school!

".1 School Selection

-he ' schools in the program were selected from a group of <'' schools that had

originally %een selected %y the Ministry of /ducation %ecause they were considered to %e

particularly in need of assistance, %ut had not participated in an earlier #orld 6ank program

which provided te$t%ooks to the schools ;udged to %e in greatest need! -hese schools scored

somewhat worse than average for the area %efore I3+ %egan working with them! I3+ had

provided te$t%ooks or modest grants to these schools %efore the inception of the teacher

incentive program! +chools were num%ered according to the year they received te$t%ooks or

grants from I3+, their geographic division, and their alpha%etical order! #ithin each group, the

odd num%ered schools were chosen to participate in the teacher incentive program! 6y

construction, the odd and even num%ered schools were split in compara%le proportions across

<C



6usia and -eso districts, geographic divisions within these districts, and across schools which

received te$t%ooks and grants in different years!

-he ' evennum%ered schools that serve as the comparison group for this evaluation

participated in another program that was designed to improve preschools %y providing training,

materials, and salary supplements conditional on preschool teacher attendance! Preschool

teachers, unlike the primary school teachers, are semivolunteers who are not hired or paid %y the

central Ministry of /ducation, %ut instead are hired locally %y parents* committees and receive

only contri%utions from parents, which are often irregular! 0nlike primary school teachers, pre

school teachers often have no formal training and do not %elong to the Kenya teachers union!

+ince preschool and primary school teachers are Fuite distinct, and since in any case the pre

school program had little effect on performance in the preschool classes, it seems unlikely that

this program affected outcomes in grades = to C during the time period we e$amine!

&. I#%act of Incentives on Teachers

-his section e$amines the impact of the program on teacher %ehavior %y comparing

outcomes %etween treatment and comparison schools! It is worth noting, however, that

interviews of teachers in treatment schools a%out their satisfaction with the program suggest that

teachers in participating schools liked the program! In particular, in the middle of the second year

of the incentives program, the headmaster and three other teachers were interviewed in each

program school! All teachers interviewed supported the idea of motivating teachers %y providing

them with incentives! Most reported a change in school activities and teacher attitudes %ecause of

the program! C( reported that pri)es were ;ustly awarded in <BBC! J( of teachers in program

schools reported an increase in homework assignment due to the program, DJ( reported an

increase in cooperation among teachers, and CC( reported an increase in preps!

<B



&.1 Teacher Assign#ent

-he incentives program was designed so as not to provide incentives for teachers to ;oin

program schools! 9nly teachers who were already assigned to an incentive school as of March of

the first year were eligi%le for a pri)e! 4owever, the program could potentially have reduced the

e$it rate of teachers from the incentive schools!

In fact, the e$it rate was not significantly different %etween program and comparison

schools ?-a%le &, 3olumns < and &@!1& -he entry rate was higher in the incentive schools for the

first year of the program, and lower for the second, although in neither case was the difference

statistically significant ?-a%le &, 3olumns and =@! 1'

#e also considered the possi%ility that teachers in treatment schools in lower grades

would attempt to transfer into teaching higher grades, even though this would not actually make

them eligi%le for the program! -here is no evidence of differential transfers across grades! In

treatment schools J!=( of teachers transfer from a nonincentive grade to an incentive grade

during the program5 in comparison schools J!( do!

&.2 Teacher Attendance

-eacher attendance was not affected %y the incentive program! In the year prior to the

esta%lishment of the program, each of the <'' schools was su%;ect to two random, unannounced

visits at which the presentRa%sent status of each teacher in grades = to C was recorded! +imilar

visits were made five times in year < and three times in year &!1( :or each teacher in each year, an

< All regressions in this paper allow for schoollevel random effects to take account of the possi%ility that there

may %e correlation %etween error terms for students or teachers in the same school! Note that in the random effectregression framework the coefficient on the constant term is not e$actly eFual to the mean of the omitted category!

<D -he transfers include voluntary transfers due to family reasons ?such as marriage@ or involuntary transfers such as

disciplinary actions against teachers or staff %alancing needs ?to replace teachers who retire, die, or move!@

<J +ome visits did not take place, for e$ample due to vehicle %reakdowns! <!== visits were made to the average

school in year ', =!JC in year <, and &!B in year &! #e focus on teacher a%sence data %ased on visits to schools,

&'



attendance rate was computed as the proportion of visits during which the teacher was present!

Note that teachers were recorded as present if they were at the school, even if they were not

teaching when the visit took place! :ollowing standard Intentionto-reat ?I--@ methodology, the

sample included only those teachers who were assigned to program or comparison schools in

year '! Any teachers who changed schools %etween year ' and year < or %etween year < and year

& were classified with their initial schools!1)

Prior to the program, schools that would later %e selected to %e program schools have

slightly higher teacher attendance, although the difference was insignificant ?-a%le , 3olumn

<@!1* In year < of the program, teacher attendance was actually slightly lower in the incentive

schools, and in year & the attendance was slightly higher in incentive schools ?-a%le , 3olumns

& and @, although %oth coefficients are insignificant and Fuite small!2+

&.3 ,o#e-or! Assign#ent

:or a random su%set of students in grades = to C for each school, information was

collected from the students on whether they were assigned any homework on the previous day!

In general, homework assignment was much more common in the higher grades! In <BBJ, <=(

of grade = students report having homework assigned the previous day, versus =( of grade C

students! -reatment schools assigned slightly more homework than comparison schools prior to

the program, although the difference is far from significant ?-a%le =, 3olumn <@! After the launch

rather than on official school logs, %ecause school logs are often not filled out! 4owever, schoollog data alsosuggest no effect of the program on a%sence!

<C -his could only %e done for those teachers who switched schools and remained in the sample of <'' schools!

+ince there are no data on the teachers who switched to other schools, they were dropped from the analysis!

<B -he results here are ro%ust to a specification in which each visit is treated as a %inary opportunity for attendance

and month of visit is controlled for!

&' 7esults are also similar when lower primary school teachers are used as a control in a regression in which

attendance of all the teachers in the schools is regressed on a dummy for the program, on whether they are a lower primary teacher or an upper primary teacher, and on an interaction term!

&<



of the program, treatment schools assigned slightly less homework, although the gap was

insignificant %oth in levels and in differences ?3olumns & and @!

&." Pedagog

-eacher %ehavior was not significantly different %etween the incentive and comparison

schools! -rained o%servers o%served each teacher annually, spending one period in a class and

recording teacher %ehavior on a num%er of measures, including %oth o%;ective information a%out

what the teacher was doing and su%;ective impressions a%out their energy level and caring for the

students! #e e$amined a wide variety of pedagogy measures, and the results for two o%;ective

measures ?%lack%oard use and teaching aid use@ and two su%;ective ones ?teacher caring and

energy@ are presented here!

-here was no significant difference in pedagogy %etween the incentive and comparison

schools for any of the classroom o%servations prior to the program ?-a%le , 3olumn <@! #e also

find no significant difference during the intervention period %etween the two school groups in

any of the pedagogical practices ?3olumns & and @! -he point estimates are close to )ero for

each o%servation type! -he differenceindifference estimates shown in the last two columns

were computed at the schoolgrade level since it was not possi%le to match individual teachers

across o%servations years! -hese estimates are also close to )ero for every o%servation type!

&.& Test Pre%aration Sessions

Incentive schools conducted more preps than comparison schools! 4eadmasters in each

school provided information on whether there were any preps for grades = through C in si$ time

periods during the yearLthe three school vacations ?April, August and .ecem%er@ and out of

school hours during each of the three terms!

&&



Prior to the program, incentive schools were slightly less likely to offer preps ?-a%le D,

3olumn <@, %ut after the introduction of the program, treatment schools started to conduct more

preps ?3olumns & and @! -hey were =!& percentage points more likely to conduct preps in the

first year and J!= percentage points more likely in the second, with the latter estimate %eing

significant at the ( level! -he results shown a%ove are driven primarily %y preps over

vacations, as can %e seen %y the stronger results in the lower panel of -a%le D!

'. The I#%act of Incentives on Students

3onsistent with the hypothesis that teachers responded to the program primarily %y

seeking to manipulate the varia%les determining pri)e allocation rather than %y increasing efforts

to promote longrun learning, the program had little impact on dropout and repetition rates, %ut

increased student participation in e$ams! .uring the period the program was in place, student

scores increased, significantly so on some test measures! -here is some suggestive evidence that

the effect was larger in the su%;ects more vulnera%le to coaching! After the end of the program

the effect on test scores did not persist! +tudents who had %een in program schools during the

program scored no higher than their counterparts who had %een in comparison schools! 6elow

we discuss the program impact on dropout and repetition ?su%section D!<@, review the structure of

Kenyan e$ams ?su%section D!&@, and discuss program impact on e$am participation ?su%section

D!@ and test scores ?su%section D!=@!

'.1 $ro%out and Re%etition

&



As mentioned a%ove, schools were penali)ed for those students who did not take the test!

4owever, dropout and repetition rates in incentive schools were not significantly different from

those in comparison schools! .ropout dummy varia%les were set eFual to one if a student

enrolled in the previous year did not continue schooling in the current year! 7epetition dummy

varia%les were set eFual to one if the student repeated the same class in the following year!

.ropout rates were insignificantly higher in treatment schools ?-a%le J, 3olumns < and &@, while

the repetition rate was insignificantly lower in incentive schools ?-a%le J, 3olumns and =@!

'.2 The /enan E0a#s

Incentives for teachers were %ased on their students> performance on the district e$ams,

which are administered in seven su%;ects1 /nglish, Math, +cience, +wahili, Geography4istory

3hristian 7eligion ?G!4!37!@, Arts3raftsMusic ?A!3!M!@, and 4ome +cience6usiness

/ducation ?4+!6/!@! +tudents in grades = through C take these e$ams in 9cto%er of each year!

Participation is incomplete since students have to pay a fee of <&' +hillings ?0+ H&@ to

participate in the e$ams! +ince the rules of the incentive program stipulated that any student who

did not take the district e$ams would %e assigned lower scores than students could o%tain %y

guessing, teachers in program schools had incentives to encourage their students to take the

district e$ams!

In addition to the district e$ams, students also took K3P/ and I3+ e$ams, which were

not tied to the teacher incentive program! -hus, these tests provide us with an independent way

to assess the impact of the program!

-he Kenyan primary school leaving e$am ?K3P/@ is administered %y the Ministry of

/ducation each year to pupils completing grade C! It determines what secondary school, if any,

&=



students attend! I3+ also administered e$ams to students in grades through C!&< In year <, the

I3+ tests were administered in /nglish, Math, and +cience! In years & and , they were

administered only in /nglish and Math!&& #e have data on the district e$am scores and the K3P/

test scores from %oth intervention years ?< and &@ as well as the postprogram year ?@! :inally,

we have information on the I3+ tests for all participating schools for %oth of the intervention

years!& #e were una%le to o%tain the data for comparison schools in -eso district for year <!

3onseFuently, analysis of the district e$am scores for year < is restricted to schools in 6usia

district!

+ecurity is generally tight in Kenyan e$ams to prevent cheating! .istrict e$ams were

supervised %y three to four teachers from a neigh%oring school! 6ecause the K3P/ e$am

determines the future scholastic paths of the eighth grade students who take it, these e$ams are

even more strictly monitored and supervised!

In <BBC, one case was identified in which the headmaster of a program school colluded

with the teachers assigned to supervise the schools to allow cheating on the district e$am! -hat

school was disFualified from the competition in <BBC %ut was allowed to participate in <BBB!

-he scores from that school were not included in the analysis in <BBC, %ut their scores were

included in <BBB!

'.3 E0a# Partici%ation

&< :or a complete description of the I3+ tests and their administration, see Glewwe et al! ?&''<@! -hese tests were

also administered in <BBD and <BBJ!

&& In year ' and year <, all three e$ams were fairly similar in format and content! +eparate e$ams were given for

each grade and the e$ams had a multiplechoice format! 4owever, the I3+ e$ams in year & and were Qmultilevel,"with the same test given to all students in grades through C! /asy Fuestions in the %eginning of the test could %eanswered %y all students, including those in grade , while Fuestions %ecame progressively harder! -he finalFuestions were %ased on material seen only in the eighth grade! -hese e$ams also had a Qfill in the %lank," asopposed to a multiple choice format!

& In the postprogram year, &J of the <'' schools were involved in a deworming pro;ect! -his ena%led us to collect

I3+ test score data from that su%set of schools for that program year!

&



/$am participation is important %oth as an outcome in its own right and %ecause

differential e$am participation could complicate the interpretation of test score differences

%etween program and comparison schools! /$am participation rates were higher in program

schools than in comparison schools for the district e$ams ?on which the incentives were %ased@,

%ut were similar %etween treatment and comparison schools on the I3+ and K3P/ e$ams!

:ollowing standard I-- methodology, we restrict attention to only those students who

were enrolled as of :e%ruary <BBC ?year <@ and assign students who switched schools during the

program to their original schools!

6aseline participation on the district e$ams was around J'(5 on the I3+ and K3P/

e$ams it was around C(! In year <, participation in the district e$ams was higher %y D!'

percentage points in the incentive schools, a difference which is statistically significant at the

<'( level ?-a%le C, 3olumn <@! 6y year &, participation was higher %y <'!C percentage points in

the treatment schools, a significant difference at the ( level! ?-he main differences %etween

incentive and treatment schools in e$am participation were in grades = through J5 participation in

grade C on the district e$am was close to B'( prior to the program!@ In the postprogram year,

when there was no longer an incentive to encourage students to take the test, the participation

rate was actually &!&( lower in the incentive schools than in the comparison schools, though the

difference was not significant! In contrast, the participation rates in the I3+ and K3P/ e$ams,

which were not linked to teacher incentives, were similar %etween the two school groups in

either year ?3olumns & and @!

-heoretically, efforts %y teachers in treatment schools to increase e$am participation

could %ias scores either upwards or downwards, %ut availa%le evidence suggests the %ias is

likely downward! If teachers in the treatment schools put eFual effort into encouraging all

students who would not otherwise have taken the e$am to do so, then the addition of marginal

students would likely have dragged down average test scores, since poorer students are less

&D



likely to pay the fee to take the district e$am! 6ut if teachers selectively chose to concentrate on

convincing potentially highscoring students and their parents of the e$am>s importance, then

average scores in the treatment schools could %e potentially %iased upward! -o get a sense of the

direction of %ias, we compared prescores of test takers in treatment and comparison schools!

-he additional students who took the tests in the incentive schools had lower pretest scores, %ut

not significantly so!

'." Test Scores

-est scores on the district e$am are higher in the incentive schools during the years of the

program ?significantly higher in the second year@! +cores on other e$ams are also higher in

program schools during the duration of the program, %ut not significantly higher! In the post

program year, however, we see no persistence of the test score gains! -his provides some support

for the hypothesis that teachers are focusing primarily on e$tra coaching specifically for the test

in Fuestion! 9n the other hand, there is little evidence of outright cheating, and we did not see

any evidence that teaching effort actually decreased during the program, such as a decline in

longrun test scores for students e$posed to the program!

#e e$amine differences in test scores %etween the incentive and comparison schools

using a random effect regression framework that allows for the possi%ility that scores of students

in the same grade and same school might %e correlated due to uno%served characteristics of

teachers and headmasters! In particular, we use an error components econometric model with

school, grade, and su%;ect random effects1 random effects at the school level and at the level of

individual su%;ects and grades within the school!

?<@ ti;ks α=k .=i Sαk .i S S αCk .Ci S βk ps S uks S v ;ks S ei;ks

where k /nglish, Math, +cience, +wahili, G!4!37!, A!3!M!, and 4+!6/!

&J



/Fuation ?<@ com%ines data from several grades to measure the impact of the incentive

program for a given su%;ect! -he test score of student i in grade ; in su%;ect k in school s is ti;ks!

-he dummy varia%les . ;i indicate whether child i is in grade ;! -he varia%le p s is a dummy

varia%le that eFuals < if school s is an incentive school ?i!e! a school which was eligi%le for

teacher incentives@ and ' if not! -hus if the impact of the incentive program varies across grades,

βk will measure the ?weighted@ average impact of the program across all grades! -he error term

contains three components, the schoolspecific error term ?for su%;ect k@, uks, a gradespecific

term conditional on %eing in that school, v ;ks, and a child specific term, e i;ks!

#e estimate these eFuations using Generali)ed east +Fuares ?G+@ without imposing a

specific distri%ution ?e!g! the normal distri%ution@ on the error terms! -he regressions also include

controls for se$ and geographic division within 6usia! Given the prospective design of the

program, regressions without such controls are consistent, %ut adding controls to the regression

may increase the precision of the estimates! As a check, we ran regressions without the controls

for region and se$5 they yield similar results!

6ecause the units in which test scores are measured are ar%itrary, for each su%;ect and

grade com%ination we normali)e all test scores ?including district, I3+, and K3P/ tests@ %y

su%tracting the mean test score in the comparison schools and then dividing %y the corresponding

standard deviation for those schools! -hus, a student with a normali)ed score of '!< was '!<

standard deviations a%ove the mean score in the comparison schools! Note that for a normal

distri%ution, an increase of '!< standard deviation would move a student from the ' th percentile

to the =th! +ince the district test e$ams were different for 6usia and -eso districts, the

normali)ation of these tests was done separately for each district!

-here is no significant difference in preprogram scores on the district e$am %etween

incentive and comparison schools for any su%;ect or grade! 9verall, prior to the program,

&C



treatment schools scored almost the same as comparison schools! +ince grade & students were

not given district e$ams in <BBD, we used <BBJ I3+ tests as pretests where availa%le for students

who were in grade = in <BBC!

-he difference in test scores %etween treatment and comparison schools, and the

differenceindifference estimator of the effect of the program are shown in -a%les B ?for the

district e$am@ and <' ?for the I3+ and K3P/ e$ams@!&= -he difference estimates were calculated

using the full sample, i!e! all the students in grades = through C who took the e$ams in either

intervention year, while the differenceindifference estimates use a restricted sample, i!e! those

students who took e$ams in at least one su%;ect, in the preprogram year and at least one of the

intervention years! As discussed, we restrict attention to only those students who were enrolled

prior to the announcement of the program, as of :e%ruary <BBC ?year <@! #e also restrict the

sample to those students who did not repeat or dropout in any year, since students who repeated

would %e taking a different e$am!

Averaging across all su%;ects and grades, the difference estimate for the district e$am is

insignificantly negative in the first year of the program ?point estimate T'!'=@, %ut this could

potentially %e due to the differential e$am participation %etween treatment and comparison

schools on the district e$ams! -he differenceindifference estimate is positive, although not

significant! 6oth the difference and differenceindifference estimate of the treatment effect in

year & are significantly positive at '!<D and '!<B respectively! :or the I3+ e$ams, differences

are not significant in either year, and the point estimates across years are similar, at around '!'C

?-a%le <', Panel A@! :or the K3P/, the overall difference estimate in year < is '!<C and the

differenceindifference is '!<'=, with the former %eing significant at the ( level ?-a%le <',

&= Normali)ed district test scores from year ' ?<BBD@ were used as the K3P/ preprogram scores, since the K3P/

e$am is taken %y grade C students only!

&B





and shortrun teaching strategies! Primary school science also involves a fair amount of

memori)ation, %ut math presuma%ly reFuires less memori)ation!2'

:or the <BBB I3+ test, we have itemlevel data on whether students had correct answers

to individual Fuestions in their /nglish and math tests! -reatment students scored higher than

comparison students on the later part of the test %ut not on the early part of the test! -his may %e

%ecause teachers in treatment schools taught students to %etter allocate time across sections of the

test! ?4owever, given that %oth tests were designed so as to increase in difficulty as the test

progressed, another possi%ility is that the program induced increased teaching effort, %ut that this

was most effective at raising scores on more difficult Fuestions!@

Analysis of the item responses to detect cheating using techniFues developed %y 8aco%

and evitt ?&''&@ provide little evidence of suspicious strings of Fuestions for which all students

in the class got the Fuestion right, suggesting cheating was not widespread, although there was

one instance in which cheating was discovered at a program school! #ere cheating widespread

we likely would have seen much larger test score differentials on the district e$am ?on which

incentives were %ased@ than on other e$ams! Although the estimated program impact on the

district e$am is somewhat larger than on the I3+ e$ams, it is lower than on the heavily

monitored K3P/ e$ams, suggesting that cheating was not the main source of the program effect!

(. Ti#e%ath of hanges in Teacher ehavior

-here is evidence that teachers learned over time how %etter to take advantage of the

program! /stimated differences in preparation sessions %etween treatment and comparison

schools grew %etween the first and second year! Anecdotal evidence from the first year>s pri)e

&D I3+ staff mem%ers familiar with the curriculum suggested that G!4!37 and 4+!6/! reFuire the most

memori)ation, science reFuires a medium amount of memori)ation and /nglish, Math and +wahili reFuire the mostcreative thinking!

<



award ceremonies suggests that prior to these ceremonies some teachers did not fully understand

that having students drop out or not take the test would reduce their chances of receiving a pri)e!

After this e$perience, differences in e$am participation rates %etween program and comparison

schools rose presuma%ly %ecause teachers worked harder to persuade students to take the e$am!

-he test score gap %etween treatment and comparison schools was greater in the second year

than in the first year!

). onclusion

+tudents in schools with a teacher incentive program in were more likely to take e$ams

and had higher test scores in the short run! -here is little evidence, however, that teachers

responded to the program %y taking steps to reduce dropouts or increasing effort on stimulating

longrun learning! -eachers in program schools had no higher attendance rates or homework

assignment rates! Pedagogy and student dropout rates were similar across schools! Instead,

teachers in program schools increased test preparation activities and encouraged students

enrolled in school to take the test! :ollowing the end of the program, the test score difference

%etween students who had attended treatment and comparison schools disappeared, consistent

with a model in which teachers increased signaling effort %ut did not significantly increase effort

to promote longrun learning! +imilarly, dropout rates were no lower in program schools, %ut

conditional on %eing enrolled in school, students were more likely to take e$ams linked to

incentives!

It is worth noting several caveats ?as well as caveats to these caveats@! :irst, we cannot

rule out the possi%ility that a larger incentive program or teacherspecific incentives would have

induced not only increased test preparation, %ut also increased effort to improve underlying

learning! 4owever, at up to ='( of monthly income, the incentives were compara%le in

magnitude to those in most 0!+! programs and in the Israeli program analy)ed %y avy ?&''&%@!

&



Although the %onuses were a small percentage of yearly salary so the implied increase in daily

wages was modest, if teachers chose attendance optimally prior to the program given their

intrinsic motivation to teach, other incentives implicit in the system, and their value of time in

other activities, they should have %een indifferent at the margin to small changes in attendance,

and hence modest incentives could potentially have had a su%stantial effect! -he incentives in the

program were sufficient to induce teachers to change their %ehaviorLthey simply did so in ways

that may not have %een consistent with the intentions of the program sponsor! Moreover, while

larger incentives might induce more effort %y teachers, they could also have induced effort at

counterproductive signaling, for e$ample through cheating on tests or forcing weak students to

drop out! -hey would also force teachers to %ear more risk! +ome argue that individuallevel

incentives for teachers could potentially undermine cooperation within the school!

A second caveat is that incentives may work as much %y encouraging people who will %e

good teachers to enter the profession as %y eliciting higher effort from those who would %ecome

teachers in any case! 4owever, given the Fueuing for teaching positions in Kenya, it is unlikely

that people who either already have teaching ;o%s or who have the academic Fualifications to

enter teacher training colleges ?%ut not universities@ will select out of the profession! Any effect

on this margin in Kenya and other developing countries with Fueues for teaching ;o%s is

therefore likely to %e small!

-hird, the program was e$plicitly temporary! If teachers e$pected the program to

continue indefinitely, and if they e$pected to remain at the schools for many years, they would

have had more incentive to make longrun investments in learning! 2( 9n the other hand, %ecause

the program was temporary it was possi%le to %ase incentives on improvements over %aseline

performance, to incorporate incentives to prevent students from dropping out, and to restrict the

program to teachers already in school and thus to avoid strengthening incentives for teachers to

&J In practice, many teachers transfer %etween schools!



seek transfers to schools with pupils from more advantaged %ackgrounds! A program without

these features would %e much less attractive since it would %e difficult to provide incentives to

teachers in weak schools, to prevent teachers from trying to influence the pool of pupils entering

their school, or to avoid increasing incentives for good teachers to try to transfer to the %est

schools!

:ourth, teachers in program schools may have e$erted little effort %ecause they %elieved

that the test was such that learning has only a small impact on test scores! Alternative tests that

%etter measure longrun learning might potentially create %etter incentives! 4owever, since the

incentives provided %y I3+ were %ased on the official government of Kenya e$ams, which in

turn are %ased on the official curriculum, any incentive program %ased around these e$ams is

likely to run into similar difficulties!

4owever, whatever the pro%lems with teacher incentives, the status FuoLwith its &'(

teacher a%sence rateLis inadeFuate, so it seems worth e$ploring other alternatives!

9ne strategy for improving incentives would %e to attach incentives to measura%le inputs,

such as teacher attendance! In many countries, teachers> pay is not linked to pupils* test scores,

%ut a teacher who is a%sent &'( of the time would typically face some sort of disciplinary

sanction! 9ne pro%lem with this approach is that attaching incentives to inputs rather than

outputs can lead to undersupply of %adly measured inputs! :or e$ample, since random audits can

verify whether a teacher is in school, %ut not whether a teacher is in class, teachers might come

to school, %ut not come to class, since outside inspectors cannot easily monitor presence in class!

#hile it would %e prohi%itively e$pensive for outside inspectors to visit schools regularly

enough to keep track of teachers> presence for incentive purposes, headmasters could keep these

records! 4owever, headmasters do not currently have incentives to risk getting into a

confrontation with teachers over a%sences! 9ne indication of this comes from an evaluation of

the program the NG9 implemented at preschools! ?Preschool teachers are not formal Ministry

=



of /ducation employees, %ut instead are locally hired and paid %y parents. In this program,

school committees were given funds with which to provide %onuses to preschool teachers,

conditional on their not missing more than a specified num%er of days of class! If the funds were

not spent on the teachers, school committees could use them for other preschool purposes, so

headmasters and school committees argua%ly had a strong incentive to monitor preschool

teachers! Preliminary work suggests that the program yielded little if any improvement in

a%sence rates, and it is clear that headmasters did not strictly enforce the rules reFuiring teacher

attendance as a condition of the %onus %eing provided! More %roadly, headmasters are already

reFuired to keep log %ooks of teacher attendance, and inspectors are supposed to monitor them

%ut log %ooks are often not even filled out!

0ltimately, an analysis of the pro%lem must turn to the political economy of education!

Given that Kenya>s centrali)ed education system is not producing adeFuate incentives, it may %e

worth considering decentrali)ing control over teachers to local school committees or allowing

parents to choose schools and tying school finance more tightly to their decisions!&C

&C +ince students> placement in secondary school depends on performance on the primaryschool leaving e$am, local

communities and parents would share some of the same incentives to focus on test preparation as teachers ?seeAcemoglu et al!, &''&@! Nonetheless, since teachers transfer schools fairly freFuently, they likely have greaterincentives than parents to focus on the short run!



References

Acemoglu, .aron, Kremer, Michael, and Atif Mian ?&''&@, QMarkets, :irms and Governments,Qunpu%lished!

American :ederation of -eachers, -eacher +alary +urvey Archives at

htt%455---.aft.org5research5surve++5salarsurve++.%df

3hapman, .avid #!, +nyder, 3onrad #!, and +hirley A! 6urchfield ?<BB<@, -eacher Incentivesin the -hird #orld," Agency for International .evelopment 7eport no! <= ?#ashington, .3@!

3haudhury, Na)mul, Kremer, Michael, Muralidharan, Karthik, and 4alsey 7ogers ?&''@,A%sence Among +ervice Providers1 A Preliminary Note, work in progress, #orld 6ank!

3lotfelter, 3harles -! and 4elen :! add ?<BBD@, 7ecogni)ing and 7ewarding +uccess in Pu%lic+chools," in 4elen :! add ed! Holding Schools cco!nt"#le$ %er&orm"nce'B"sed (e&orm in )d!c"tion, 6rookings Institution, #ashington, .!3!

/ducation 3ommission of the +tates ?&'''@,PayforPerformance1 Key Uuestions and essonsfrom :ive 3urrent Models," /3+ Issue Paper, /ducation 3ommission of the +tates, at---.ecs.org5clearinghouse52)53+52)3+.ht# !

:iglio, .avid N! and 8oshua #inicki ?&''&@, :ood for -hought1 -he /ffects of +choolAccounta%ility Plans on +chool Nutrition," National 6ureau of /conomics #orking Paper B<B!

Glewwe, Paul #!, Kremer, Michael, and +ylvie Moulin ?&''<@, -e$t%ooks and -est +cores1/vidence from a Prospective /valuation in Kenya," mimeo!

Gramlich, /dward and Patricia Koshel ?<BJ@, /ducational Performance 3ontracting,"6rookings Institution, #ashington, .!3!

4annaway, 8ane ?<BB&@, 4igher 9rder -hinking, 8o% .esign, and Incentives1 An Analysis andProposal," meric"n )d!c"tion (ese"rch Jo!rn"l , vol! &B, <, pp! &<!

4ar%ison, 7alph #! and /ric A! 4anushek ?<BB&@, )d!c"tion"l %er&orm"nce o& the %oor$ Lessons &rom (!r"l Northe"st Br"*il , NV1 9$ford 0niversity Press!

4anushek, /ric A!, Kain, 8ohn :!, and +teven 7! 7ivkin ?<BBB@, .o 4igher +alaries 6uy 6etter-eachersW" unpu%lished!

4anushek, /ric A!, Kain, 8ohn :!, and +teven 7! 7ivkin ?<BBC@, -eachers, +chools, andAcademic Achievement," National 6ureau of /conomic 7esearch #orking Paper DDB< !

4anushek, /ric A! ?<BBD@, 9utcomes, 3ost, and Incentives in +chools," in 4anushek, /ric A!and .ale #! 8orgenson, eds! +mproving meric"s schools$ The (ole o& +ncentives, NationalAcademy, #ashington, .!3!

4anushek, /ric A! with 3harles +! 6enson et al ?<BB=@, M"-ing Schools or-$ +mproving

%er&orm"nce "nd Controlling Costs, 6rookings Institution, #ashington, .!3!

D

http://www.aft.org/research/survey00/salarysurvey00.pdf

http://www.ecs.org/clearinghouse/28/30/2830.htm

http://www.aft.org/research/survey00/salarysurvey00.pdf

http://www.ecs.org/clearinghouse/28/30/2830.htm



4olmstrom, 6engt and Paul Milgrom ?<BB<@, Multi-ask PrincipalAgent Analysis1 Incentive3ontracts, Asset 9wnership, and 8o% .esign," Jo!rn"l o& L"/, )conomics "nd rg"ni*"tion, vol!J, ', pp! &=&!

8aco%, 6rian ?&''&@, Accounta%ility, Incentives and 6ehavior1 -he Impact of 4igh+takes

-esting in the 3hicago Pu%lic +chools," National 6ureau of /conomics #orking Paper CBDC!

8aco%, 6rian and +tephen evitt ?&''&@, 7otten Apples1 An Investigation of the Prevalence andPredictors of -eacher 3heating," unpu%lished manuscript!

Kingdon, Geeta and :rancis -eal ?&''&@, .oes Performance 7elated Pay for -eachers Improve+tudent PerformanceW +ome /vidence from India," mimeo!

Koret), .aniel ?&''&@, Qimitations in the 0se of Achievement -ests as Measures of /ducators*Productivity,Q Jo!rn"l o& H!m"n (eso!rces, vol! J, =, pp! J&JJC!

avy, Eictor ?&''&a@, Paying for Performance1 -he /ffect of -eachers> :inancial 9utcomes on+tudents> +cholastic 9utcomes," mimeo!

avy, Eictor ?&''&%@, /valuating the /ffect of -eacher Group Performance Incentives on+tudents Achievements," Jo!rn"l o& %olitic"l )conom, vol! <<', D, pp! <&CD<<C!

ockheed, Marlaine /! and Adriaan M! Eerspoor ?<BB<@, +mproving %rim"r )d!c"tion in

eveloping Co!ntries, NV1 9$ford 0niversity Press!

9lsen, ynn ?<BBB@, PayPerformance ink in +alaries Gains Momentum," )d!c"tion ee- 9cto%er <, <BBB!

P796/ ?Pu%lic 7eport on 6asic /ducation for India@ ?<BBB@, 9$ford 0niversity Press!

7ichards, 3raig and -ian M! +heu ?<BB&@, -he +outh 3arolina +chool Incentive 7ewardProgram1 A Policy Analysis," /conomics of /ducation 7eview, vol! <<, <, pp! J<CD!

J





Table 24 Progra# Effect on Teacher Entr and E0it

?<@ ?&@ ?@ ?=@

ependent

6"ri"#le$

/$it 3urrent +chool ?'R<@ /nter New +chool ?'R<@

/$it in <BBJ /$it in <BBC /nter in <BBC /nter in <BBB

Incentive+chool

'!'=< '!''J '!'&D '!''&?'!'@ ?'!'&D@ ?'!''@ ?'!'=@

Male '!'=J '!'&' '!'= '!'B&?'!'&@ ?'!'<@ ?'!'&@ ?'!'@XX

3onstant '!<J '!&'B '!<B' '!&=?'!'&=@XX ?'!'<B@XX ?'!'&&@XX ?'!'&@XX

9%servations <<J <&&J <&&J <&&C Notes1+tandard errors in parentheses5 regressions include schoollevel random effects! X significant at <'(5 XX significant at (5:or e$it regressions, incentiveRnonincentive refers to the originating school5 for entry regressions incentiveRnon

incentive refers to the destination school! -he unit of o%servation in all regressions is the teacher!

B



Table 34 Progra# Effect on Teacher Attendance

?<@ ?&@ ?@ ?=@ ?@

ependent

6"ri"#le$

-eacher Attendance PercentageAttendance .ifferences7ttend"nce progr"m e"r 8

ttend"nce pre'progr"m e"r 9

Vear ' Vear < Vear & Vear < Vear ' Vear & T Vear '

Incentive+chool

'!'<& '!''C '!'<< '!''J '!'D?'!'=@ ?'!'<B@ ?'!'&&@ ?'!'=C@ ?'!'=B@

Grade '!'' '!'<' '!''' '!''B '!''&?'!'<&@ ?'!''J@ ?'!''B@ ?'!'<@ ?'!'<D@

Male ?'R<@ '!'< '!''J '!<'C '!'&C '!'B?'!'=@ ?'!'&&@ ?'!'&@XX ?'!'@ ?'!'@X

3onstant '!C&C '!CC& '!B'= '!'=B '!'D=?'!'J@XX ?'!'==@XX ?'!'=B@XX ?'!'B'@ ?'!'B=@

9%servations =DD BJ &' BD <B

Notes1+tandard errors in parentheses5 regressions include schoollevel random effects!X significant at <'(5 XX significant at (-he dependent varia%le is the percentage of the visits for which the teacher was present, %ased on up to two visits in<BBJ, five visits in <BBC and three visits in <BBB! -he unit of o%servation is the teacher!

='



Table "4 Progra# Effect on ,o#e-or! Assign#ent

?<@ ?&@ ?@ ?=@ ?@

ependent

6"ri"#le$

4omework Assignment ?'R<@4omework .ifferences7Home/or- progr"m e"r 8

Home/or- pre'progr"m e"r 9

Vear ' Vear < Vear & Vear < Vear ' Vear & T Vear 'Incentive+chool

'!'<&?'!'=&@

'!'&?'!'=@

'!''B?'!'=J@

'!'B&?'!'@X

'!'=&?'!'B@

Grade '!'JB?'!''J@XX

'!'D&?'!''J@XX

'!<=B?'!''J@XX

'!'<J?'!'<J@

'!'D?'!'<J@XX

3onstant '!<JD?'!'=B@XX

'!'D'?'!'@

'!CD?'!'@XX

'!<J?'!<<<@

'!<?'!<<<@

9%servations <B<= <DJD &J< =< =&J Notes1+tandard errors in parentheses5 regressions include schoollevel random effects!X significant at <'(5 XX significant at (5

In columns < through each o%servation represents a student asked a%out homework assignment in the previousday5 in columns = and differences across years are calculated at the schoolgrade level!

=<



Table &4 Progra# Effects on Pedagog

?<@ ?&@ ?@ ?=@ ?@

<ear + <ear 1 <ear 2 <ear 1 <ear + <ear 2 = <ear +

Panel A

Dependent Variable: >se of lac!board ?+51Incentive+chool

'!'<C '!'& '!'C '!'< '!'JC?'!'<@ ?'!'&D@ ?'!'<@ ?'!'D@ ?'!'DB@

Grade '!'<' '!'' '!'<C '!''< '!'&B?'!''B@ ?'!''J@ ?'!'<@ ?'!'<=@ ?'!'&&@

3onstant '!CJ '!BJ '!BCB '!'&< '!'BC?'!'=@XX ?'!'==@XX ?'!'C=@XX ?'!'C@ ?'!<@

9%servations ='= BC &J &=D <=B

Panel

Dependent Variable: >se Teaching Aid ?+51Incentive+chool

'!'&D '!''D '!'<& '!'& '!'&?'!'&@ ?'!'<@ ?'!'@ ?'!'&@ ?'!'DJ@

Grade '!'&< '!''D '!''= '!'<D '!''&?'!'<&@X ?'!''B@ ?'!'<@ ?'!'&<@ ?'!'&@

3onstant '!& '!<= '!'B= '!'B '!'='?'!'J@XX ?'!'B@XX ?'!'C'@ ?'!<&=@ ?'!<<@

9%servations BB DJ & &=< <=J

Panel

Dependent Variable: Teacher aring

?1 to &4 1:ver caringIncentive+chool

'!'C' '!'D '!'< '!'& '!'C?'!<'=@ ?'!'D&@ ?'!<&@ ?'!<@ ?'!<JC@

Grade '!'<C '!'<' '!<& '!'& '!'B

?'!'=@ ?'!'&&@ ?'!'<@XX ?'!'=C@ ?'!'D&@3onstant <!CD <!J'< <!<C= '!<&& '!&C'?'!&'=@XX ?'!<@XX ?'!&'@XX ?'!&B&@ ?'!J@

9%servations C& J< &= &C <=D

Panel $

Dependent Variable: Teacher Energ

?1 to &4 1:energeticIncentive+chool

'!'' '!'=< '!<D= '!'' '!'J'?'!'BD@ ?'!'C'@ ?'!<&'@ ?'!<DJ@ ?'!<B@

Grade '!'& '!'<B '!'J' '!'<J '!'B&?'!'@ ?'!'&@ ?'!'&J@XX ?'!'&@ ?'!'D&@

3onstant <!B&D <!CJ' <!<&D '!'J '!JBC

?'!&<<@XX ?'!<=D@XX ?'!<C'@XX ?'!&=@ ?'!JJ@XX

9%servations C J' & &B <=D

Notes1+tandard errors in parentheses5 regressions include schoollevel random effects!X significant at <'(5 XX significant at (!/ach o%servation in columns < through represents a classroom5 differences in columns = and are calculated at theschoolgrade level! -here are fewer o%servations in <BBB %ecause only one class per schoolRgrade was o%served thatyear!

=&



Table '4 Progra# Effect on Pre%arations

?<@ ?&@ ?@ ?=@ ?@ ependent

6"ri"#le$

Preparations ?'R<@ Preparation .ifferences7%rep"r"tion progr"m e"r 8

%rep"r"tion pre'progr"m e"r 9<ear + <ear 1 <ear 2 <ear 1 <ear + <ear 2 = <ear +

Preparations (Vacation and During School)

Incentive +chool '!''J '!'=& '!'J= '!'=B '!'C<?'!'==@ ?'!'J@ ?'!'=@XX ?'!'=&@ ?'!'=J@X

Grade '!< '!< '!<' '!'&< '!'&?'!''B@XXX ?'!''J@XXX ?'!''J@XXX ?'!''B@XX ?'!''B@XXX

August 4oliday?'R<@

'!'&' '!'C '!<&& '!'JC '!<'&

?'!'<D@ ?'!''@X ?'!'=@XXX ?'!'J@XX ?'!'[email protected]%er4oliday ?'R<@

'!J' '!=& '!= '!'C& '!<D=

?'!'&@XXX ?'!'&B@XXX ?'!''@XXX ?'!'<@XX ?'!'D@XXX-erm Eisit <

?'R<@

'!'B= '!<&D '!<' '!'& '!'D

?'!'@XXX ?'!'='@XXX ?'!'C@XXX ?'!'@ ?'!'&@-erm Eisit &?'R<@

'!'B= '!<DC '!&C& '!'J= '!<CC

?'!'@XXX ?'!'=<@XXX ?'!'='@XXX ?'!'&@ ?'!'&@XXX-erm Eisit ?'R<@

'!'BD '!<C '!&=& '!'D& '!<=D

?'!'@XXX ?'!'='@XXX ?'!'=@XXX ?'!'<@ ?'!'@XXX3onstant '!'& '!J& '!<&< '!<' '!C<

?'!'D=@XXX ?'!'@XXX ?'!'&@XX ?'!'D=@XX ?'!'D=@XXX

9%servations ''' ''' ''' ''' '''

Vacation Preparations

Incentive +chool '!' '!'CB '!'B< '!' '!'D?'!'=@ ?'!'<@XXX ?'!'@XX ?'!'C@ ?'!'=D@

Grade '!<D '!<B '!<<C '!'<J '!'C?'!''C@XXX ?'!''D@XXX ?'!''J@XXX ?'!''C@X ?'!''B@XXX

August 4oliday?'R<@

'!'&' '!'C '!<&& '!'JC '!<'&

?'!'<D@ ?'!''@X ?'!'=@XXX ?'!'J@XX ?'!'[email protected]%er4oliday ?'R<@

'!J' '!=& '!= '!'C& '!<D=

?'!'&@XXX ?'!'&B@XXX ?'!''@XXX ?'!'<@XX ?'!'D@XXX3onstant '!&J '!=& '!&<B '!<' '!'C

?'!'=B@XXX ?'!'=B@XXX ?'!'=@XXX ?'!'J@X ?'!'B@XXX

9%servations <'' <'' <'' <'' <'' Notes1+tandard errors in parentheses5 regressions include schoollevel random effects! X significant at <'(5 XX significantat (5Preparations are reported at D times during the year for each grade1 vacation terms and three periods during theyear5 each o%servation represents a school grade at a given time during the year! 7ates for given time periods arereported compared to the omitted time period, the April holiday!

=



Table (4 Progra# Effect on $ro%out and Re%etition Rates

?<@ ?&@ ?@ ?=@<ear 1 <ear 2

$ro%out Re%etition $ro%out Re%etition

Incentive +chool '!''= '!'<& '!''& '!'<'?'!'<'@ ?'!'<B@ ?'!'<@ ?'!'&D@

Male ?'R<@ '!'&< '!'<' '!'<C '!''B?'!''@XX ?'!''C@ ?'!''@XX ?'!''C@

3onstant '!'J '!&B '!<<D '!&D?'!''J@XX ?'!'<=@XX ?'!'<<@XX ?'!'<C@XX

9%servations <=< <&DCD <== <&DJ< Notes1+tandard errors in parentheses5 regressions include schoollevel random effects!X significant at <'(5 XX significant at (5

Each observation re%resents an u%%er %ri#ar school student.

==



Table )4 Progra# Effects on Partici%ation in E0a#s

?<@ ?&@ ?@<ear 1 <ear 2 <ear 3

?PostProgra#

Panel A

Dependent Variable: Ta!e $istrict E0a# ?+51

Incentive +chool '!'D' '!<'C '!'&&?'!'@X ?'!'&C@XX ?'!'=@

Male ?'R<@ '!'< '!'<< '!''J?'!''C@X ?'!''B@ ?'!'<D@

Grade '!'C '!'D '!'<'?'!''@XX ?'!''=@XX ?'!'<'@

3onstant '!CB '!& '!D'?'!'&B@XX ?'!'=@XX ?'!'DB@XX

9%servations <'DB' J<C D=&

Panel

Dependent Variable: Ta!e IS E0a# ?+51

Incentive +chool '!'' '!'<?'!''J@ ?'!'&=@

Male ?'R<@ '!''= '!'''?'!''=@ ?'!''J@

Grade '!'<' '!''B?'!''<@XX ?'!''@XX

3onstant '!CC' '!C&&?'!'<'@XX ?'!'&J@XX

9%servations <=BJ J<C

Panel

Dependent Variable: Ta!e /PE E0a# ?+51

Incentive +chool '!'<= '!'&D '!'&<?'!'=J@ ?'!'<J@ ?'!'B@

Male ?'R<@ '!''' '!''< '!'<

?'!'<B@ ?'!'<@ ?'!'&B@3onstant '!JJB '!B<& '!C<

?'!'=@XX ?'!'<=@XX ?'!'&B@XX

9%servations <D&= <&D DC<

Note1 +tandard errors in parentheses5 schoollevel random effects included!X significant at <'(5 XX significant at (5.istrict test data was not availa%le for -eso .istrict in <BBC!I-- methodology employed!/ach o%servation represents an upper primary school pupil in year '5 columns & and are limited to pupils who didnot repeat or drop out in any year!

=



Table *4 Progra# Effect on Test Scores b Sub@ect ?$istrict E0a#

?<@ ?&@ ?@ ?=@ ?@ ?D@ Dependent

Variable

Test Scores

?Standardied relative to

co#%arison schools

Test Score $ifferences

Test Score program year –

Test Score pre-program year

<ear 1 <ear 2 <ear 3 <ear 1 =

<ear +

<ear 2 =

<ear +

<ear 3 =

<ear +/nglish '!'B

?'!<'J@'!'B=?'!'B=@

'!'<J?'!<<&@

'!'&=?'!'J<@

'!''?'!'CD@

'!'B<?'!<&&@

Math '!'C?'!'CB@

'!'BB?'!'C=@

'!'JJ?'!'CB@

'!'JD?'!'=@

'!<'XX?'!'D=@

'!<'D?'!'CB@

+cience '!'<?'!'B<@

'!<?'!<'&@

'!<&<?'!<<@

'!''?'!'JD@

'!&'DX?'!'B=@

'!<B=?'!<&C@

+wahili '!'&?'!'B@

'!<'?'!'J&@

'!'B<?'!'C=@

'!'&?'!'C@

'!'<B?'!'B=@

'!<=?'!&&<@

G!4!37! '!'B?'!'CB@

'!&'&XX?'!'BJ@

'!'?'!<'@

'!&'X?'!<'J@

'!=<XX?'!<&B@

'!'&<?'!&D&@

A!3!M! '!''J

?'!'BD@

'!'<'

?'!'B&@

'!'=B

?'!<'&@

'!<<D

?'!<&<@

'!<'C

?'!<=@

'!&<C

?'!&=B@4+! 6/! '!'=B

?'!'B&@'!'J?'!<'J@

'!'JB?'!<<@

'!'J?'!<D<@

'!<DJ?'!<BD@

<!&&XX?'!&@

All +u%;ects YGrades

'!'='?'!'JB@

'!<DX?'!'JJ@

'!'CJ?'!'C@

'!'=?'!'=@

'!<BXX?'!'D@

'!''C?'!'C=@

9%servations ',C=& J,D&' <,CB &=,DJJ <,D=< ,'

Note1 +tandard errors in parentheses5 regressions include schoollevel random effects!X significant at <'(5 XX significant at (5Vear < district test results are availa%le only for 6usia/ach row represents a random effects regression of test scores on a dummy varia%le for teacher incentive schoolsand on region and se$ dummy varia%les, %ased on data on the <'' schools in -eso and 6usia .istricts! :or each

gradeRsu%;ect com%ination, test scores were standardi)ed %y su%tracting the mean score and dividing %y the standarddeviation of the test score from the comparison schools!/ach o%servation represents a test score in a particular su%;ect for an upper primary school pupil5 columns & and are limited to pupils who were enrolled in year < and did not repeat or drop out! 3olumns =, , D impose theadditional restriction that a pretest score is availa%le!J,C=C students ?grades = to C@ took at least one district e$am in year <! 9f these, ,J< had pretest scores from a preprogram year, in this case <BBD! In year &, when e$am results are also availa%le for -eso, <',B&J students ?grade= to C@ took at least one e$am and D,D of these students also had pretest scores from the same preprogram year!In the postprogram year, B,D< students ?grade = to C@ took at least one e$am and =,'<D of these had pretest scores!In later years more students have no pretest scores %ecause students who enter the sample ?%y reaching = th grade@after the first year will not have pretest scores! +o, for e$ample, in the postprogram year students in = th and th grade will not have pretest scores!

=D



Table 1+4 Progra# Effect on Test Scores b Sub@ect ?9onIncentive Tests

?<@ ?&@ ?@ ?=@ ?@ ?D@ Dependent

Variable

Test Scores

?Standardied relative to co#%arison

schools

Test Score $ifferences

Test Score program year –

Test Score pre-program year

<ear 1 <ear 2 <ear 3 <ear 1 = <ear

+

<ear 2 =

<ear +

<ear 3 =

<ear +

Panel A

Dependent Variable: IS Sub@ect Test Scores

/nglish '!'JJ?'!'B'@

'!'JJ?'!<C@

'!''<?'!'='@

'!'<?'!'BB@

Math '!'?'!'J=@

'!'DB?'!'J=@

'!'=&?'!'=<@

'!''B?'!'C@

+cience '!<&B?'!'C&@

'!'B<XX?'!'=@


'!'CB?'!'JB@

'!'C?'!'B'@

'!'<J?'!'@

'!'<D?'!'D@

9%servations B,<' <&,BBD &,BB <',<&

Panel Dependent Variable: /PE Test Scores

/nglish '!<<D?'!'B=@

'!<'?'!<&D@

'!''&?'!<&@

'!'=?'!<'@

'!<&'?'!<J@

'!<'?'!<B&@

Math '!<DD?'!<'&@

'!<&'?'!'BB@

'!'==?'!<&=@

'!<&?'!<'@

'!<=?'!<&=@

'!'J<?'!<C@

+cience '!<&?'!'BC@

'!<<?'!<<=@

'!'='?'!<=&@

'!&''X?'!<<=@

'!<CB?'!<=&@

'!<D?'!<B=@

+wahili '!&<&X?'!<&<@

'!&&DXX?'!<<&@

'!<&D?'!<<@

'!<'J?'!<&D@

'!'C<?'!<&@

'!=JX?'!&J'@

G!4! 37 '!<DJX?'!'CC@

'!&JXX?'!<<@

'!''=?'!<&'@

'!<=B?'!<'C@

'!DXX?'!<D@

'!'?'!&@

A!3!M! '!'=

?'!'BC@

'!<DB

?'!<<J@

'!'&J

?'!<@

'!<B

?'!<&<@

'!<DB

?'!<DC@

'!'C

?'!&DJ@

4+! 6/! '!<&?'!'B<@

'!<=?'!<<<@

'!'&?'!<&C@

'!''C?'!<=D@

'!&DC?'!<J@

<!&CX?'!DC@


'!<CX?'!'J=@

'!<DX?'!'B'@

'!''B?'!<'<@

'!<'=?'!'C'@

'!<&?'!'BJ@

'!''D?'!<C@

9%servations <',=' C,=&J =,' J,<& ,&=J <,'

Note1 +tandard errors in parenthesis5 regressions include schoollevel random effects!X significant at <'(5 XX significant at (5Vear I3+ tests were given only in &J schools so scores are not reported5 K3P/ tests are taken %y grade C studentsonly! /ach row represents a random effects regression of test scores on a dummy varia%le for teacher incentiveschools and on region and se$ dummy varia%les, %ased on data on the <'' schools in -eso and 6usia .istricts! :or

each gradeRsu%;ect com%ination, test scores were standardi)ed %y su%tracting the mean score and dividing %y thestandard deviation of the test score from the comparison schools!/ach o%servation represents a test score in a particular su%;ect for an upper primary school pupil5 columns & and are limited to pupils who were enrolled in year < and did not repeat or drop out! 3olumns =, , and D impose theadditional restriction that a pretest score is availa%le!<,B students ?grades = to C@ took at least one su%;ect of the I3+ e$ams in year <! 9f these, <<,&BC had pretestscores from year ', in the form of normali)ed district e$am scores from year '! <,D=J students took at least one I3+e$am in year &, of which C,DC had pretest scores from year '! <,=B' eight graders took at least one K3P/ e$am inyear <, of which <,'&D had pretest scores from year '! <,C= students took at least once K3P/ e$am in year &, of which B== had pretest scores!<,J students took at least one K3P/ e$am in year , of which CB had pretestscores!

=J



6igure 1

Three Models of Teacher Absence $istribution

+

+.+&

+.1

+.1&

+.2

+.2&

+.3

+.3&

+."

+."&

+.+ +.1 +.2 +.3 +." +.& +.' +.( +.) +.*

Absence Percentage

P e r c e n t o f T e a c h e r s 8 E # % i r i c a l a n d

9 o n 1 P a r a # e t r i c $ i s t r i b u t i o n s

+.+++

+.++&

+.+1+

+.+1&

+.+2+

+.+2&

+.+3+

+.+3&

+.+"+

+.+"&

+.+&+

$ e n s i t . o f T e a c h e r s 8 e t a

$ i s t r i b u t i o n

Non,Parametric

/mpirical

6eta .istri%ution

=C

Documents

Teacher Incentives