52
Distorted Information: Evidence from Quality Signals in School Markets Jos´ e Ignacio Cuesta Felipe Gonz´ alez Cristi´ an Larroulet § March, 2016 Abstract When information in a market is distorted, choices will potentially be inecient. We study educational markets in Chile, where school quality is signaled using test scores. We show that quality signals are distorted due to dierences in student attendance the day of the test, which translates into misleading information used by parents when choosing schools. We then estimate a school choice model and show that a low-cost information policy that eliminates distortions has positive and non-trivial eects on welfare. Keywords: schools, quality, disclosure, competition, choice JEL Codes: I20, L15 We would like to thank Gustavo Bobonis, Michael Dinerstein, Fred Finan, Matt Gentzkow, Ali Horta¸csu, Casey Mulligan, Derek Neal, Nicol´ as Rojas, Jesse Rothstein, Chris Walters, and seminar participants at UC Berkeley, the Bay Area Labor and Public Economics Graduate Student Conference, the First Conference of the RISE Programme, and the Pacific Conference for Development Economics at Stanford for comments and suggestions. We would also like to thank the Ministry of Education and the Agency for the Quality of Education of the government of Chile for access to the data and useful discussions. University of Chicago. Email: [email protected] University of California, Berkeley. Email: [email protected] § London School of Economics. Email: [email protected] 1

Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

  • Upload
    others

  • View
    5

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Distorted Information:

Evidence from Quality Signals in School Markets

Jose Ignacio Cuesta† Felipe Gonzalez‡ Cristian Larroulet§

March, 2016

Abstract

When information in a market is distorted, choices will potentially be ine�cient. We

study educational markets in Chile, where school quality is signaled using test scores.

We show that quality signals are distorted due to di↵erences in student attendance

the day of the test, which translates into misleading information used by parents when

choosing schools. We then estimate a school choice model and show that a low-cost

information policy that eliminates distortions has positive and non-trivial e↵ects on

welfare.

Keywords: schools, quality, disclosure, competition, choice

JEL Codes: I20, L15

⇤We would like to thank Gustavo Bobonis, Michael Dinerstein, Fred Finan, Matt Gentzkow, Ali Hortacsu,Casey Mulligan, Derek Neal, Nicolas Rojas, Jesse Rothstein, Chris Walters, and seminar participants at UCBerkeley, the Bay Area Labor and Public Economics Graduate Student Conference, the First Conferenceof the RISE Programme, and the Pacific Conference for Development Economics at Stanford for commentsand suggestions. We would also like to thank the Ministry of Education and the Agency for the Quality ofEducation of the government of Chile for access to the data and useful discussions.

†University of Chicago. Email: [email protected]

‡University of California, Berkeley. Email: [email protected]

§London School of Economics. Email: [email protected]

1

Page 2: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

1 Introduction

In markets where quality is imperfectly observed, firms utilize prices and advertising to

inform consumers (Milgrom and Roberts, 1986). In the interest of consumers, regulators

often implement disclosure policies, by which firms are required to provide information to

the public. However, due to the multiple e↵ects that disclosure policies have on firms,

this type of policy has the potential to distort information.1 Distorted information directly

a↵ects consumer choices and could have non-trivial welfare consequences due to di↵erences

in market opportunities and heterogeneity in preferences.

In this paper, we study distortions in quality signals in a school choice environment. There

are two reasons why school markets are particularly interesting. First, governments have

developed school accountability systems that utilize quality signals as measures of success.

The implementation of programs that rely on distorted quality signals has the potential to

cause misallocation of resources. Second, school choices that rely on distorted information

could lead to ine�cient choices and welfare loss. We are not aware of any empirical attempt

to quantify information distortions nor the welfare consequences of providing undistorted

information. This is important because if quality signals are distorted in a known way, then

correcting these ine�ciencies is extremely low-cost.

We study the case of Chile, a market-oriented school system in which quality is signaled

through media outlets using a standardized test called SIMCE. This test is taken every year

by all students in 4th grade and is the focus of public debates about the quality of education.

Using administrative data sets, we document that quality signals (test scores at the school

level) are distorted due to non-random heterogeneous changes in attendance of students the

day of the test. We provide a comprehensive empirical analysis to understand the variation

in distortions and estimate a school choice model to calculate the welfare consequences of

providing undistorted information.

Our analysis proceeds in three steps. First, we use administrative data on test-day

attendance at the student level to construct measures of distortions in quality signals. We

document the existence of distortions using a comparison of test takers and non-takers within

schools. This comparison allows us to calculate how the distribution of attending students

1Disclosure policies might, for example, reduce seller e↵ort in detecting quality or create trade-o↵s acrossdisclosed and undisclosed dimensions of quality. Dranove et al. (2003) shows how providers in the U.S. healthsystem responded to a quality disclosure program by avoiding to serve sicker patients. See Dranove and Jin(2010) for a review of quality disclosure.

2

Page 3: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

changes on test days. Then, drawing from the multiple imputation literature in statistics,

we propose a method to recover undistorted quality signals and, as a result, distortions. We

utilize these estimated distortions in quality signals for the rest of our analysis.

In the second part of our analysis, we present a discussion of the empirical determinants

of distortions in quality signals. We construct a panel dataset of distortions at the school

level for the entire educational system between 2005 and 2013. Robust patterns emerge in

our analysis. Distortions are not driven by within-school variation in observable character-

istics. On the contrary, the variation in distortions is mostly explained by observable and

unobservable time-invariant school characteristics. Low quality schools display more dis-

torted quality signals. In addition, because of the salient role of competitive environments

in the literature, we perform a comprehensive study of school competition using a spatial

analysis.2 Our results provide no supporting evidence for the hypothesis of larger distortions

in more competitive environments. Finally, we use quasi-experimental variation from two

government programs and find a limited role for teacher performance prizes and informa-

tional channels. We discuss the existing limitations to understand the observed variation in

distortions.

In the third part of our analysis, we use a school choice model to estimate the e↵ect

of distorted quality signals on school choices and households’ welfare. This exercise allows

us to calculate the market value of providing undistorted information about quality. For

estimation, we use the exact geographic location of approximately 100,000 students and

1,500 schools for the period 2011–2014. Exploiting the spatial content in our data, we

define school markets as the connected components of the spatial network of all schools in

the country. Identification of the parameters of the model is achieved using a combination

of quasi-experimental variation in government programs, climate-induced variation in test

scores, and fixed characteristics of competitors in a given year, i.e., BLP instruments. Our

demand estimates are consistent with previous findings and display significant heterogeneity

across poor and non-poor households.

Using the estimated preferences, we compute school choices in a counterfactual world

with no distortions. Our results suggest that providing households with undistorted quality

information would cause five percent of students to switch to higher-quality schools. The

welfare gains from providing undistorted information are small on average (less than 1 USD

2Recent studies have suggested a link between competitive environments and cheating behavior (e.g.,Shleifer 2004, Schwieren and Weichselbaumer 2010, Gilpatric 2011, Cartwright and Menezes 2014).

3

Page 4: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

per student). This result is driven by the fact that (1) most students would not switch

schools, and (2) choice and experience utility are equal for households who would not switch.

Welfare gains are heterogeneous and larger for household that would switch schools in a

counterfactual world with no distortions (10 USD per student). In particular, non-poor

households benefit more (14 USD) than poor households (5 USD). These di↵erential benefits

are explained by the observed heterogeneity in preferences and not by di↵erences in market

opportunities.

Although welfare gains from a policy that provides undistorted information are arguably

small, we emphasize the following. First, welfare gains aggregate to approximately 0.5 million

USD annually. Second, an intervention that corrects information about quality is free, as it

only entails calculating a corrected average instead of an average. Third, distortions could

be larger in other markets and countries. And fourth, this is the first estimate of welfare

losses due to distorted quality information in school markets.

This paper has two main contributions. In the first place, we quantify the magnitude

of distortions in quality signals in school markets. We propose a method to recover distor-

tions that might be applicable to other sectors of the economy. Certain shocks or events

might a↵ect the distribution of quality signals in a known way, which makes our method

to calculate distortions potentially appealing. In the second place, we quantify the choice

and welfare implications of distorted information using an empirical school choice model and

distinguishing between choice utility and experience utility.

Our work is related to three strands of the literature. First, it is related to a literature

in industrial organization that study the roles of disclosure and advertising in di↵erent mar-

kets.3 A relevant question in this literature is whether quality disclosure e↵ectively improves

consumer choice. Most of the impacts seem to operate in the margin of vertical sorting, i.e.,

consumers switch to higher quality products following increased disclosure. Work on analyz-

ing the e↵ects of quality disclosure in educational markets is somehow scarce and has yielded

mixed results.4 Bagwell (2007) provides a thorough review of the advertising literature. We

focus on a case in which advertising is informative. Moreover, following the distinction pro-

3Dranove and Jin (2010) provide a survey and describe theoretical arguments by which mandatorydisclosure may cause undesirable e↵ects.

4Hastings and Weinstein (2008) report substantial impacts of providing households in Charlotte withreport cards on the quality of chosen schools. Cooper et al. (2013) find evidence in the same directionfor households in di↵erent cities of Chile, although of a smaller magnitude. On the contrary, Mizala andUrquiola (2013) analyze household responses to a program that disclosed school quality in Chile and find noimpacts.

4

Page 5: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

posed by Nelson (1970), the fact that schooling is an experience good implies that quality

is hardly verifiable ex-ante, which implies that information acquired from advertising may

be particularly important for consumers. Our paper adds to this literature in two lines.

First, by focusing on educational markets, where there is limited work from an advertising

perspective. Second, by measuring the implications of deceptive advertising in this market.

This paper is also related to a second literature that focuses on studying the perverse

incentives of accountability systems in schooling markets, which have became increasingly

frequent in the last decade (e.g., Jacob and Levitt 2003, Figlio and Winicki 2005, Neal and

Schanzenbach 2010, Rouse et al. 2013). Moreover, as discussed by Figlio and Loeb (2011),

incentive programs designed on the basis of standardized tests are often subject to perverse

incentives, which is related to the discussion proposed in Neal (2013) regarding the usage of

test scores for multiple objectives. Our paper shows how heterogeneous attendance on test

day a↵ects the provision of information and endows households with inaccurate information

for their school choices.

Finally, this paper is also related to the literature studying school choice in competitive

markets. This literature has focused on estimating households’ preferences over schools’

characteristics (e.g., Gallego and Hernando 2009 and Neilson 2014 for Chile, and Bayer

et al. 2007, Hastings et al. 2009 and Walters 2014 for the U.S.). These papers generally find

that school fees, distance between home and school, and school quality are the most relevant

attributes of schools. The literature that focuses on understanding the role of information

in school choice is still at an early stage (e.g., Hastings and Weinstein 2008, Cooper et al.

2013, Andrabi et al. 2014). To our knowledge, there is no paper measuring the value of

information for school choice.

The remainder of the paper is organized as follows. In section 2, we describe the ed-

ucational market in Chile, we present data sources, and study attendance on test day. In

section 3, we construct measures of distortions in quality signals and provide an empirical

discussion of explanations behind observed distortions. In section 4, we estimate a school

choice model and study the impacts of a policy that eliminates distortions.

2 School Markets and Attendance on Test Day

Our analysis focuses on the Chilean market for primary schooling. After a market-oriented

reform was implemented in 1980, education has been served by a mixture of public and pri-

5

Page 6: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

vate (voucher and non-voucher) schools. Public schools are fully funded by the government.

Private voucher schools are privately managed, although eligible for receiving public funding

through vouchers. Private voucher schools are allowed to charge fees to parents in the form

of copayments, although vouchers are phased out on the basis of those. Private non-voucher

schools are privately managed and not eligible for receiving public funding. Over the last

three decades, the private sector has steadily increased its market share.5

In our analysis, we use multiple administrative data sets provided by the Ministry of

Education. First, we use the administrative record of schools operating every year between

2005 and 2013. We observe the following characteristics of schools: type (public, private),

enrollment, fees, participation in government programs and, importantly, their addresses,

which we use to construct markets. Second, we use the administrative record of students

between 2005 and 2013 (approximately 3.5 million every year). In this data set we observe

demographic characteristics of students and their annual academic performance. Third,

we use the performance of students at the national standardized test (SIMCE) to estimate

distortions in quality signals. This test is implemented every year, during two days, and at

the national level for a subset of grades. We focus on 4th grade, as it is tested every year in

the period 2005–2013. Fourth, we use daily attendance to school, available for all students

in 2013, to study heterogeneity in attendance on test day.

2.1 Descriptive statistics

We construct two datasets: (1) a panel dataset of schools observed every year between 2005–

2013, and (2) a panel data set of students observed daily in 2013. Although remarkably rich,

the student level data set is not available for all years, and is not available for non-voucher

schools. Therefore, we only consider public and voucher schools in our analysis, which

represent 93% of enrollment in 2013. The school level data set contains yearly information

on schools o↵ering 4th grade. The entry and exit of schools makes this panel unbalanced.

There are 8,254 di↵erent schools and, on average, 6,200 schools operating in a given year.

Panel A in Table 1 presents summary statistics for these schools: 51% are public, 42% are

voucher schools, 6% are private, and 31% are located in rural areas. The average school

serves approximately 39 students in 4th grade. More than half of schools charge no fees, and

the average monthly fee is approximately 22 USD. The average SIMCE test score is 245 and

5In 2013 public schools concentrated 38% of enrollment, while private voucher and non-voucher schoolsconcentrated 54% and 8% of enrollment respectively (MINEDUC, 2013).

6

Page 7: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

the standard deviation is 31.

Panel B in Table 1 presents descriptive statistics for the student level data set. For

simplicity, we use math test scores throughout the paper. The academic performance of a

student is captured by her GPA, which goes from 1 to 7, with a threshold of 4 to pass a

class. The mean of this variable is 5.9. The last two variables are attendance rates in test

day and non-test day. The former is simply the average of two indicator variables that take

the value of one if a student went to school in test day—recall that there are two test days,

so this variable takes the value of 0, 0.5, or 1 at the student level. The latter is the average

attendance in the five non-test days previous to the test day.

2.2 Attendance on test day

School quality is signaled through media outlets using the average test score of students:

if average test score is relatively high, then parents infer relatively high quality. As the

average test score of a school is a function of which students take the test, quality signals

are distorted if there is non-random variation in attendance on test day. We now show how

attendance to school changes the day of the test. Because the central government attempts to

increase attendance on test day through advertising, and schools have incentives to prevent

low-performing students taking the test, it is not a priori clear if attendance should increase

or decrease on test day. Moreover, our main concern is not with the average change, but

with the heterogeneity behind this change.

We compare the attendance rate of 4th graders (who take the test) to attendance of 3rd

graders (who do not take the test) daily around test day in 2013 (October 8th and 9th) to

estimate changes in attendance:

At = A4t � A3t (1)

where t = {�5, . . . , 5} indexes days, with t = {0, 1} denoting the two test days. We calculate

At in four subsamples of students: high-performing, above the 90th and 75th percentile of

the GPA distribution, and low-performing, below the 10th and 25th percentile of the GPA

distribution. In addition, to study the heterogeneity behind At, we calculate the following

school-specific changes in attendance on test day:

Ajt = Aj4t � Aj3t (2)

7

Page 8: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

where Ajkt is average attendance of students in kth grade at school j in day t. In the next

section, we show that a larger variance of Ajt translates into more distorted signals.

Figure 1 presents At and Ajt. In the upper panel, we plot the di↵erential attendance rate

around test day. On average, attendance increases by 2 percentage points, with the increase

being larger among high-performing students than among low-performing students. Against

the hypothesis of schools asking low-ability students to stay home on test day, we do not

observe a decrease in attendance of students with low GPA. These averages, however, mask

significant heterogeneity. In the lower upper, we plot the distribution of Ajt. The vertical

red line denotes the average increase increase of 2 percentage points. It is this variance in

attendance that causes quality signals to be distorted.

3 Distortions in Quality Signals

We argue that the quality signal of a school is “undistorted” if all students take the test.

Empirically, however, the existence of non-random heterogeneous absenteeism the day of the

test distorts quality signals.6 In this section, we describe our methodology to estimate the

magnitude of distortions at the school-year level. In addition, we perform a comprehensive

analysis to understand why these distortions exist.

3.1 Estimating undistorted quality signals

We consider a counterfactual scenario with no absenteeism to calculate undistorted quality

signals. Although daily attendance is not available for all years, it is possible to identify

absenteeism on test day at the student level using the administrative records of annual

academic performance and test scores: students with non-missing academic performance but

without test scores were absent on test day. The empirical challenge consists in estimating

test scores for absent students. We use multiple imputation methods developed by Rubin

(1987) to estimate test scores of absent students.

There are three steps in the multiple imputation method. In the first step, we define the

variable with missing data and the predictors. Let sijt be the test score of student i in school

6As described by Rubin (1976), random absenteeism within a school would not lead to bias in qualitysignals. Absenteeism within a school is, however, non-random.

8

Page 9: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

j and year t, and xijt be a matrix of variables that predict test scores at the student level:

sijt = f(xijt;�jt) (3)

where �jt is a matrix of parameters, which are to be estimated using the statistical model

f(·) in the sample of observed test scores. The second step corresponds to the choice of f(·).

For simplicity, we use a linear regression as statistical model, although results are robust to

using other models. Importantly, the parameters �jt are estimated with uncertainty. This

uncertainty can be taken into account in the imputations using draws from the asymptotic

variance of the estimated parameters �jt.7

In the third step, we specify predictors of student test scores within schools, i.e., xijt.

These variables need to be strong predictors of test scores, both statistically and theoretically,

and need to be available for all students in our dataset. We choose GPA at the end of the

academic year and the following indicator variables: students who were in 4th grade the

previous year, and students who studied at a di↵erent school the previous year. The latter

variables capture academic history and context respectively. This model can be used to

predict test scores at the student level during the period 2005–2013. As displayed by Figure

A.1, GPA is a strong predictor of test scores.8

After estimating test scores of absent students, we can estimate the “undistorted” quality

signal Yjt. This can be done by taking the average of test scores across students within schools

every year. As we have multiple estimates of missing test scores, we estimate multiple average

test scores for each school-year. Our estimate of an undistorted quality signal is the average

of multiple averages. Let Y (n)jt be the average test score at school j in year t calculated using

draw n = 1, . . . , N from the asymptotic variance of the estimated parameters. Then, our

estimate for the undistorted quality signal is:

Yjt =1

N

NX

n=1

Y

(n)jt (4)

The uncertainty of our estimates is reflected in the variance of Y (n)jt . We order Y

(n)jt from

7Note that parameters are specific to a school-year. One might worry that the asymptotic variance ispoorly calculated when using a small number students. In order to address this concern, we have repeatedour analysis using a bootstrap procedure and results are similar.

8In additional exercises we replace average GPA by average GPA in Math and Language separately.Unfortunately, these variables are only available for the period 2011–2013.

9

Page 10: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

lowest to highest within a school-year and take the percentiles 2.5 and 97.5 to generate

bounds for our estimate Yjt.

3.2 Descriptive statistics of distortions

Let jt be the distortion in quality signal at school j in year t, defined as the di↵erence

between observed (yjt) and undistorted (Yjt) quality signals. As Yjt is unobservable, we use

Yjt from equation (4). Then, our estimates for distortions in quality signals are defined as:

jt ⌘ yjt � Yjt (5)

We generate bounds for these quantities using the bounds for Yjt. Every school-year in

our dataset has an estimated distortion, together with a lower and upper bound, thus ac-

knowledging the uncertainty that involves estimating an unobserved parameter such as the

arithmetic mean of test scores across all students within a school.

To increase precision, our analysis restricts attention to schools in which the statistical

model in equation (3) uses more than 10 observations. More precisely, we study schools in

which the di↵erence between the number of students enrolled and the number of students

taking the test is greater than 10. Schools dropped from our analysis have few students,

are usually located in rural areas and (not surprisingly) their estimated distortions have

extremely wide bounds. We are confident that this restriction does not diminish the external

validity of our results, as our analysis includes 96 percent of enrolled students in 4th grade.9

Figure 2-A presents estimated distortions for all schools and years in our data set. The

y-axis represents distortions (in test score points), while the x-axis orders schools from lowest

to highest distortion. In addition, distortions in green (gray) are (not) statistically di↵erent

from zero. Approximately 24 percent of distortions are statistically larger than zero, and 65

percent of schools have a positive distortion. Some schools experience no absenteeism on

test day and their distortions are zero with no confidence interval.

Figure 2-B presents the distribution of distortions. The average school in our dataset

has a positive distortion of 2.7 test score points, approximately 0.1 standard deviations of

observed test scores. The distribution of distortions has a standard deviation of 3.9 test

score points and it is skewed towards the right. The facts that (1) the average distortion is

9The total number of students in the full dataset is 2,203,707, a number that decreases to 2,114,347 whenrestricting attention to schools with more than 10 students taking the test.

10

Page 11: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

di↵erent from zero, and (2) the distribution is far from normal, make clear that distortions

in quality signals are not random variation in test scores.10

3.3 Understanding distortions in quality signals

What explains the variation in distortions in quality signals? We now present a discussion

of the empirical determinants of distortions. For this purpose, we employ a panel data set

of distortions at the school level for the entire educational system between 2005 and 2013.

A significant share of the variation in distortions is explained by school time-invariant

characteristics. Consider a regression of distortions on school fixed e↵ects and three estima-

tion samples. In the first sample, we regress all estimated distortions on school indicators

and find that we can explain 35 percent of the variance in distortions. In a second sample,

we restrict attention to schools in which at least one student did not take the test and find

that school fixed characteristics explain 42 percent of the variance in distortions. In a third

sample, we restrict attention to schools with distortions that are statistically positive and

find that we can explain 69 percent of the variance in distortions. This is a significant share

of the variation, specially considering that the maximum variation that can be explained is

probably lower than one due to measurement error in the dependent variable. Consistent

with these patterns, in the appendix we show that within school variation in observable

characteristics of schools have little predictive power over distortions.11

What fixed characteristics of schools predict distortions? In Table 2-A we provide cor-

relations that are consistent with distortions being larger in small public schools with low

historical attendance rates. These regressions correspond to pooled OLS estimates weighted

by the inverse of the uncertainty associated with the estimation of distortions, where un-

certainty is defined as the size of the 95 percent confidence interval. These correlations are

particularly pronounced in schools with distortions that are statistically di↵erent from zero.

Additionally, Table 2-B presents the autocorrelation of distortions, which is always positive

10In Figure A.2 of the appendix we present an empirical analysis of the rank correlation between undis-torted and distorted quality signals at the market-year level. See section 4.1.2 for market definition. Approx-imately 40 percent of rank correlations are di↵erent from one, which suggests distortions in quality signalscause changes in the rankings of schools.

11Figure A.3 displays the results of non-parametric regressions of distortions in quality signals on schoolcharacteristics, including school and year fixed e↵ects. The only covariate that shows a strong relationshipwith distortions is the number of students absent on test day. This was expected as it is related to howdistortions are computed.

11

Page 12: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

and significantly di↵erent from zero. This positive autocorrelation is additional evidence

that distortions are non-random.

A particularly salient explanation behind distortions could be the role competitive en-

vironments play in causing perverse incentives (e.g., Shleifer 2004). To study the role of

competition, we perform a comprehensive spatial analysis of distortions using the presence

and observable characteristics of schools within 3 kilometers (1.9 miles). In particular, we

estimate the following regression equation:

b jt = f(Xjt) + ⌘j + ⌫t + "jt (6)

where Xjt is an observable characteristic of schools within 3 kilometers from the reference

school j in year t, ⌘j is a school fixed e↵ect, ⌫t is a year fixed e↵ect, and "jt is an error term

clustered at the school level. Due to the extensive number of variables that we cover, we

present these results in the appendix.12 A leading Xjt variable corresponds to the number

of schools operating close to the reference school, variation that essentially exploits the

plausibly exogenous entry and exit of schools. The empirical evidence provides no support

for the hypothesis of larger distortions in more competitive environments.

Finally, we exploit quasi-experimental variation from two government programs to un-

derstand the variation in distortions. More detail on this exercise is presented in Appendix

A and results are displayed in Figure A.5. The first is a biannual performance pay system

called SNED.13 This program e↵ectively increases the wages of teachers in a school if students

obtain high test scores and it provides variation in incentives depending on the probability

that a school will earn the prize. Distortions are not higher in schools that are more likely

12Figure A.4 displays the results of non-parametric regressions of distortions in quality signals on anumber of covariates related to the competitive environment faced by the school and its position relativeto it, including school and year fixed e↵ects. The only covariate that shows a notable relationship withdistortions in this analysis is the percentile of the school in the distribution of quality in the market, forwhich the relationship is positive although small in magnitude.

13SNED is a school performance evaluation system that takes the form of a tournament and providesawards to improved schools. SNED operates as follows: (i) groups of homogeneous schools are constructed,within which the contest is implemented; (ii) every two years, a multidimensional index is computed at theschool level, which considers academic performance and improvement and socioeconomic integration amongother outcomes; (iii) schools are ranked within their groups according to the value of such index; and (iv)schools covering the 25% � 35% of the total enrollment of each group get a monetary prize in an amountequivalent to around 40% of a monthly wage of a teacher, for each teacher in the school –the coverage ofthe prize was increased to 35% of the enrollment of the group since 2006–. Importantly, across dimensionsof the index, SIMCE test scores account for as much as 70% of the weight of the components used for itscalculation (Contreras and Rau, 2012).

12

Page 13: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

to win the prize, providing evidence against the hypothesis that teachers will manipulate

attendance to increase test scores.

A second government program we exploit, called “Educational Tra�c Lights,” classified

schools in 2010 according to their quality (test scores) in three categories: red (bad), yellow

(regular), and good (green).14 More detail on this exercise is presented in Appendix A and

results are displayed in Figure A.6. The cuto↵s of these categories provide quasi-experimental

variation in the incentives to manipulate test scores. We find some evidence that low quality

schools have higher distortions around the low cuto↵ (red to yellow), but no di↵erential

distortions in the higher cuto↵ (yellow to green).

Overall, the empirical patterns presented in this section are consistent with distortions

being a non-random phenomenon that is a function of school fixed characteristics. The fact

that distortions are not explained by within-school variation in observable characteristics

shed some light in the mechanisms generating distorted signals. However, it is certainly

possible that a heterogeneous response of schools to idiosyncratic events generate a non-

random distribution of attendance on test day. For example, schools might heterogeneously

react to the government program that attempts to increase attendance on test day. The

possibility of heterogeneous responses to events limits our empirical ability to understand

distortions, but it does not prevent us from estimating the ine�ciencies they create.

4 Welfare Consequences of Distorted Information

In the final part of our analysis, we estimate a school choice model to evaluate the impacts

of distorted quality signals. Using the estimated preferences of parents, we implement the

counterfactual exercise of providing undistorted quality signals. This exercise allows us to

compare observed with counterfactual outcomes, as well as to compute the welfare loss caused

by distortions.

14ETL was announced in April, 2010 and consisted of sending information to all households in the marketon the performance of schools in the SIMCE test implemented in 2009. That information included both testscores and a classification of schools as Red, Yellow or Green according to their test scores, with clear cuto↵sdetermining such outcome. An evaluation of this policy by Allende (2012) that uses the discontinuities insuch classification for identification, finds that it e↵ectively impacted school enrollment: households in themargin responded by enrolling more in yellow than red schools and more in green than yellow schools.

13

Page 14: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

4.1 School choice model

We estimate a simple model of school choice in the lines of Bayer et al. (2007). When con-

structing the model, we impose certain assumptions. First, households are assumed to have

full information both regarding the set of available schools and their observed characteris-

tics.15 Second, we assume schools do not select students based on attributes and do not face

capacity constraints, i.e., households can enroll their children in any school in their choice

set. As discussed by Neilson (2014), these assumption is likely to hold in the Chilean school

system. Finally, we assume the household’s location choice is independent of the school

choice problem. Although strong, this assumption is supported by the fact that there are no

institutional constraints on the choice set based on residential location.

Let households be indexed by i and schools by j. Households derive utility from schools’

fees, quality, and distance from their household, denoted respectively pj, qj and dij. They

also derive utility from other school characteristics Wj. For notational simplicity, we denote

Xj = [pj, qj,Wj], which includes K attributes. Preferences are heterogeneous depending on

households’ type, indexed by r. In our model, only observed heterogeneity in preferences is

considered, as further explained below. Moreover, we allow for households to derive utility

from schools’ unobserved characteristics ⇠j. Finally, each household has an idiosyncratic

preference shock, "ij, which we assume is distributed T1EV.

Under these assumptions, household’s i of type r indirect utility from enrolling their

children in school j is:

u

rij =

X

k

xk,j�rk + ⇠

rj + �

rddij + "ij (7)

where the first two terms measure utility from characteristics that depend only on the school

and are therefore constant across households of type r for a given school j, while the third

term measures disutility from distance between household i and school j for households of

type r, which varies across households. We can therefore rewrite equation (7) as follows:

u

rij = �

rj + �

rddij + "ij (8)

such that the parameters of the model are contained in the vector �r, but can be alternatively

15If this assumption does not hold, our estimation would potentially yield attenuated elasticities. However,it is not in the scope of this paper to explore further in this direction.

14

Page 15: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

represented by the vector �r and by �rd. Note that �rj is the component of utility derived

from choosing school j that is constant across individuals, the mean value of school j for

households of type r.

The probability of household i choosing school j can be derived analytically using house-

holds’ utility.16 The choice probability of school j by household i of type r predicted by the

model is a function of schools’ and households characteristics:

p

rij(�

r,d

r, �

rd) =

exp(�rj + �

rddij)P

l2Jiexp(�rl + �

rddil)

(9)

where Ji is the set of schools in the market where household i is located. We use this result in

the next subsections for both estimation of the model and for computing the counterfactual

exercise of interest.

4.1.1 Estimation

We estimate the parameters of the model using a two step procedure. First, we estimate

standard conditional logit models for each group r in each market and year in the data.

Second, we exploit the assumed linear functional form of the the utility function of households

in order to estimate the relationship between the preference parameters and school-level

characteristics.

The first stage of the estimation procedure consists of estimating equation (9), which

can be done by maximum likelihood. In order to allow for heterogeneity in preferences, this

procedure is implemented within each of multiple cells defined in on the basis of R socioe-

conomic levels, T time periods and M markets. The former is determined by the eligibility

of a student for a national program called Subvencion Escolar Preferencial (SEP), which is

determined fundamentally by participation in social programs that aim at supporting poor

households.17 Therefore, we estimate R⇥ T ⇥M conditional logit models in the first stage,

which yields the same number of estimates for �r and �rd.

16In school choice models there is no obvious outside option. Therefore, we follow Neilson (2014) andinstead normalize �1 = 0 within each market.

17The SEP program was introduced in 2008 as a targeted voucher for poor students, with the objectiveof providing additional funding for schools serving them. Students’ eligibility to the program is essentiallydetermined by household income and social programs’ participation. All public schools are eligible for SEPvouchers, while only vouchers schools that sign up for the policy are so. Schools that choose to sign up withthe policy must comply with additional programs related to the usage of funds provided by SEP.

15

Page 16: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

The second stage exploits the assumed linear functional form of the utility function in

order to estimate the following linear regression:

rjmt = �

r0,mt +

X

k

xk,jmt�rk + ✏

rjmt (10)

where �r0,mt is a constant term specific to each market, year and household type; �rk measures

the e↵ect of xk on school mean value for households of type r and maps to the preference

parameters of our model; and ✏rjmt is a mean-zero error term. Note that �r0,mt + ✏

rjmt maps to

the unobserved school characteristic ⇠rjmt in our model.

A concern with this regression is the potential endogeneity of school characteristics, in

particular of prices and quality. Therefore, we estimate this regression using an instrumental

variables approach. We use various instruments. First, for each school, we include as instru-

ments the non-price and non-quality characteristics of other schools in the market, in lines

with instruments suggested in Berry et al. (1995). Second, we follow Neilson (2014) and use

teachers’ wages of other schools in the market, which arguably operates as a cost shifter for

schools, such that it may a↵ect their choices of prices, but not their unobserved attributes.

Third, we use the amounts of funding provided by di↵erent components of voucher programs

to schools, which display within market variation due to school characteristics that are fixed

in the short run. Moreover, we instrument for quality o↵ered by schools using a dummy

variable for whether a school was awarded a SNED prize in its most recent version before

each school choice year. This instrument is motivated by Contreras and Rau (2012), who

show how these prizes impact quality in subsequent years.18 Finally, we utilize tempera-

ture data at the county level on the day in which the quality signal was generated (i.e., the

days in which the SIMCE test was taken) as an instrument for quality, which is motivated

by a literature that studies the relationship between climate and academic achievement, as

discussed in Gra↵ Zivin et al. (2015).19

We estimate the model using data for years 2011 through 2014, the only years in which

students’ location data is available. In addition, we only utilize data for students in 1st

18In practice, we utilize the residual of a regression of the SNED award dummy on quality in the year ofthe award, in order to further control for di↵erences in quality between SNED awardees and non-awardeesthat may be driven by other factors that could be persistent in time.

19We construct this variable using data from the Berkeley Earth dataset, which provides population-weighted estimates of daily temperature at the county level. In implementing the regression, we includeboth temperature and temperature squared in order to account for non-linear e↵ects of temperature onacademic achievement documented in Gra↵ Zivin et al. (2015)

16

Page 17: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

grade in order to focus on the margin in which most school choices are made. In terms of

covariates to be included in the vector Xj, we include school fees, quality as measured by

average SIMCE test score of the school, whether the school has a religious orientation or not,

whether a school is public, and whether a school is part of the SEP program.20 Finally, we

are able to compute the distance between households and schools using georeferenced data

on their locations.21

4.1.2 Market definition and estimating dataset

Defining markets in contexts of spatial competition is di�cult because determining which

suppliers belong to the choice set of consumers is not straightforward. In contrast to the

case of many school systems, in Chile there are no institutional constraints that limit the

extent to which students can travel. Therefore, we need to define markets.

We adopt an approach based on the spatial distance between schools. Distance has been

shown to be the most relevant determinant of school choice by previous studies in the lit-

erature. In our data, students average distance to chosen schools is of 2 kilometers (1.3

miles) and the 90th percentile of such distribution is 4.8 kilometers (3 miles). Therefore, it

is sensible to argue that schools located far enough from each other may belong to di↵erent

educational markets. We define an educational market as a cluster of schools in a closed

polygon with no other school closer than 5 kilometers (3.1 miles) from its boundaries. Oper-

ationally, a market is uniquely identified from the adjacency matrix of schools, where links

are defined as two schools being closer than 5 kilometers from each other. In implementing

this procedure, and therefore in estimation as well, we only consider schools located in urban

areas. For estimation, we only include markets with at least 20 schools and for which we

have data for at least 300 students. The map presented in Figure A.7 provides an example

for the resulting market definitions and Table A.1 displays summary statistics for it.22

A description of the resulting sample is displayed by Table 3. The number of households

20We use data on monthly copayment faced by households as a measure of school fees. Moreover, we usedata on whether students’ eligibility for SEP in order to adjust school fees accordingly: eligible students donot pay any school fees in schools that operate under the SEP regime.

21We compute the Euclidean distance between every household and school in each market. We thenproceed to clean these results by (i) removing mass points, which arise from imperfect georeference; and (ii)removing students located further than 55 kilometers from the median location in the market.

22As a robustness exercise, we estimated the model using counties as markets. For estimation, we includedcounties for which a large share of students resided in the market (at least 90%) and were we had availabledata for more than 300 students. Our estimates were quantitatively similar.

17

Page 18: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

types is R = 2, the number of markets included in the sample is M = 25, and the number

of periods covered is T = 4. Therefore, the estimating dataset is comprised by 200 cells.

The estimating dataset includes 1,556 schools and 97,471 students. On average, 32.6% of

the students attending schools in markets in our sample are included in it, and 91.7% of

the schools operating in each market are so respectively. Moreover, an average of 49.4% of

students included in the sample across markets are eligible for the SEP program, which is

the subsample of poor students.23

4.1.3 Results

Given that the most relevant dimension of households’ types is socioeconomic levels, we

present all the results by poor and non-poor households separately. Figure A.8 displays

the resulting coe�cients in each market for distance between households and schools, for

both poor and non-poor households. In all these cases, the coe�cient is negative, which

reflects a decreasing utility for households from choosing a school further away from home.

Interestingly, poor households show to be in average 14.2% more distance-sensitive than

non-poor households.

Table 4 presents results for di↵erent specifications of instrumental variables linear re-

gressions of the estimates of �jmt on di↵erent sets of school characteristics and fixed e↵ects.

Columns 1 through 3 display results for all households in the sample, columns 4 through

6 display results for poor households and columns 6 through 9 do so for non-poor house-

holds. Overall, results point in the expected direction: households’ utility is decreasing in

fees charged by schools and increasing in their reported quality. Adding market-year fixed

e↵ects to the regression increases the point estimates with respect to the baseline case, while

adding other school attributes do not change them substantially.

There are interesting patterns of heterogeneity across poor and non-poor households.

For instance, our preferred specifications in columns 6 and 9 imply that poor households

are 92.3% more price-sensitive than non-poor households. Inversely, poor households are

estimated to be 38% less quality-sensitive than non-poor households. These results imply

in turn that the willingness to pay for quality of non-poor households is almost 3 times

higher than that of poor households. This heterogeneity suggests that policies regarding

23We tested for di↵erences in observables across students included and excluded of the sample, within eachmarket. While some of the di↵erences across groups are statistically significant, these are not economicallysignificant and do not show a clear pattern.

18

Page 19: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

quality disclosure will have heterogeneous e↵ects across these demographic groups. These

patterns of heterogeneity coincide with previous findings within the school choice literature

(e.g., Gallego and Hernando 2009, Hastings et al. 2009, and Neilson 2014).

4.2 Welfare analysis

In our setting, schools’ quality signals are distorted and therefore households choose schools

on the basis of a misperceived vector of attributes. A key aspect of the situation however,

is that while perceived school quality may be di↵erent than true quality, the school quality

households ultimately obtain from a school is the true quality of their school choice. Through-

out this section, we emphasize this aspect and account for it when measuring implications

of distorted quality signals.

In order to compute the e↵ects of distorted quality signals on choices and welfare, we de-

fine two scenarios: baseline and counterfactual. The former corresponds to the environment

in which households actually choose schools. The latter corresponds to a counterfactual

world in which households are provided with undistorted information about school quality.

This exercise rules out changes in other variables (e.g., schools fees and school investments)

as well as the existence of capacity constraints.

Throughout this section, we utilize our estimates for �r and �rd, and the observed vector

of schools’ characteristics Xj to compute choice probabilities and consumer welfare for the

baseline scenario. For the case of the counterfactual scenario, calculations additionally use

estimates of �rk from the second stage of the demand model, and a counterfactual vector of

school characteristics Xij = [pj, qj,Wj], where qj stands for undistorted quality of school j.24

4.2.1 Choices

We begin the analysis by examining schools’ choice probabilities by households across both

scenarios. We do so by adjusting the choice probabilities predicted by equation (9) of our

demand model and using the estimates from such model and the data on school attributes

for both scenarios. Following equation (9), choice probabilities are therefore computed as

as p

rijmt(d

r,

ˆ

r, �

rd) and p

rijmt(d

r,

˜

r, �

rd), where �

rjmt =

Pk xk,jmt�

rk + ⇠

rjmt is the mean util-

ity of school j in market m in period t, computed using demand estimates and data on

24More precisely, we utilize the results for the second stage from our preferred specifications: columns 6and 9 of Table 4.

19

Page 20: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

counterfactual school school quality.

Figure A.9 displays the computed changes in choice probabilities between both scenar-

ios. It is interesting to note that, despite the fact that the magnitude of the distortions is

moderate, there is significant heterogeneity. This pattern holds when restricting the analysis

to the set of schools actually chosen by parents, as displayed by Figures A.9-C and A.9-D.

This shows that changes in the quality disclosure system would induce changes in choices by

households. However, given that households have a limited number of schools in their choice

sets, these changes in choice probabilities may only induce actual changes in choices for a

small fraction of households. This, as marginal changes in the observed vector of schools’

quality may not be strong enough as to induce households to change their schools choices.

Note that non-poor households display more variation in the computed changes, which is

driven by their higher quality-sensitivity. Relatedly, a simple simulation based on the pro-

posed model and our estimates shows that 2.2% of poor households and 2.7% of non-poor

households would be induced to changing their school choice when provided undistorted

quality information. We denote this subpopulation as switchers.25

We compute the predicted attributes of schools chosen by households under both scenar-

ios. Table 5 displays results from these calculations for both poor and non-poor households.

We report both the average across all households as well as the average within the subpop-

ulation of switchers. Columns 1 and 3 in Table 5 display results for the average across all

households. It is easy to note that changes in predicted distance to chosen schools and fees

charged by those are small. This is as expected, since non-switching households have their

choices una↵ected by the information policy we evaluate. The average changes in attributes

of chosen schools by poor and non-poor households are not larger than 0.03 s.d. for any of

the attributes we consider.

When considering the subpopulation of switchers, however, changes in attributes of cho-

sen schools between both scenarios are substantive. Columns 2 and 4 in Table 5 display

results for these groups of households. First, note that in the baseline scenario, switchers

where receiving substantially less quality than the average household, which suggests that

switchers come out mostly from schools that had highly distorted quality signals. Conditional

on switching, we observe that households choose to travel substantially longer distances to

chosen schools, to pay higher fees and, importantly, that they choose schools with remarkably

25We calculate switching rates by simulating choices of consumers in our sample in both the baseline andcounterfactual scenarios. Reported results corresponds to average switching rates for poor and non-poorhouseholds over 1,000 simulations across all households in the sample.

20

Page 21: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

higher levels of true quality. In particular, our results show that poor (non-poor) switchers

would choose schools with 1.02 s.d. (0.8 s.d.) higher true quality in the counterfactual than

the baseline scenario. This would be coupled by an increase in fees charged by chosen schools

of 0.79 s.d. (0.4 s.d.) for poor (non-poor) switchers and, similarly, an increase in distance

travelled to chosen schools of 0.1 s.d. (0.07 s.d.). These results imply that the subpopu-

lation of switchers would change its choices in a substantial way. Switchers move towards

higher-quality schools, for which they would be willing to both travel more and pay higher

fees.

4.2.2 Welfare

We now calculate the welfare changes of providing undistorted quality signals. In the base-

line scenario households choose schools using the observed measure of school quality which,

as discussed, is distorted. However, the e↵ective utility of consumers is determined by undis-

torted school quality. Thus, our baseline scenario is a case in which choice utility and

experience utility di↵er (Train, 2015). This is not the case in the counterfactual scenario, in

which households choice and experience utility coincide.

Let uij be the utility of household i from school j under distorted school quality, choice

utility. Similarly, let uij be the utility of household i from school j under undistorted school

quality, experience utility. In our setting, these two utilities are related. Given that the

only di↵erence between choice and experience utility is the misperception of quality under

the former, we know that uij = uij + ⌧j, where ⌧j measures the wedge between choice and

experience utility from school j. Under the utility function assumed in section 4.1, we know

that ⌧j = �q(qj � qj).26

The choices household i would make in each scenario would be:

j

⇤i = argmax

j{uij}j2Ji

j

⇤i = argmax

j{uij}j2Ji

which may or may not di↵er. Importantly, if the choice is the same in both scenarios then

there is no welfare loss from distorted quality signals for that households, as experience utility

is the same in both cases. This makes clear that welfare losses will be driven by households

26From this expression, it becomes clear that at baseline all schools with positive distortions have ⌧j < 0,such that experience utility from those schools is lower than choice utility from them.

21

Page 22: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

that were changing their behavior in the baseline scenario due to distorted quality signals,

the switchers.

The change in household welfare from providing undistorted information would therefore

be the di↵erence in experience utility between the counterfactual and baseline scenarios,

uij⇤ � uij⇤ . Using the fact that uij⇤ = uij⇤ + ⌧j⇤ , we can compute the expected welfare change

as:

E[�Wi] =1

�p

"log

X

j

exp(vij)� logX

j

exp(vij)�X

j

Pij⌧j

#(11)

where we define vij = �j + �ddij and vij = �j + �ddij for notational simplicity. The first

and second terms measure consumer surplus under undistorted and distorted school quality

information respectively, and the results follow from the inclusive value formula in Small

and Rosen (1981) given the assumed utility function. The third term measures the ex-

pected di↵erence between choice and experience utility at baseline. Dividing by �p simply

transforms the welfare loss to monetary units. This formula calculates the average welfare

gain over the whole sample of households. We can then compute average welfare gains for

switchers or aggregate these gains across di↵erent dimensions. This welfare gains can alter-

natively be interpreted as the average willingness to pay for undistorted quality information

by households.

The results from show that expected welfare would increase in the counterfactual sce-

nario for all households. This is expected, non-switchers will obtain the same welfare in both

scenarios, while switchers will be strictly better o↵ in the counterfactual. For poor house-

holds, the average welfare gain we estimate is less than 1 USD. The average welfare gain for

non-poor households is also less than 1 USD. Scaling up these results for the educational

system, welfare gains would add up to 472,824 USD annually.

These results suggest small gains from providing undistorted information across all house-

holds. However, the relevant population for an intervention like the one proposed in our coun-

terfactual is not the average household, but rather the subpopulation of switchers, households

that would change their school choice in the counterfactual with respect to the baseline sce-

nario. Welfare gains are larger for this subpopulation. The average welfare gain for switchers

is of 5 USD among poor households and of 14 USD among non-poor households.

22

Page 23: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

4.2.3 Heterogeneity in welfare gains

The fact that non-poor households benefit more than poor households from the information

policy is evident, and the magnitudes of the di↵erences, notable. There are two explanations

for these di↵erences. First, the former are more quality-sensitive, less price-sensitive and less

distance-sensitive than the latter. Therefore, they will be more willing to take advantage of

relative changes in perceived quality of schools in the market. A second channel that may

explain part of the di↵erences is related to the spatial distribution of households and schools

in the market, which di↵ers systematically across poor and non-poor households.

We can use our model and estimates to explore how heterogeneity in preferences and

market opportunities determine the observed gap in welfare gains from disclosure of true

quality. Results from this additional counterfactual calculations are displayed in Table 6.

We start by studying how di↵erences in preferences determine lower welfare gains for poor

households. First, we let poor households be as quality-sensitive as non-poor households.

The share of switchers among poor households would increase by 1 percentage point to 3.2%,

and the average welfare gains for switchers would increase to 9 USD.

Second, we let poor households have the same preferences as non-poor households over all

schools’ attributes. The share of switchers increases by 0.8 percentage point to 3%. Average

gains for poor switchers in this counterfactual would climb up to 16 USD, more than three

times those in the first counterfactual and higher than those for non-poor switchers.27 These

results imply that di↵erences in preferences are enough to explain the gap in welfare gains

from the proposed information policy across groups. Moreover, they highlight the key role

that households’ quality-elasticity plays in determining the impacts of information policies

for school choice.

Finally, we explore the role that the spatial distribution of schools and households play in

explaining the gap welfare gains across groups. We measure welfare gains from the evaluated

policy for poor households if they were located were non-poor households are. Our results

show that average welfare gains in that setting would be essentially the same we found in our

baseline results above. The share of switchers in this case would be lower than in the first

counterfactual, at 1.9%, while welfare gains for poor switchers would be only slightly larger

27The fact that welfare gains for the poor when endowed with preferences of non-poor households overall attributes are larger than those when endowed with such preference only over school quality comespartly from the fact that we estimate non-poor households to be less price-sensitive. This implies that thewillingness to pay of them for a given increase in quality is higher than under poor preferences, as can benoted in equation 11.

23

Page 24: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

than in such counterfactual, at 5.4 USD. This result implies that, in our setting, di↵erences

in market opportunities faced by poor and non-poor households play basically no role in

explaining the gap in welfare gains from undistorted quality information.

4.2.4 Discussion

We have estimated a school choice model and studied a counterfactual exercise by which

information on undistorted quality signals is provided to households. Results point in three

directions. First, distortions in quality signals have e↵ects in choices, as choice probabilities

would change significantly in the counterfactual scenario. Second, households would react to

the change in the quality disclosure system mostly by increasing the probability of choosing

higher quality schools. This is, there would be a shift of students towards relatively high

quality schools available in the market. Third, our welfare calculation points towards small

average gains across households but larger gains for switchers. In both cases, gains are

larger for non-poor households, which is driven by them being more quality-sensitive and

less price-sensitive. We show however that complementary policies that could increase poor

households’ quality-sensitivity may increase welfare gains from this policy for them. The

magnitude of welfare gains for switchers suggests that it may be socially beneficial to invest

in quality disclosure systems that reduce the extent of distortions in quality signals in the

context of educations markets.

5 Concluding Remarks

In this paper, we have calculated the magnitude of distortions in quality signals in school

markets, and studied the welfare consequences associated to choices made with distorted

information. We have focused on the Chilean educational system, for which we have all the

necessary data to recover distortions. Our findings indicate that distorted quality signals

have the potential to influence school choice. The relevant population a↵ected by a policy

that delivers undistorted information are the switchers, households with di↵erent school

choices in a world with undistorted information. Households whose choices are una↵ected

by distorted information are not a↵ected in terms of welfare. The subpopulation of switchers

in our context is small, which is explained by the fact that the magnitude of distortions in

quality signals is relatively small. Nevertheless, we document that switchers would choose

schools of remarkably higher quality in an environment without distortions, and would derive

24

Page 25: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

welfare gains from adjusting their choices.

The results of this paper shed light on di↵erent policy-relevant aspects. On the one

hand, we have shown that informational policies aimed at improving precision of information

may a↵ect choices. Although this has already been discussed in previous literature, our

work studies a di↵erent margin to improve information. Previous work emphasizes the

importance of having some information. Our work extends previous analysis and emphasizes

the importance of providing undistorted information. On there other hand, we have showed

that heterogeneity in preferences determines the extent to which households can benefit from

information policies. In school markets, we argue that complementary policies that increase

quality-sensitivity of the poor may enable them to obtain further benefits from information.

Our work has a simple and potentially powerful policy implication for school markets.

The findings presented in this paper suggest that quality signals need to be calculated as

a corrected average of test scores instead of an arithmetic average. One way to accomplish

this correction is using the imputation method we have proposed. Importantly, government

institutions that o↵er incentives to achieve full attendance on test day might provoke more

distortions in quality signals. This is more likely to happen if incentives have heterogeneous

impacts on schools. We believe that the imputation method we proposed is the most cost-

e↵ective alternative to correct distorted information. Other policy alternatives, such as

choosing random test days, are inherently more expensive to coordinate and subject to

cheating behavior in the form of collusion agreements among subset of schools.

Promising areas for future research are to study the magnitude and consequences of

distorted information in other sectors of the economy, and to calculate the misallocation of

resources associated to distortions. Distortions in quality signals might cause misallocation

of public resources. Consider, for example, a government program that o↵ers increases in

wages to teachers in schools with higher average test scores. It is possible that teachers will

incorrectly obtain bonuses in some schools due changes in students’ attendance the day of the

test. Another fruitful area for future research corresponds to the measurement of distortions

in environments that are more susceptible to corruption. Interestingly, municipalities with

higher levels of corruption in Chile have larger distortions in the quality signals we study (see

Tables A.2 and A.3). This suggests that our findings might be exacerbated in other countries

with di↵erent levels of corruption. Correcting distorted information in such environments

has the potential to provide substantial welfare gains.

25

Page 26: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

References

Allende, C. (2012). The impact of information on academic achievement and school choice:

evidence from Chilean “Tra�c lights”.

Andrabi, T., Das, V., and Khwaja, A. (2014). Report cards: the impact of providing school

and child test scores on education markets.

Bagwell, K. (2007). The economic analysis of advertising. In Armstrong, M. and Porter, R.,

editors, Handbook of Industrial Organization, volume 3, pages 1701–1844. Elsevier.

Bayer, P., Ferreira, F., and McMillan, R. (2007). A unified framework for measuring prefer-

ences for schools and neighborhoods. Journal of Political Economy, 115(4):588–638.

Berry, S., Levinsohn, J., and Pakes, A. (1995). Automobile prices in market equilibrium.

Econometrica, 63(4):841–890.

Cartwright, E. and Menezes, M. L. (2014). Cheating to win: dishonesty and the intensity of

competition. Economics Letters, (122):55–58.

Contreras, D. and Rau, T. (2012). Tournament incentives for teachers: evidence from a

scaled-up intervention in Chile. Economic Development and Cultural Change, 61(1):219–

246.

Cooper, R., Gallego, F., and Neilson, C. (2013). Types of information and school choice: an

experimental study in the Chilean voucher system.

Dranove, D. and Jin, G. Z. (2010). Quality disclosure and certification: theory and practice.

Journal of Economic Literature, 48(4):935–963.

Dranove, D., Kessler, D., McClellan, M., and Satterthwaite, M. (2003). Is more information

better? the e↵ects of report cards on health care providers. Journal of Political Economy,

111(3):555–588.

Figlio, D. and Loeb, S. (2011). School accountability. In Eric A. Hanushek, S. M. and

Woessmann, L., editors, Handbook of the Economics of Education, volume 3, pages 383–

421. Elsevier.

Figlio, D. and Winicki, J. (2005). Food for thought: the e↵ects of school accountability plans

on school nutrition. Journal of Public Economics, 89(2-3):381–394.

26

Page 27: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Gallego, F. and Hernando, A. (2009). School choice in chile: Looking at the demand side.

Gilpatric, S. M. (2011). Cheating in contests. Economic Inquiry, 49(4):1042–1053.

Gra↵ Zivin, J., Hsiang, S., and Neidell, M. (2015). Temperature and human capital in the

short- and long-run.

Hastings, J., Kane, T. J., and Staiger, D. O. (2009). Heterogeneous preferences and the

e�cacy of public school choice.

Hastings, J. S. and Weinstein, J. M. (2008). Information, school choice, and aca-

demic achievement: Evidence from two experiments. Quarterly Journal of Economics,

123(4):1373–1414.

Jacob, B. A. and Levitt, S. D. (2003). Rotten apples: An investigation of the prevalence

and predictors of teacher cheating. Quarterly Journal of Economics, 118(3):843–877.

Milgrom, P. and Roberts, J. (1986). Price and advertising signals of product quality. Journal

of Political Economy, 94(4):796–821.

MINEDUC (2013). Estadısticas de la educacion 2013.

Mizala, A. and Urquiola, M. (2013). School markets: The impact of information approxi-

mating schools’ e↵ectiveness. Journal of Development Economics, 103(0):313 – 335.

Neal, D. (2013). The consequences of using one assessment system to pursue two objectives.

Journal of Economic Education, 44(4):pp. 339–352.

Neal, D. and Schanzenbach, D. W. (2010). Left behind by design: Proficiency counts and

test-based accountability. The Review of Economics and Statistics, 92(2):263–283.

Neilson, C. (2014). Targeted vouchers, competition among schools, and the academic achieve-

ment of poor students.

Nelson, P. (1970). Information and consumer behavior. Journal of Political Economy,

78(2):311–329.

Rouse, C. E., Hannaway, J., Goldhaber, D., and Figlio, D. (2013). Feeling the Florida heat?

how low-performing schools respond to voucher and accountability pressure. American

Economic Journal: Economic Policy, 5(2):251–81.

27

Page 28: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Rubin, D. B. (1976). Inference and missing data. Biometrika, 63(3):581–92.

Rubin, D. B. (1987). Multiple Imputation for Nonresponse in Surveys. New York: Wiley &

Sons.

Schwieren, C. and Weichselbaumer, D. (2010). Does competition enhance performance or

cheating? A laboratory experiment. Journal of Economic Psychology, (31):241–253.

Shleifer, A. (2004). Does competition destroy ethical behavior? American Economic Review,

94(2):414–418.

Small, K. and Rosen, H. (1981). Applied welfare economics with discrete choice models.

Econometrica, 49(1):105–30.

Train, K. (2015). Welfare calculations in discrete choice models when anticipated and expe-

rienced attributes di↵er: A guide with examples. Journal of Choice Modelling, 16:15–22.

Walters, C. (2014). The demand for e↵ective charter schools.

28

Page 29: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Figure 1: Attendance on test day

-4

-2

0

2

4

Δ attendance

-5 -4 -3 -2 -1 0 1 2 3 4 5

AverageAbove 75thAbove 90thBelow 10thBelow 25th

(a) Di↵erence in average attendance rate (%) between 4th graders (who take the test)and 3rd graders (who do not take the test) around test days in 2013.

0

0.02

0.04

0.06

-15 -10 -5 0 5 10 15 Attendance on test day

(b) Distribution of changes in attendance the day of the test by school in 2013. Thevertical red line denotes the average change in attendance at the school level (2.4 per-centage points).

29

Page 30: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Figure 2: Distortions in quality signals

(a) Estimated distortions with the associated 95% confidence interval

0

0.05

0.10

0.15

0.20

0.25

0 5 10 15 20Distortion in quality signals

(b) Histogram of all distortions in our data set

Distortion in quality signals (y-axis) are defined as (minus) the di↵erence between observedtest scores in a school and a counterfactual scenario in which all students in the school takethe test. Schools are ordered from lower to higher distortions in the x-axis. Vertical linesrepresent the 95% confidence interval. Green (gray) lines represents distortions that are(not) statistically di↵erent from zero. Distortions without confidence interval representschools in which all students took the test. Approximately 5,000 schools every year.

30

Page 31: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Table 1: Descriptive statistics

Obs. Mean St. Dev. p10 p50 p90

Panel A: Schools in 2005-2013

Test score (SIMCE) 54,585 245 31 205 244 287

Students in 4th grade 55,928 39 35 6 29 84

Students absent on test day 55,928 3 4 0 2 7

Average class size 55,810 29 10 15 29 41

Average annual attendance 55,928 94 3 90 94 98

Public 55,928 0.51 0.50 0 1 1

Voucher 55,928 0.42 0.49 0 0 1

Private 55,928 0.06 0.24 0 0 0

Rural 55,928 0.31 0.46 0 0 1

Religious 54,051 0.49 0.50 0 0 1

Monthly fee (Ch$) 55,594 15,989 37,568 0 0 57,013

Panel B: Students in 2013

Test score (SIMCE) 140,982 263 46 200 267 321

GPA 159,356 5.9 0.6 5.1 5.9 6.5

Attendance in test-day 137,604 0.95 0.20 1.0 1.0 1.0

Attendance in non-test day 137,127 0.92 0.17 0.8 1.0 1.0

Notes: Own construction based on administrative data provided by the Ministry of Edu-cation. Attention is restricted to schools in which distortions in quality signals are zero orthere is su�cient data to calculate it. See section 3 for details. There are 8,254 schools inthe period 2005–2013.

31

Page 32: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Table 2: Understanding distortions in quality signals

Dependent variable is distortions in quality signals (in test score points)

AllSome absenteeism

on test dayDistortions> 0

(1) (2) (3) (4) (5) (6)Panel A: Schools’ attributes

Public schools 1.01*** 0.90*** 1.96*** 1.51*** 2.06*** 1.50***(0.03) (0.04) (0.03) (0.04) (0.12) (0.14)

Voucher schools 0.53*** 0.40*** 0.63*** 0.32*** 1.01*** 0.57***(0.03) (0.03) (0.03) (0.04) (0.11) (0.13)

Religious schools -0.21*** -0.12*** 0.03 -0.04* 0.06 -0.03(0.03) (0.03) (0.02) (0.02) (0.06) (0.06)

Average annual attendance -0.24*** -0.25*** 0.01 -0.12*** -0.24*** -0.42***(0.03) (0.03) (0.02) (0.02) (0.04) (0.04)

Students in 4th grade 0.01 -0.03 -0.26*** -0.22*** -0.97*** -0.92***(0.06) (0.05) (0.04) (0.04) (0.12) (0.10)

Enrollment in grades 1st-8th 0.07 0.09* -0.22*** -0.20*** -0.04 -0.03(0.06) (0.05) (0.04) (0.04) (0.12) (0.11)

Constant (private schools) 0.37*** 0.42*** 1.18*** 1.46*** 3.61*** 4.04***(0.03) (0.04) (0.03) (0.04) (0.13) (0.14)

Panel B: Autocorrelation

Lagged distortion 0.19*** 0.12*** 0.28*** 0.22*** 0.37*** 0.29***(0.01) (0.01) (0.01) (0.01) (0.01) (0.01)

Constant 0.70*** 0.84*** 1.24*** 1.36*** 3.19*** 3.38***(0.02) (0.02) (0.01) (0.01) (0.04) (0.04)

Mean of dep. variable 2.75 2.75 3.56 3.56 6.84 6.84Municipality F.E. No Yes No Yes No YesExplained by schools F.E. 0.35 0.35 0.42 0.42 0.69 0.69Schools 7,612 7,612 5,945 5,945 4,295 4,295Observations 54,043 54,043 39,020 39,020 11,066 11,066

Notes: All non-indicator variables have been normalized (except for lagged distortion). All regressions areweighted by the inverse of the uncertainty associated to the calculation of distortions, where uncertainty isthe size of the confidence interval. Columns 3-4 restrict attention to schools in which at least 1 student didnot take the test. Columns 5-6 restrict attention to school-year observations with distortions statisticallydi↵erent from zero. Robust standard errors in parentheses. Significance level: *** p<0.01, ** p<0.05, *p<0.1.

32

Page 33: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Table 3: Summary statistics of demand estimation dataset

Variable Mean SD p10 p50 p90

Students In sample 1,009 844 324 665 2,446Coverage rate 33% 12% 17% 31% 48%

Schools In sample 63 62 19 45 134Coverage rate 92% 13% 72% 97% 100%

Poor students In sample 479 391 166 323 1,184Sample share 49% 11% 35% 50% 60%

Notes: This table displays market-level summary statistics for the sample utilized for demand estimation.The sample covers 25 markets for the period 2011–2014. For the number of students and the number ofschool per market, summary statistics for levels and coverage rate of full school markets are provided. Forthe number of poor students, summary statistics for levels and for their share over the sample size for eachmarket are provided.

33

Page 34: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Table4:IV

resultsfrom

thesecondstageof

dem

andestimation

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

(9)

Allsample

Poo

rstudents

Non

-poo

rstudents

Fee

-0.006***

-0.008***

-0.011***

-0.012***

-0.013***

-0.019***

-0.004***

-0.006***

-0.010***

(0.001)

(0.001)

(0.001)

(0.001)

(0.001)

(0.001)

(0.001)

(0.001)

(0.001)

Quality

0.012***

0.019***

0.019***

0.004**

0.011***

0.014***

0.021***

0.028***

0.022***

(0.001)

(0.001)

(0.001)

(0.002)

(0.001)

(0.002)

(0.002)

(0.002)

(0.002)

Religious

-0.054**

-0.086***

-0.018

(0.024)

(0.030)

(0.029)

Gender

constraint

0.154***

0.128**

0.169***

(0.046)

(0.059)

(0.054)

Public

0.083***

0.221***

-0.066*

(0.031)

(0.039)

(0.039)

SEPschoo

l-0.323***

-0.525***

-0.580***

(0.064)

(0.092)

(0.065)

Market-YearF.E.

No

Yes

Yes

No

Yes

Yes

No

Yes

Yes

Observations

10,774

10,774

10,774

5,335

5,335

5,335

5,439

5,439

5,439

First

stageF-test

F-testFee

1,166

1,814

362

393

515

68907

1,429

301

F-testQuality

77.6

18.3

15.4

36.7

10.1

8.4

40.3

8.3

7.0

Notes:Instrumentalvariab

leestimates

utilize

twosets

ofinstruments:(i)theam

ount

awarded

byschoo

lvouchers,meanfixed

characteristicsof

rivalsin

themarket(i.e.BLPinstruments)an

drivals’marketwages

areusedas

instruments

forschoo

ls’fees;

and(ii)alinearan

dqu

adraticterm

oncounty-specifictemperature

andtheresidual

ofaregression

ofbeingaw

arded

aSNED

prize

inthepreviou

speriodon

lagged

schoo

lqu

alityareuse

asinstruments

forschoo

lqu

ality.

Allregression

sareweigh

tedby

schoo

lenrollment.

Rob

ust

stan

darderrors

inparentheses.***p<0.01,**

p<0.05,*p<0.1.

34

Page 35: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Table 5: Means of predicted school attributes of households’ choices

(1) (2) (3) (4)

Poor students Non-poor students

Attribute Scenario Average Switchers Average Switchers

Distance Baseline 2.3 2.4 2.6 2.6Counterfactual 2.3 2.6 2.6 2.8Change 0.0 0.2 0.0 0.2

Fee Baseline 8.7 12.8 36.6 23.5Counterfactual 8.8 46.3 37.2 52.3Change 0.2 33.5 0.6 28.8

Quality Baseline 256 248 268 255Counterfactual 256 273 268 276Change 0 25 0 21

Notes: Columns 1 and 3 display average attributes of chosen schools across all poor andnon-poor households. Columns 2 and 4 display average attributes of chosen schools for poorand non-poor switchers. Results for distance are measured in kilometers, results for schoolfees are measured in thousands of Ch$ and results for quality are measured in SIMCE testscores, net of distortions.

35

Page 36: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Table 6: Welfare gains from true quality disclosure

(1) (2) (3) (4) (6) (6)

Poor students Non-poor students

Average Switchers’ Switch Average Switchers’ SwitchComparison E[�Wi] E[�Wi] rate E[�Wi] E[�Wi] rate

Counterfactual scenario $0.11 $5.15 2.22% $0.36 $13.45 2.67%

Poor households withnon-poor preference for quality $0.27 $8.59 3.20% - - -

Poor households withnon-poor preferences $0.47 $15.73 2.97% - - -

Poor households withnon-poor market opportunities $0.11 $5.44 1.95% - - -

Notes: All welfare results correspond to the average welfare gain between the counterfactualand baseline scenarios under each setting for each subpopulation. Columns 1 and 3 displayaverage welfare gains across all poor and non-poor households. Columns 2 and 4 displayaverage welfare gains for poor and non-poor switchers. Columns 3 and 6 display the share ofswitchers among the population of poor and non-poor households respectively. All welfareresults are measured in US dollars.

36

Page 37: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

For Online Publication

Distorted Information: Evidence from Quality Signals in School Markets

Jose Ignacio Cuesta, Felipe Gonzalez, and Cristian Larroulet

Contents

A Understanding Distortions in Quality Signals 3

A.1 Competition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

A.2 Monetary incentives for teachers . . . . . . . . . . . . . . . . . . . . . . . . . 3

A.3 Educational tra�c lights . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

List of Figures

A.1 Predictability of test scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

A.2 Distribution of rank correlations over time . . . . . . . . . . . . . . . . . . . 8

A.3 Distortions on own variables . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

A.4 Distortions on variables of schools within 3km . . . . . . . . . . . . . . . . . 10

A.5 Monetary incentives for teachers . . . . . . . . . . . . . . . . . . . . . . . . . 11

A.6 “Educational Tra�c lights” policy . . . . . . . . . . . . . . . . . . . . . . . . 11

A.7 Market definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

A.8 Estimated coe�cients on distance from the first stage . . . . . . . . . . . . . 13

A.9 Changes in choice probabilities . . . . . . . . . . . . . . . . . . . . . . . . . . 13

List of Tables

A.1 School markets as connected components . . . . . . . . . . . . . . . . . . . . 14

A.2 Schools in corrupt municipalities have larger distortions . . . . . . . . . . . . 15

1

Page 38: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

A.3 Di↵erences-in-di↵erences of audits . . . . . . . . . . . . . . . . . . . . . . . . 16

2

Page 39: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

A Understanding Distortions in Quality Signals

A.1 Competition

A competitive environment could change incentives to distort quality signals. We generate

a school-year specific measure of competition calculating the number of schools operating

within 3 kilometers. If competitive pressure pushes schools to signal higher quality in order

to attract demand, then we would expect competition to have a positive e↵ect on distortions.

Conversely, if competition disciplines schools, we could expect competition to have a negative

e↵ect on distortions.

To test for the e↵ect of competition, we estimate the following equation:

b jt = f(Xjt) + ⌘j + ⌫t + "jt

where b jt are the estimated distortions in quality signals, and Xjt is the number of com-

petitors within 3 kilometers as independent variable. Given the inclusion of school and year

fixed e↵ects, we are using within school variation over time. This means that we are using

�Njt = Entryjt � Exitjt as variation. Grau et al. (2015) documents significant variation in

this variable. Panels A-C in Figure ?? presents results. There is no economically significant

variation of distortions with the competitiveness of the environment.

A.2 Monetary incentives for teachers

An additional mechanism that could explain distortions are incentives placed by performance

pay systems that reward teachers based on test scores. In the market we study, there is

biannual performance pay contest called SNED which we can use to test for the role of

teachers. SNED operates as follows: (i) groups of homogeneous schools are constructed; (ii)

every two years, a multidimensional index is computed at the school level, which considers

academic performance, improvement, and socioeconomic integration among other outcomes;

(iii) schools are ranked within their groups according to the value of such index; and (iv)

all teachers in schools covering the 25% � 35% of the total enrollment of each group get a

monetary prize equivalent to around 40% of a teacher’s wage.1 Importantly, SIMCE test

1Since 2006, the coverage of the prize was increased to 35% of the enrollment of the group. More detailsabout this program can be found in Contreras and Rau (2012).

3

Page 40: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

scores account for 70% of the weight in the index calculation.

Given that (i) prizes are provided according to an index, and (ii) after each contest schools

are informed of their outcomes, we can use a school’s index as a measure of incentives. We

compute the distance of each school to the threshold for obtaining the prize. Schools closer

to the threshold have more incentives to increase their test scores through distortions than

those further away from the threshold either upwards (sure winners) or downwards (sure

losers). Using this rationale, we estimate:

jt = 1INfIN(SNEDIN

jt�1) + 1OUTfOUT (SNEDOUT

jt�1 ) + ⌘j + ⌫t + "jt (1)

where SNEDINjt�1 measures distance to the threshold for winners, and SNEDOUT

jt�1 measures

distance to threshold for losers, both in terms of index points. We use information from

the previous contest to construct these variables. Our objects of interest are the functions

f IN and fOUT . If schools closer to the threshold have larger distortions, there is evidence of

teachers manipulating attendance on test day.

Figure A.5 presents six di↵erent plots for the relationship between distortions and schools’

distance to the threshold. We present results for the two years after the prize is awarded

and both for raw distortions in quality signals and residualized distortions (net of school

and year fixed e↵ects, as well as school characteristics). Estimates of f IN and fOUT show, if

anything, the opposite pattern: schools closer to the cuto↵ have lower or similar distortions

to quality signals. These results suggest teachers are unlikely to be the source of distortions.

A.3 Educational tra�c lights

Other quality disclosure policies could incentivize schools to introduce distortions in qual-

ity signals. In 2010, the Ministry of Education implemented a policy called “Educational

Tra�c Lights” (ETL) that we can be use to test for this mechanism. The ETL policy

consisted in sending information about 2009 SIMCE test scores to all households. This in-

formation included test scores and a classification of schools in “Red”, “Yellow” or “Green”

categories, with clear cuto↵s in test scores determining the category. Allende (2012) uses

these discontinuities and finds that the policy a↵ected school enrollment: households in red

schools responded by enrolling in yellow schools, and households in yellow schools enrolled

more in green schools. The ETL policy generated discontinuities in perceived school quality.

Therefore, we expect schools closer to the cuto↵s to have larger distortions.

4

Page 41: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Figure A.6 presents the linear relationships between test scores and distortions around the

ETL policy cuto↵s. Again, we present results for distortions and residualized distortions.2

These plots show that distortions increase around the cuto↵ between red and yellow schools.

This means that schools introduce larger distortions in order to move towards the yellow

category or avoid moving to the red category. This pattern, however, is not the same around

the second cuto↵. These results provide evidence that (some) low-quality schools generate

distortions to signal higher quality.

2Note that this is a cross-sectional exercise, so we cannot include school and year fixed e↵ects in thiscase, just school characteristics.

5

Page 42: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

References

Allende, C. (2012). The impact of information on academic achievement and school choice:

evidence from Chilean “Tra�c lights”.

CIPER (2012). Ley sep: Contralorıa revela masiva perdida de recursos destinados a edu-

cacion de los mas pobres. Centro de Investigacion Periodıstica.

Contreras, D. and Rau, T. (2012). Tournament incentives for teachers: evidence from a

scaled-up intervention in Chile. Economic Development and Cultural Change, 61(1):219–

246.

Grau, N., Hojman, D., and Mizala, A. (2015). Destructive creation: school turnover and

educational attainment.

6

Page 43: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Figure A.1: Predictability of test scores

200

250

300

350

Test

sco

re

4.5 5 5.5 6 6.5 7GPA

Coe�cient estimates and 95% confidence interval from a linear regression of test scoreon (1) a full set of indicators for a student’s GPA, and (2) school fixed e↵ects. Standarderrors clustered at the school level. Gray lines indicate the mean.

7

Page 44: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Figure A.2: Distribution of rank correlations over time

.5

.6

.7

.8

.9

1

2005 2006 2007 2008 2009 2010 2011 2012 2013

25th percentile 10th percentile 5th percentile

(a) Percentiles in rank correlation distribution f(⇢mt)

0

.1

.2

.3

.4

.5

2005 2006 2007 2008 2009 2010 2011 2012 2013

(b) Percentage of markets with changes in ranking (i.e., ⇢mt < 1)

Notes: Let ⇢mt be the rank correlation of distorted and undistorted quality in marketm and year t. Approximately 210 markets every year.

8

Page 45: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Figure A.3: Distortions on own variables

-2

-1

0

1

2

-4 -2 0 2 4

(a) Average attendance rate

-2

-1

0

1

2

-10 -5 0 5 10

(b) Class size

-2

-1

0

1

2

-100 -50 0 50 100

(c) Enrollment

-2

-1

0

1

2

-4 -2 0 2 4

(d) Missing students on test day

-2

-1

0

1

2

-3000 -2000 -1000 0 1000 2000

(e) Monthly fee

-2

-1

0

1

2

-.4 -.2 0 .2 .4

(f) Share SEP students

-2

-1

0

1

2

-40 -20 0 20 40

(g) Distorted test score

-2

-1

0

1

2

-40 -20 0 20 40

(h) Undistorted test score

-2

-1

0

1

2

-20 -10 0 10 20

(i) Students in 4th grade

Notes: All variables have been residualized with school and year fixed e↵ects. Themean of distortion is 2.7 test score points.

9

Page 46: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Figure

A.4:Distortionson

variab

lesof

schoo

lswithin

3km

-1-.50.51

-1-.5

0.5

11.5

(a)Publicschools�i

-1-.50.51

-2-1

01

2

(b)Private-voucher

�i

-1-.50.51

-.50

.5

(c)Private

�i

-1-.50.51

-1000

-500

0500

1000

(d)Enrollment�i

-1-.50.51

-20

-10

010

20

(e)Average

test

score�i

-1-.50.51

-5000

05000

(f)Averagefees

�i

-1-.50.51

-4-2

02

4

(g)Averagedistortion�i

-1-.50.51

-.04

-.02

0.02

.04

(h)HHIIndex

8i

-1-.50.51

-10

-50

510

(i)Perc.

fees

i

-1-.50.51

-40

-20

020

40

(j)Perc.

test

score

i

-1-.50.51

-20

-10

010

20

(k)Perc.

enrollmentshare

i

-1-.50.51

-40

-20

020

40

(l)Perc.

schoolattendance

i

Notes:Allvariab

leshavebeenresidualized

withschoo

lan

dyear

fixede↵

ects.Themeanof

distortionis

2.7test

score

points.

10

Page 47: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Figure A.5: Monetary incentives for teachers

2.0

2.5

3.0

3.5

4.0

Dis

tort

ion

-20 -10 0 10 20Slots from winning prize

(a) Distortion in year t+ 1

-1.0

-0.5

0.0

0.5

Dis

tort

ion

resi

dual

s

-20 -10 0 10 20Slots from winning prize

(b) Distortion in year t+ 1

2.0

2.5

3.0

3.5

Dis

tort

ion

-20 -10 0 10 20Slots from winning prize

(c) Distortion in year t+ 2

-1.5

-1.0

-0.5

0.0

0.5D

isto

rtio

n re

sidu

als

-20 -10 0 10 20Slots from winning prize

(d) Distortion in year t+ 2

Figure A.6: “Educational Tra�c lights” policy

-5

0

5

10

Dis

tort

ion

-3 -2 -1 0 1 2 3Traffic lights score

(a) Distortion in year 2010

-5

0

5

10

Dis

torti

on re

sidu

als

-3 -2 -1 0 1 2 3Traffic lights score

(b) Distortion in year 2010

11

Page 48: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Figure

A.7:Marketdefinition

Notes:See

description

ofTab

leA.1

fordetailsab

outmarketdefinition.

12

Page 49: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Figure A.8: Estimated coe�cients on distance from the first stage

0

.05

.1

.15

.2

.25

Frac

tion

-1.5 -.75 0 .75 1.5Coefficient on Distance

(a) Poor students, distance

0

.05

.1

.15

.2

.25

Frac

tion

-1.5 -.75 0 .75 1.5Coefficient on Distance

(b) Non-poor students, distance

Figure A.9: Changes in choice probabilities

0

.1

.2

.3

Frac

tion

-30 -15 0 15 30% Change in choice probability

(a) Poor students, all schools

0

.1

.2

.3

Frac

tion

-30 -15 0 15 30% Change in choice probability

(b) Non-poor students, all schools

0

.1

.2

.3

Frac

tion

-30 -15 0 15 30% Change in choice probability for actual choice

(c) Poor students, chosen school

0

.1

.2

.3

Frac

tion

-30 -15 0 15 30% Change in choice probability for actual choice

(d) Non-poor students, chosen school

13

Page 50: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Table A.1: School markets as connected components

3km 4km 5km 6km 7km 8km 9km 10km

Markets 451 413 380 348 322 295 273 251

Markets with more than 1 schools 262 248 233 219 208 196 191 181

Markets with more than 5 schools 106 104 99 93 90 88 86 86

Markets with more than 10 schools 63 63 60 55 52 49 48 50

Markets with more than 20 schools 36 36 33 31 30 29 28 29

Notes: Let A be a N ⇥ N matrix representing the network of N =5,416 urban schools inChile in the period 2005–2013. In network theory, A is referred to as adjacency matrix.This adjacency matrix represents an undirected network, i.e., A is a symmetric matrix. Theelement A(i, j) in this adjacency matrix takes the value of one if school i and j are closer than kilometers from each other and zero otherwise. A “component” or “connected component”of A is a sub-network in which any two schools are connected to each other through someother school, i.e., we can always find a “path” that connects any two pair of schools in thesub-network. A market is defined as a connected component of A. In the paper, we use = 5 (highlighted in gray), but results are robust to di↵erent definitions.

14

Page 51: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Table A.2: Schools in corrupt municipalities have larger distortions

Dependent variable is distortions (in test score points)

Years withtransfers

Before auditsrevealed

After auditsrevealed

(1) (2) (3)

Irregular payments 0.33*** 0.38*** 0.22**(0.07) (0.09) (0.10)

Government transfers 0.06 0.05 0.10(0.04) (0.05) (0.08)

Public schools 1.04*** 0.99*** 1.12***(0.08) (0.10) (0.14)

Voucher schools 0.73*** 0.71*** 0.74***(0.07) (0.09) (0.13)

Religious schools -0.03 -0.07 0.04(0.09) (0.10) (0.15)

Average annual attendance -0.19*** -0.15* -0.30***(0.07) (0.08) (0.11)

Students in 4th grade 0.25 0.35* 0.09(0.17) (0.20) (0.30)

Enrollment in grades 1st-8th -0.24 -0.30 -0.17(0.17) (0.20) (0.30)

Constant (private schools) 0.42*** 0.45*** 0.36***(0.08) (0.11) (0.11)

Schools 2,426 2,332 2,197Municipalities 76 76 76Observations 11,426 7,349 4,077

Notes: All variables have been normalized. All regressions are weighted by the inverse ofthe size of the confidence interval of distortions to account for estimation of the dependentvariable. Audits in 76 randomly chosen municipalities were implemented by the ComptrollerGeneral of Chile to disclose irregular payments from government transfers. The time ofdisclosure of irregular payments was May of 2012. “All years with transfers” correspondto the period 2008–2013. Column 2 restricts attention to years 2008–2012, and column 3restricts attention to years 2012–2013. Robust standard errors in parentheses. Significancelevel: *** p<0.01, ** p<0.05, * p<0.1.

15

Page 52: Distorted Information: Evidence from Quality Signals in School … · 2019-12-19 · Distorted Information: Evidence from Quality Signals in School Markets⇤ Jos´e Ignacio Cuesta†

Table A.3: Di↵erences-in-di↵erences of audits

Dependent variable is distortions (in test score points)

All schoolsSchools in auditedmunicipalities

(1) (2) (3) (4)

Audit ⇥ Post 0.04 0.07(0.10) (0.09)

Corrupt ⇥ Post -0.29* -0.32**(0.15) (0.14)

Post -0.05 -0.06 0.12 0.15(0.05) (0.05) (0.11) (0.10)

Mean of dep. variable 2.8 2.8 2.8 2.8School-level controls No Yes No YesMunicipality F.E. Yes Yes Yes YesMunicipalities 344 344 76 76Schools 7,335 7,335 2,526 2,526Observations 36,951 36,188 11,629 11,426

Notes: These regressions restrict attention to the period in which the government transferredmonetary resources to be spent under the Subvencion Escolar Preferencial program (2008–2013). All regressions are weighted by the inverse of the size of the confidence interval ofdistortions to account for estimation of the dependent variable. Audits in 76 randomly chosenmunicipalities were implemented by the Comptroller General of Chile to disclose “irregular”expenditures of government transfers. The time of disclosure of irregular payments was Mayof 2012. The post period are years 2012 and 2013. The “Corrupt” indicator takes the value ofone if a municipality has more than 10 percent of the government transfers under “irregularpayments.” More about irregular payments can be found in CIPER (2012). Standard errorsclustered at the municipality level in parentheses. Significance level: *** p<0.01, ** p<0.05,* p<0.1.

16