Rank and Response:

A Field Experiment on Peer Information and Water Use Behavior

Syon P. Bhanot∗

June 23, 2017

Abstract

Perception of peer rank, or how we perform relative to our peers, can be a powerful motivator.

While research exists on the e�ect of social information on decision making, there is less work

on how ranked comparisons with our peers in�uence our behavior. This paper outlines a �eld

experiment conducted with 3,896 households in Castro Valley, California, which uses household

mailers with various forms of social information and peer rank messaging to motivate water

conservation. The experiment tests the e�ect of a visible peer rank on water use, and how

the competitive framing of rank information in�uences behavioral response. The results show

that households with relatively low or high water use in the pre-treatment period responded

di�erently to how rank information was framed. I �nd that a neutrally-framed peer rank caused

a small �boomerang e�ect� (i.e., an increase in average water use) for low water use households,

but this e�ect was eliminated by competitive framing. At the same time, a competitively-framed

peer rank demotivated high water use households, increasing their average water use over the

full period of the experiment. This result is supported by evidence that the competitive frame

on rank information increased water use for households who ranked �last� in the peer group � a

detrimental �last place e�ect� from competitively-framed rankings.

∗Swarthmore College. Email: sbhanot1@swarthmore.edu. I would like to thank Richard Zeckhauser, BrigitteMadrian, and Michael Norton for their guidance and advice. Additionally, I want to thank Alberto Abadie, HuntAllcott, Dan Ariely, Gary Charness, John List, Tim McCarthy, Duncan Simester, Monica Singhal, and seminarparticipants at the UK Behavioral Insights Team, UCSD, and Harvard for their feedback. A special thanks is alsodue to Ora Chaiken, Chad Haynes, and Peter Yolles of WaterSmart Software, without whom this work would not bepossible. Finally, I want to acknowledge Peter Hadar, Shahrukh Khan, Stephanie Kestelman, and especially VivienCaetano for their excellent work as research assistants.

1

1 Introduction

Traditionally, economists studying human behavior have focused more on �nancial motivators than

on social norms, peer pressure, and other social motivators. However, when pricing is not salient or

the bene�ts from behavior change are di�use, social motivators can be a useful tool for encouraging

behavior change (Ferraro et al., 2011; Allcott, 2011; Olmstead and Stavins, 2007; Kraft-Todd et al.,

2015; Brent et al., 2015). In particular, recent research in psychology and behavioral economics has

demonstrated that people are in�uenced by how they compare to their peers, and motivated by the

desire to obtain a high rank relative to others (Schultz et al., 2007; Beshears et al., 2015; Tran and

Zeckhauser, 2012; Barankay, 2012). In this paper, I present a �eld experiment that tests the e�ect

of peer rank on behavioral response and explores how the framing of information can in�uence this

response.

Existing work has explored the e�ects of social information in a variety of contexts, including

energy conservation, voting, and savings (Allcott, 2011; Gerber et al., 2008; Kast et al., 2014;

Beshears et al., 2015). Most interventions have provided individuals with information on the average

performance of a broader social group, with mixed results. Allcott (2011), for example, �nds

that showing individuals how their energy use compares to the mean of both their most e�cient

neighbors and all of their neighbors reduces electricity consumption by roughly 2% on average.

However, other research suggests that sharing peer information can lead to socially undesirable

behavior. Beshears et al. (2015) �nd that the provision of peer information about savings for

retirement can reduce savings rates by demotivating low-performers. Bursztyn and Jensen (2015)

document similar performance declines from �leaderboards� that publicly displayed the performance

of top students in computer-based remedial high school courses. John and Norton (2013) observe

a related phenomenon in the context of workplace exercise �walkstations,� �nding that people tend

to converge to the bottom performer, exercising less at walkstations when given information about

the low rates of use by others.

One limitation of existing work is that it does not isolate the elements of social information that are

central to both positive and negative behavioral responses. There is also limited evidence on how

ranked comparisons to speci�c peers in�uence behavior, or on heterogeneities in the motivational

e�ects of rankings (though some work on both topics does exist � see Barankay (2012), Beshears

2

et al. (2015), and Eisenkopf and Friehe (2014), for example). I provide evidence on some of the

outstanding questions in this area. Do ranked comparisons to people who are �like us� motivate us

di�erently than aggregate social comparisons? And how does our response to peer rank information

relate to our competitive drive?

In this paper, I outline a �eld experiment with 3,896 households in Castro Valley, California, which

tests how peer rank in�uences behavior. The experiment was conducted with a partner �rm, Wa-

terSmart Software, which works with local utilities to reduce water use at the household level

through mailers and other outreach campaigns. The experiment involved sending mailers with dif-

ferent forms of peer information and peer rank messaging to households to motivate reductions in

water use. Through the experimental design, I am able to address existing theories about how peer

rank and the competitive framing of rank messaging can in�uence behavior. The goal of this study

is to improve our understanding of social and peer information and their potentially heterogeneous

e�ects on behavior.

The results suggest that while social information can reduce water use, peer rankings and com-

petitive framing can also have detrimental impacts on behavior. Speci�cally, I �nd evidence of

heterogeneity in treatment e�ects from peer rank information. In particular, households that were

low water users prior to the experiment showed a �boomerang e�ect� (i.e., an increase in water

use) from peer rank information, except when a competitive frame was included. This result is

consistent with Garcia et al. (2006), who posit that a competitive drive triggered by a high rank

might make people less likely to �boomerang.� However, the competitive frame had detrimental

e�ects on the behavior of households that were high water users prior to the experiment, increasing

their water use on average. Further analysis of rankings suggests the possible existence of a �last

place e�ect,� whereby a competitively-framed peer rank led to an increase in water use by the worst

performer in the peer group � a movement away from the social norm. I argue that this stems from

the potentially demotivating power of peer information, in line with the results on peer information

and savings in Beshears et al. (2015).

3

2 Background

Household water use behavior is both important to change and di�cult to in�uence. The salience

of ine�cient water consumption and the price of water are both low � most families are not aware

of existing leaks or other ine�ciencies, and even when they are, the low price of water limits their

responsiveness to such problems. Indeed, the average family in the United States spends only 0.5%

of household income on water and sewage bills (United States Environmental Protection Agency and

Water, 2009). Consequently, the price elasticity of demand for water is low, with recent estimates

from California �nding elasticities in the -0.2 to -0.5 range (Lee and Tanverakul, 2015). This inelastic

demand, along with a variety of political economy and legal considerations, limits the in�uence of

price-based strategies for water reduction (like tiered pricing) in many places, including California

(Sillers, 2015).

In such scenarios, it might be cost-e�ective and welfare-improving to utilize non-price incentives that

target speci�c factors driving behavior and motivation. For example, households are arguably unsure

of what constitutes �good� and �bad� water consumption behavior. Social information interventions

that compare households to their neighbors o�er a solution, by providing a relevant reference point

for household consumption and social pressure to conform. Such an approach can change behavior

without raising the �nancial cost to households from water use; Allcott (2011) found that providing

households with social norms information decreased energy use by roughly the same amount as

an 11-20% increase in price. This paper reports on an experiment that tests the e�ect of ranked

comparisons with peers to motivate behavior change. A number of important social science theories

help explain how peer rank could a�ect behavior � a brief discussion of these theories and their

predictions is presented here.

2.1 Social Norms, the Boomerang E�ect, and Framing

Social norms theory predicts that social information, including peer rank, motivates behavior change

because it provides a social standard to follow. Most notably, the theory of social comparison

processes presented in Festinger (1954) suggests that social comparison occurs when objective,

non-social standards are unavailable. This could lead individuals to evaluate their opinions and

4

abilities by comparing themselves to others, and encourage them to take action to reduce any found

discrepancies. Furthermore, Festinger argues that individuals are most likely to compare themselves

to, and more likely to reduce discrepancies when compared to, people who are similar to them. Social

norms theory therefore implies that providing individuals with peer rank information would cause

their outcomes to compress towards the displayed social norm.

In recent years, there have been an increasing number of experimental tests of these theories. For

example, Schultz et al. (2007) conducted a �eld study with several hundred households in San

Marcos, California, using door hangers with aggregate-level social information on energy use to

motivate energy reduction. They �nd that social information caused high energy use households to

decrease their energy use, but encouraged low energy use households to increase energy use. On the

one hand, this implies a desirable response to social information from low-performing individuals.

However, the results also show a detrimental response from high-performers, referred to as the

�boomerang e�ect.�

However, while the body of literature on social norms and conservation using �eld experiments is

growing (Allcott, 2011; Ayres et al., 2013; Brent et al., 2015), much of the work on the boomerang

e�ect is theoretical or from experiments in which subjects opt into participating, which threatens

internal and external validity through selection into the study and observer e�ects from subjects

knowing that they are being studied (Clee and Wicklund, 1980; Schultz et al., 2007; Fischer, 2008).

In this experiment, I provide evidence from a natural �eld experiment � where subjects are unaware

that they are being studied � on possible boomerang e�ects from rank information, and how com-

petitive framing might in�uence it. While existing work has not explored competitive framing in the

context of peer rank or the boomerang e�ect, there is work that provides testable predictions. For

example, Garcia et al. (2006) argue that ranking may itself drive competitiveness; the authors �nd

that individuals are most competitive when they or their competitors are highly ranked. Therefore,

amongst top performers, rankings and competitive framing may mutually reinforce in a way that

motivates positive behavior change and o�sets possible boomerang e�ects. Further, Schultz et al.

(2007) provides evidence that framing manipulations can in�uence the boomerang e�ect; speci�-

cally, the authors show that low energy users receiving an injunctive, visual message � a �smiley

5

face� that conveyed social approval � maintained their low energy use rates, while those who did

not receive an injunctive message displayed a boomerang e�ect.

Meanwhile, there is evidence that low-performing individuals might change their behavior to avoid

ranking poorly in a peer comparison. In Gill et al. (2016), the authors present a real e�ort experiment

and observe �last place loathing,� whereby subjects who were ranked last within their group for a

laboratory task increased e�ort by 12%. However, research on goal-setting and attainment suggests

that the use of competitive framing and rank can also be demotivating. For example, Harding

and Hsiaw (2014) suggest that individuals may do worse if they feel that their target goals are

unachievable, but they also �nd that goal setting can be e�ective if goals are perceived as attainable

(a result also found in Corgnet et al., 2015). Overall, there is little consensus on the relative value

of rank or competitive framing as a tool for behavior change. While this experiment will not resolve

these debates, it does contribute evidence from a natural �eld experiment, which has been scarce

to date.

2.2 Motivation E�ects

Academic literature on motivation and self-e�cacy suggests another possibility: that individual

outcomes will spread further from the mean, as those who rank well among their peers will work

harder to improve and those who rank poorly will �give up.� There is a rich body of research

underpinning this prediction in the social sciences (for a summary of literature in this area, see

Pajares (1997)). A core �nding is that when people feel they are good at an activity, they engage in

it more, whereas people avoid activities they think they are bad at (Bandura, 1977). For example,

Shelton and Mahoney (1978) �nd that verbally reciting instruction messages that convey positive

beliefs improves ensuing outcomes. This result suggests that individuals with positive beliefs about

their ability may set higher goals for themselves and try harder to achieve them. Meanwhile, low-

e�cacy individuals (e.g. those who receive low rankings) may quit once they learn of their poor

rank (Hagger et al., 2002). For example, Beshears et al. (2015) found that low-savings individuals

were discouraged by information about peers' savings rates, which the authors attributed to the

discouraging e�ects of upward social comparison.

6

3 Experiment Overview

3.1 Experimental Design

3.1.1 Partners

My research partner for this �eld experiment was WaterSmart Software, a �rm based in California

that works directly with public water utilities to promote more e�cient water use by California

homeowners. WaterSmart sends a personalized mailer, called a Home Water Report (HWR), to

households every two months. See Appendix B for a sample HWR. The HWRs are transmitted either

electronically or through traditional mail and incorporate messages designed to engage customers

and reduce water use. Approximately 10% of customers in this study received Home Water Report

by email, with the rest receiving paper mailers. Through the utilities, the �rm tracks water use and

customer engagement over time. The public utility partner and data source in the study was a local

water provider that serves a subset of homes in the greater San Francisco Bay Area.

3.1.2 Subjects

I conducted the �eld experiment in Castro Valley, a town of 60,000 residents in Alameda County,

California, roughly 15 miles southeast of Oakland. Subjects in the study were residents of 3,896

single-family households in the C2A pressure zone in Castro Valley, who receive water through the

public utility. A �pressure zone� is a geographical area de�ned by the public utility based on the

area's elevation above sea level. The �gures in Appendix A show the speci�c location of both Castro

Valley and the C2A pressure zone within Castro Valley. Prior to the start of this experiment, the

�rm was already working with roughly 4,000 households in the other pressure zones in Castro Valley.

This study speci�cally targeted households in the C2A pressure zone because they were receiving

the HWRs for the �rst time.

3.1.3 Study Design

The households in the experiment area were �rst subdivided into 20 �cohorts� based on two categor-

ical variables: 1) outdoor irrigable area; and 2) the number of occupants in the household. There

7

were four possible irrigable area sizes for a household (small, medium, large, and extra-large), with

irrigable area computed by the �rm using real estate data on lot size and home footprint from

DataQuick. There were �ve possible household occupant categories (1, 2, 3, 4, and 5+). Appendix

Chart C.1 outlines the number of households in each of the 20 cohorts.

Every household was then individually assigned a random subset of four households in their cohort,

referred to as their �water group.� Using the cohorts ensured that households were only grouped

with others with roughly the same water needs. Importantly, a water group was assigned for

all households in the experiment, including those receiving the control mailer.1 This design feature

allowed me to use the control group directly to analyze the e�ects of ranking, group performance, and

other characteristics from the peer comparison. Speci�cally, I can consider what control households

�would have� received as a peer ranking, had they been assigned to receive one.

Finally, households were randomly assigned to one of four experimental conditions � a control

mailer group (�Control�), the two treatment mailer groups reported in this paper (the �Rank� and

�Competitive Rank� treatment groups), and an additional treatment mailer group not reported in

this paper.2 These experimental conditions di�ered in what was shown to subjects in the �treatment

area� of the mailer, which is visible in Appendix B. Each of these groups received up to four HWRs

over the course of the experiment; some homes did not receive four mailers as planned because of

logistical issues or asynchronous timing of water delivery and water readings. An HWR was delivered

to each household in the experiment every two months, by postal mail or email. Households in each

experimental group got the same version of the mailer each time (in other words, a household

assigned to the �Rank� treatment group received up to four �Rank� treatment mailers).

A few things are worth noting about this setup. First, each household was linked to a unique water

meter. Second, all subjects received HWRs, which contained social information about water use

above and beyond what was randomly assigned in the treatment area of the mailer. This informa-

tion, which included a �WaterScore� driven by overall data on mean water use in the town, would

1Note that the �water groups� for those receiving the control mailer were created after the experiment began butprior to analysis, in consultation with the partner �rm to ensure matching procedures.

2A third treatment group was part of the original experiment, which displayed a �Team Challenge� using thewater groups. I have excluded it from this paper, since the messaging in that treatment did not include explicit rankinformation and thus does not speak to the same theories discussed in this paper. From this point forward, I willexclude mention of this fourth �Team Challenge� treatment, though it was used for group formation. The existenceof this group does not a�ect the results in this paper, due to random assignment to mailer conditions.

8

Figure 1: Control and Treatment Mailer Versions

likely have had an e�ect on water use independent of the experiment treatments. This information,

and in particular the speci�c WaterScore, is controlled for in the analysis where necessary. Third,

each individual household's water group was unique � just because household A was assigned a �wa-

ter group� consisting of households B, C, D, and E, this did not mean that household A appeared

in B, C, D, or E's water group. Fourth, each household's water group consisted of homes in the

same cohort but not necessarily in the same treatment group.

3.1.4 Treatments and Controls for Households

Appendix B shows a sample HWR mailer, with the treatment area labeled. Figure 1 displays the

key visual di�erence in the mailer across conditions. Households assigned to the Control group

received the standard HWR, with a �Got water questions?� insert placed in the treatment area.

Note that no information about the water group was transmitted to Control households, nor were

the Control households made aware that any comparison water group had been created. Households

in the Rank treatment group received an HWR with a neutrally-framed rank comparison placed in

the treatment area. Households in the Competitive Rank treatment group received an HWR with a

competitively-framed rank comparison placed in the treatment area. This treatment provided the

same social comparison as in the Rank treatment, but with �Go for the Win!� messaging and a

ribbon icon, intended to encourage behavior change through competitive framing.

9

Table 1: Demographics by Treatment Group

Control Rank Competitive Rank

Home Size (sqft.) 1650.5 1627.5 1622.2(561.7) (533.4) (538.3)

Lot Size (sqft.) 7503.7 7320.8 7376.4(4390.9) (3615.7) (5175.4)

Year Home Built 1958.0 1957.5 1957.4(13.81) (13.46) (13.05)

Number of Bedrooms 3.129 3.110 3.109(0.725) (0.735) (0.757)

Number of Bathrooms 2.071 2.055 2.022(0.827) (0.859) (0.833)

Mean Water Use (Pre-Exp 2012) 229.7 231.0 236.1(129.6) (136.9) (136.9)

N 1308 1288 1300

Notes: Means, with standard deviations in parentheses.

3.1.5 Timeline

The experiment began in November 2012. The �rm sent out the �rst mailers at the end of Novem-

ber, using October 2012 meter reads. The �rm then sent three additional mailers, with the same

treatment/control messaging, in January 2013 (based on December 2012 meter reads), March 2013

(based on February 2013 meter reads), and May 2013 (based on April 2013 meter reads). House-

holds in the experiment that had meter reads outside of the four key meter read months did not

receive an experimental mailer in the month that followed their read.

3.2 Data and Baseline Characteristics

Data was obtained from the public utility, via WaterSmart Software. Two types of data were

collected. First, I collected water use data for the households in the experiment, for the periods

before and during the experiment. Second, WaterSmart provided data on the characteristics of the

households in the study, which they obtained both from the public utility and from independent

data sources including DataQuick.

10

3.2.1 Descriptive Data and Baseline Characteristics

I observed data from all 3,896 experimental households in Castro Valley, of which 3,209 received all

four experimental mailers. Appendix Chart C.2 outlines the number of households in each treatment

and control group, and the number of households in each group that received all four mailers.

3.2.2 Pre-Treatment Water Use Trends

Meter read technicians from the water utility measured water use every two months. Most meters

used CCF units for water use (1 CCF = 100 cubic feet of water = 748 gallons), and the CCF

reads were converted into a �gallons per day� (GPD) measure by the public utility. Mean water

use prior to the experiment, measured in GPD, is visible in Table 1, while Appendix D displays

time trends in water use pre- and post-experiment. Note that Appendix Figure D.1 shows the key

role that seasonality plays in water use; water use is higher in the summer than in the winter.

Furthermore, Appendix Figure D.1 provides some visual evidence of possible di�erential seasonal

trends by treatment, with households in the Competitive Rank treatment showing a slightly higher

summer peak than those in other conditions. Because of the possibility of di�erential seasonal

trends, I use month �xed e�ects in certain speci�cations to control for such trends.

While mailers were sent on the same date for all households in each mailing cycle, households did

not have meter reads on the same date. As a result, there is some variance in how many days a

given household was treated by a single mailer. This is not an uncommon issue, having appeared

in similar experiments using read-based mailers, including Allcott (2011). Successful randomization

prevents this from being problematic to some extent, as there is no correlation between treatment

and meter read cycle.

3.2.3 Randomization Check

Randomization checks are warranted here for two main reasons. First, the randomization process

itself was conducted by the �rm and not the researcher. Though the �rm has a track record of

experimentation, a check is needed to ensure that there were no systematic errors in randomization.

Second, 256 households were dropped after randomization but prior to study implementation. These

11

Table2:

Random

izationChecks

(1)

(2)

(3)

(4)

(5)

(6)

Hom

eSize

Lot

Size

YearBuilt

Bedroom

sBathroom

sPre-Experim

entMeanWater

Use

Rank

-25.26

-203.9

-0.565

-0.0185

-0.0147

0.859

(23.04)

(169.4)

(0.575)

(0.0309)

(0.0354)

(5.264)

Com

petitiveRank

-29.98

-120.7

-0.651

-0.0172

-0.0475

5.912

(23.09)

(202.6)

(0.564)

(0.0312)

(0.0346)

(5.253)

Observations

3383

3383

3383

3353

3407

3832

R2

0.001

0.000

0.000

0.000

0.001

0.000

Notes:Standard

errors

inparentheses.Models1-6

presenttheregressionsofvarioushousehold

characteristics

ondummiesforthetwo

treatm

entgroups(R

ankandCompetitiveRank),asarandomizationcheck.Theomittedgroupishouseholdsreceivingthecontrolmailer.

*p<0.10,**p<0.05,***p<0.01

12

households did not receive a mailer despite being assigned to one of the treatment or control groups,

for logistical reasons (the subject moved from the property, the address was not veri�ed, etc.).

To test the balance of the groups on observed demographic characteristics, I run a regression of the

various demographic characteristics (yi below) on dummy variables for the two treatment groups,

omitting the Control group. I also compute f-test statistics to determine joint signi�cance. The

econometric model is as follows:

yi = β0 + β1(TRank)i + β2(TCompRank)i+ε

Table 2 presents the results from these regressions. None of the f-statistics and associated p-values

demonstrate joint signi�cance of the coe�cients, suggesting that randomization resulted in balanced

treatment and control groups.

3.2.4 Handling Outliers

The primary outcome measure, gallons per day, had occasional extreme values. First, 101 households

registered a GPD of zero at least once after October 2011. Such readings usually occur because

household members are either not at home during the read period, or because their water use is so

low that it fails to register. Second, there were two meter reads in the data that were far above

normal values (exceeding 10,000 GPD). The utility identi�ed these extreme high reads as meter

malfunctions or abnormalities. I excluded all households in these two categories from the analysis.

4 Empirical Methods

I use a variety of empirical techniques to analyze the experiment and its e�ects, which I outline

here.

4.1 Regression Framework to Compare Mailers

The central question in this paper is whether and how displayed ranks and rank framing a�ect

behavior. To determine how the two forms of rank messaging in�uenced water conservation behavior,

13

I use regressions that compare mean water use across mailers after the initiation of the treatment

in two ways. First, I compare mean water use in the �rst period (from the meter read following

receipt of the �rst mailer) across treatment groups. Second, to provide an estimate of the long-run

di�erences in water use across treatment mailers, I compare mean water use for all post-treatment

periods across conditions.

The general speci�cation I use is a regression, shown below, of water use by the household (measured

in gallons per day) in the relevant periods on dummy variables for the two rank treatments, along

with controls for home characteristics (lot size, home square footage, and the number of bathrooms,

bundled below as %i). In addition, while they are not in the speci�cation below, �xed e�ects for

read month and WaterScore are used when the analysis focuses only on the �rst mailer.

GPDi = β0+β1(TRank)i + β2(TCompRank)i + %i + ε

I then disaggregate the analyses to determine whether the average treatment e�ect di�ers across

conditions based on past water use. Speci�cally, I classify households as being �low� or �high� water

users using data on water use in the pre-experiment reads in 2012. Low-use households are de�ned as

those in the bottom third of water use within each irrigable area category, and high-use households

are de�ned as those in the top third within each irrigable area category. By assessing water use

within the irrigable area classi�cations, I am able to control for the di�erences in water needs based

on property size. Note that irrigable area is used for this classi�cation instead of cohort, since some

cohorts had very few households in them (as visible in Chart C.1).

This diversity of approaches helps to address theories about the di�erential e�ect of framing on

household response to peer information. The critical speci�cations test whether certain mailer ver-

sions were more or less e�ective for households with �high� or �low� water use, pre-experiment. This

disaggregation allows hypothesis testing around potential �boomerang� or demotivational e�ects

from social comparison and rank messaging.

4.2 Approaches to Assess Ranking E�ects

In the two rank treatments, each possible rank position can be viewed as a distinct treatment. In

other words, a ��rst place� Competitive Rank mailer may induce a di�erent response than a �last

14

place� Competitive Rank mailer. This feature of the experimental setup allows for testing of theories

about peer rank and its in�uence on behavioral response. I do this using two empirical approaches.

4.2.1 Restrict Focus to Last/First Place Mailers Only

First, I treat each mailer and the household's water use in the ensuing period as a distinct treat-

ment/outcome pair and assess the e�ect of being in ��rst� or �last� place on subsequent water use

in each of the four mailer rounds (and across all rounds, using all data). This approach requires a

model of behavioral response whereby a household's behavior in the period following a mailer is a

direct response to the content of that mailer and is independent of the content of previous mailers.

From the perspective of maintaining randomization, this is not an issue with the �rst mailer and

subsequent behavior. However, when I use later mailers in the analysis, this threatens identi�cation

by moving away from pure randomization. This is because the content of previous mailers may have

in�uenced household response to subsequent mailers.

There are precedents for this approach to assessing the impact of multiple treatments in existing

research. For example, Doherty and Adler (2014) argue that mailer e�ects in a political campaign

context are short-lived. The authors suggest that individual level responses can be considered in

the period immediately following a given mailer, as timing may be more important to outcomes

than mailer quantity. Additionally, Allcott and Rogers (2014) �nd evidence of cycles of signi�cant

backsliding in the weeks immediately following social information mailer receipt, using data from

Opower's Home Energy Reports. A similar sort of backsliding here could lead to decay in a mailer's

e�ect by the end of a single post-mailer period.

I take a few steps to address these concerns. First, I report the results from this analysis for each

round of mailers separately. Additionally, in the analysis that includes data from all mailer rounds,

I attempt to control for potential biases from repeat mailer exposure by using �xed e�ects for the

number of mailers seen prior to receiving the mailer in question. These controls do not qualitatively

change the results.

The general empirical approach for this analysis is as follows. First, I look at all mailer/outcome

pairs for households that �nished in �rst/last place in their water group in each round of mailers,

and regress subsequent household water use on the treatments (omitting the Control group). The

15

speci�cation for this analysis is below (note that this analysis is done separately for each of the four

mailer rounds):

GPDij = β0 + β1(TRank)i + β2(TCompRank)i + β3(MailerGPDij) + %i + δj + γij + εij

The speci�cation includes controls for household water use displayed (in gallons per day) in the

mailer (MailerGPDij), household demographics as in the previous regression (%i), month �xed

e�ects (δj), and WaterScore �xed e�ects (γijk). Because Control households were assigned to water

groups, the Control observations in this regression are homes who �would have� been in �rst/last

place had they seen a ranking. Therefore, the β1 and β2 coe�cients represent estimates of the e�ect

of Rank and Competitive Rank messaging for �rst/last place homes.

I then perform the analysis above using all �rst/last place mailers, across mailer rounds. In this

case, since there are multiple observations for some households and water use within household is

likely to be correlated over time, I cluster standard errors at the household level. I also add �xed

e�ects for the number of prior mailers seen. The general speci�cation for this analysis is below:

GPDijk = β0 + β1(TRank)i + β2(TCompRank)i + β3(MailerGPDijk) + %i + δj + γijk + ρk + εijk

Note that the speci�cation includes the same controls and �xed e�ects as the previous regression,

but adds �xed e�ects for the number of mailers seen prior to the observation mailer (ρk). Again, the

β1 and β2 coe�cients represent estimates of the e�ect of Rank and Competitive Rank messaging

for �rst/last place homes, using all �rst/last place mailer and outcome data.

4.2.2 Rank E�ects amongst the Middle Third of Water Users

I also use a second approach to explore ranking e�ects, for robustness. Speci�cally, I restrict

attention to homes in the middle third of water users pre-experiment, whose water use was around

the median given their irrigable area. Due to the random assignment of water groups, there is

variation in rank position amongst these homes that is not a function of their actual water use

behavior. I exploit this variation and run the following regression speci�cation, which uses only

data from mailers received by individuals in the middle third of water users and includes interaction

16

Figure 2: Average Treatment E�ects - First-Period Only and Overall

e�ects between rank position (1st, 2nd, 3rd, 4th, or 5th) and treatment mailer version (Control,

Rank, or Competitive Rank):

GPDijk = β0+[∑3

m=1(∑5

n=1 βm,n(Positionn)ijk∗(Tm)i)]+β15(MailerGPDijk)+%i+δj+γijk+ρk+ε

The interaction terms reveal whether there is a di�erential impact of rank position based on mailer

version, which provides evidence regarding the existence of a �last place e�ect� or ��rst place e�ect�

in the Rank and Competitive Rank treatments. The speci�cation includes the same set of controls

and �xed e�ects as the previous regression. Because there are multiple observations per household,

I again cluster standard errors at the household level.

5 Results

5.1 E�ects of Rank Messaging Relative to Control

5.1.1 Average Treatment E�ects of Rank Messaging

Table 3 presents results for the �rst period following experiment initiation, and Table 4 presents

results for all post-initiation periods aggregated together. Figure 2 shows the key coe�cients from

both tables visually, with the �rst two bars in each panel representing the key estimates of average

treatment e�ects across all subjects. Note that a positive average treatment e�ect represents an

increase in water use, relative to the Control group. Additionally, Appendix Figure D.1 shows water

use trends across conditions, both pre- and post-experiment.

17

The results show that the rank messaging treatments had minimal impacts on water use in the

�rst post-mailer period, though there are di�erences over the full post-mailer period. In particular,

there is evidence that the Competitive Rank mailer performed worst overall, increasing water use

by 8.22 GPD relative to the Control mailer (Table 4, model (2)). This result is signi�cant at the

10% level. While this is a notable result, we should interpret it with caution for three reasons.

First, Appendix Figure D.1, Table 1, and Table 2 provide some evidence that pre-experiment water

use in Competitive Rank households was slightly higher than Control households, though this

di�erence is not statistically signi�cant. Second, a nonparametric Mann-Whitney U test comparing

the Competitive Rank and Control households on post-mailer mean water use, reported in Appendix

Table E.1, did not show statistical signi�cance (p=0.481), casting doubt on the robustness of the

results in Table 4. Third, and most importantly, this analysis treats receipt of any version of a given

mailer as part of the same treatment, whether you performed well or poorly in the displayed peer

rank. In other words, a household receiving a Competitive Rank mailer and �nding themselves in

��rst place� in the ranking is, in this analysis, grouped with a household receiving a Competitive

Rank mailer and �nding themselves in �last place.� More analysis is needed to understand how

rankings interact with these aggregate e�ects, and follows in section 5.2.

Overall, these results suggest that the display of competitively-framed peer rank information repre-

sented the least e�ective way of motivating water conservation. These results have two important

implications. First, they suggest that displaying peer rank information may be worse on aggregate

than not displaying it, which aligns with some recent research on peer information in economics

(Beshears et al., 2015; Bursztyn and Jensen, 2015). Second, these results demonstrate the impor-

tance of looking into both immediate and long-run responses to repeat mailer campaigns. The

analysis focusing only on the �rst post-mailer period, in Table 3, does not capture the detrimental

impact of the Competitive Rank mailer, which is only visible in the longer-run analysis in Table 4.

5.1.2 Disaggregation by Past Water Use

I next repeat the analysis, but disaggregate based on a key, visible covariate � past water use.

Tables 5-8 show the full output from these regressions, while Figure 2 provides visuals of the key

coe�cients, which represent the disaggregated treatment e�ects of the rank mailers. Additionally,

18

Table 3: Average Treatment E�ects from Rank: First Post-Mailer Period

All 3 Treatment Mailers Comp. Rank and Rank Only

(1) (2) (3) (4) (5)GPD GPD GPD GPD GPD

Competitive Rank 1.751 2.202 -1.921 -3.085 -6.786(4.719) (4.831) (4.111) (5.049) (4.492)

Rank 0.774 5.291 5.020(4.767) (5.029) (4.437)

Lot Size 0.00387∗∗∗ 0.00446∗∗∗ 0.00422∗∗∗ 0.00457∗∗∗

(0.000891) (0.000694) (0.00103) (0.000845)

Num Bathrooms 4.757 1.008 5.165 1.006(3.649) (3.144) (4.589) (4.075)

Home Size (SqFt) 0.0307∗∗∗ 0.0259∗∗∗ 0.0337∗∗∗ 0.0293∗∗∗

(0.00656) (0.00557) (0.00844) (0.00749)

Observations 3796 3349 3326 2217 2204R2 0.000 0.058 0.270 0.067 0.256Read Month Fixed E�ects No No Yes No YesWaterScore Fixed E�ects No No Yes No Yes

Notes: Standard errors in parentheses. Models 1-3 compare water use in the �rst period following mailer initiationacross the three conditions. Household characteristics are used as controls in models (2) and (3), and �xed e�ectsare included for WaterScore and meter read month in model (3). Models 4-5 present similar regressions, but donot include the Control mailer, allowing for direct comparison of the Rank and Competitive Rank treatment.* p<0.10, ** p<0.05, *** p<0.01

Appendix Figure D.2 shows water use trends across conditions for both low and high water users,

both pre- and post-experiment.

The results for low water users suggest that the Rank treatment was marginally less e�ective for

these individuals, relative to both the Competitive Rank and Control mailers. In the �rst post-

mailer period, the Rank treatment increased water use by 11.14 GPD relative to the Control mailer,

statistically signi�cant at the 5% level (Table 5, model (3)). A Mann-Whitney U test provides a

similar directional result, though p=0.167 in that test as reported in Appendix Table E.1. It is

instructive to compare the Rank and Competitive Rank mailers directly as well, to assess the

e�ect of the framing of peer rank information. When compared directly with the Rank mailer,

the Competitive Rank mailer is associated with 13.73 GPD lower household water use in the �rst

post-mailer period (Table 5, model (5)), which is statistically signi�cant at the 1% level. A Mann-

Whitney U test for the same comparison also shows signi�cance, at the 5% level, as reported in

19

Table 4: Average Treatment E�ects from Rank: All Periods

All 3 Treatment Mailers Comp. Rank and Rank Only

(1) (2) (3) (4)Mean GPD Mean GPD Mean GPD Mean GPD

Competitive Rank 6.194 8.220∗ 6.934 5.920(4.615) (4.658) (4.601) (4.584)

Rank -0.740 2.316(4.356) (4.415)

Lot Size 0.00504∗∗∗ 0.00554∗∗∗

(0.000647) (0.000616)

Num Bathrooms 6.346∗ 7.695∗

(3.427) (4.215)

Home Size (SqFt) 0.0335∗∗∗ 0.0357∗∗∗

(0.00598) (0.00730)

Observations 3796 3349 2526 2217R2 0.001 0.097 0.001 0.114

Notes: Standard errors in parentheses. Models 1-2 compare mean water use in all periodsfollowing mailer initiation across the three conditions. Household characteristics are used ascontrols in model (2). Models 3-4 present similar regressions, but do not include the Controlmailer, allowing for direct comparison of the Rank and Competitive Rank treatments.* p<0.10, ** p<0.05, *** p<0.01

Appendix Table E.1. When looking at the mean water use during all periods following the initiation

of the experiment, however, the detrimental e�ect of the Rank mailer relative to the Control mailer is

smaller at 3.71 GPD, and not statistically signi�cant (Table 6, model (2)). However, the di�erence

between the Rank and Competitive Rank mailers remains signi�cant at the 10% level, with the

Competitive Rank mailer associated with 7.13 GPD lower household water use than the Rank

mailer over the entire experimental period (Table 6, model (4)). A Mann-Whitney U test again

provides a similar directional result here, though p=0.137 in that test (see Appendix Table E.1).

This is a notable result � this suggests the presence of a small �boomerang e�ect� for low water

users from rank information, but one that was counteracted by a competitive frame. One possible

explanation for this �nding is that the Rank treatment's neutral frame does not provide su�cient

motivation for e�cient households to continue conservation e�orts. The Competitive Rank treat-

ment mailer provided similar peer rank information, but did so with competitive framing, which

may have o�set the small boomerang e�ect observed for households receiving the Rank mailer.

20

Table 5: Low Water Users in the First Post-Mailer Period

All 3 Treatment Mailers Comp. Rank and Rank Only

(1) (2) (3) (4) (5)GPD GPD GPD GPD GPD

Competitive Rank -1.873 -2.712 -2.588 -14.03∗∗∗ -13.73∗∗∗

(3.459) (3.548) (3.573) (5.203) (5.224)

Rank 8.570∗ 11.36∗∗ 11.14∗∗

(4.647) (5.135) (5.184)

Lot Size 0.000974 0.00129 0.000333 0.000611(0.000793) (0.000801) (0.000579) (0.000574)

Num Bathrooms 3.055 3.105 1.992 2.151(2.219) (2.174) (2.529) (2.495)

Home Size (SqFt) 0.00173 0.00160 0.00608 0.00591(0.00421) (0.00414) (0.00504) (0.00503)

Observations 1246 1099 1087 712 704R2 0.005 0.015 0.036 0.014 0.043Read Month Fixed E�ects No No Yes No YesWaterScore Fixed E�ects No No Yes No Yes

Notes: Standard errors in parentheses. Models 1-3 compare water use across the three conditions in the �rstperiod following mailer initiation amongst households who were in the lowest third of water users (amongst similarhomes) in the 2012 months preceding the experiment. Household characteristics are used as controls in models(2) and (3), and �xed e�ects are included for WaterScore and meter read month in model (3). Models 4-5present similar regressions, but do not include the Control mailer, allowing for direct comparison of the Rank andCompetitive Rank treatments.* p<0.10, ** p<0.05, *** p<0.01

Meanwhile for households with high levels of water use pre-treatment, the e�ects are much di�erent.

Figure 2 (and Table 7) demonstrates that the mailer versions were similarly e�ective in the period

following the �rst mailer. However, analyses of mean water use in all periods indicate that the

Competitive Rank mailer performed worse than the other mailers, increasing mean water use by

12.49 GPD relative to the Control mailer (Table 8, model (2)) and by 15.62 GPD relative to the

Rank mailer (Table 8, model (4)). While the �rst of these two results is sizable, it is not statistically

signi�cant; however, the second result is statistically signi�cant at the 10% level (though a Mann-

Whitney U test for this result returns p=0.225, suggesting caution in interpreting this result).

This �nding suggests that while competitively-framed ranks had a positive e�ect on low water

use households (preventing a small boomerang e�ect), they increased water use in high water use

households. This could be because it is demotivating to perform poorly in a competitive comparison

21

Table 6: Low Water Users in All Post-Mailer Periods

All 3 Treatment Mailers Comp. Rank and Rank Only

(1) (2) (3) (4)Mean GPD Mean GPD Mean GPD Mean GPD

Competitive Rank -2.982 -3.414 -5.152 -7.131∗

(3.422) (3.625) (3.538) (3.917)

Rank 2.171 3.712(3.621) (3.928)

Lot Size 0.00100∗∗ 0.000877(0.000436) (0.000566)

Num Bathrooms 2.436 1.155(2.248) (2.375)

Home Size (SqFt) 0.00152 0.00536(0.00379) (0.00468)

Observations 1246 1099 823 712R2 0.002 0.011 0.003 0.013

Notes: Standard errors in parentheses. Models 1-2 compare mean water use across the threeconditions in all periods following mailer initiation amongst households who were in the lowestthird of water users (amongst similar homes) in the 2012 months preceding the experiment.Household characteristics are used as controls in model (2). Models 3-4 present similar regres-sions, but do not include the Control mailer, allowing for direct comparison of the Rank andCompetitive Rank treatments.* p<0.10, ** p<0.05, *** p<0.01

with your peers. Note that the higher water use relative to the Control group is not observed in

the Rank treatment (Table 8, model (2)) � the competitive framing seems to be the key element

driving the adverse reaction, not the rank information itself.

5.2 Ranking E�ects

In assessing the e�ect of speci�c rankings, I focus on �rst and last place in particular. I begin

by restricting analysis to the following mailers and subsequent outcomes: 1) households receiving

�rst/last place rank messaging in the Rank and Competitive Rank treatments; and 2) households

receiving the Control mailer who �would have� ranked in �rst/last had they been shown a ranking.

I use regressions to estimate the e�ect of displayed ��rst� and �last� place messaging on behavior

following mailer receipt using data from each mailer round separately, and then across all experi-

mental mailers (with clustered standard errors at the household level). Table 9 provides the main

22

Table 7: High Water Users in the First Post-Mailer Period

All 3 Treatment Mailers Comp. Rank and Rank Only

(1) (2) (3) (4) (5)GPD GPD GPD GPD GPD

Competitive Rank -5.985 -5.538 -6.256 -3.234 -5.433(9.734) (10.13) (9.479) (10.90) (10.56)

Rank -5.695 -2.485 -1.052(10.24) (11.19) (11.00)

Lot Size 0.00512∗∗∗ 0.00499∗∗∗ 0.00499∗∗∗ 0.00498∗∗∗

(0.000899) (0.000886) (0.00101) (0.000992)

Num Bathrooms -11.18 -9.819 -17.19 -15.85(8.358) (8.145) (10.70) (10.04)

Home Size (SqFt) 0.0358∗∗∗ 0.0372∗∗∗ 0.0477∗∗∗ 0.0466∗∗∗

(0.0132) (0.0125) (0.0179) (0.0165)

Observations 1275 1098 1095 738 737R2 0.000 0.066 0.085 0.078 0.092Read Month Fixed E�ects No No Yes No YesWaterScore Fixed E�ects No No Yes No Yes

Notes: Standard errors in parentheses. Models 1-3 compare water use across the three conditions in the �rstperiod following mailer initiation amongst households who were in the highest third of water users (amongstsimilar homes) in the 2012 months preceding the experiment. Household characteristics are used as controls inmodels (2) and (3), and �xed e�ects are included for WaterScore and meter read month in model (3). Models 4-5present similar regressions, but do not include the Control mailer, allowing for direct comparison of the Rank andCompetitive Rank treatments.* p<0.10, ** p<0.05, *** p<0.01

�ndings for the ��rst place� e�ects and Table 10 provides the main �ndings for the �last place�

e�ects.

The main result in this analysis is the evidence of a detrimental �last place e�ect� for households in

the Competitive Rank treatment. Speci�cally, the analysis that includes all mailer rounds suggests

that households ranked last in the Competitive Rank treatment use 17.85 GPD more water, on av-

erage, than households in the Control group who would have been in last place had they seen their

position (Table 10, model (6)). This result is statistically signi�cant at the 1% level. This e�ect is

also signi�cant at the 1% level relative to similar individuals in the Rank treatment. Both of these

results are highly robust to non-parametric Mann-Whitney U tests, as reported in Appendix Table

E.1. These �ndings are driven largely by e�ects in later mailers (see Table 10, models (3) and (4)

in particular), suggesting the possibility that the adverse response to competitive framing is some-

23

Table 8: High Water Users in All Post-Mailer Periods

All 3 Treatment Mailers Comp. Rank and Rank Only

(1) (2) (3) (4)Mean GPD Mean GPD Mean GPD Mean GPD

Competitive Rank 8.421 12.49 10.40 15.62∗

(8.576) (8.511) (8.535) (8.398)

Rank -1.975 -3.191(8.304) (8.424)

Lot Size 0.00669∗∗∗ 0.00648∗∗∗

(0.000864) (0.000910)

Num Bathrooms -11.40∗ -14.55∗

(6.281) (7.984)

Home Size (SqFt) 0.0404∗∗∗ 0.0489∗∗∗

(0.00985) (0.0128)

Observations 1275 1098 865 738R2 0.001 0.153 0.002 0.177

Notes: Standard errors in parentheses. Models 1-2 compare mean water use across the threeconditions in all periods following mailer initiation amongst households who were in the highestthird of water users (amongst similar homes) in the 2012 months preceding the experiment.Household characteristics are used as controls in model (2). Models 3-4 present similar regres-sions, but do not include the Control mailer, allowing for direct comparison of the Rank andCompetitive Rank treatments.* p<0.10, ** p<0.05, *** p<0.01

thing that builds up over time. One interpretation here is that a competitively-framed �last place�

ranking felt worse for households after they had become accustomed to receiving bimonthly mailers

displaying rankings. However, because the results are driven by later mailers (the e�ects of which

are hard to entirely disentangle from earlier mailers), they should be interpreted as associations and

not causal proof of a last place e�ect.

The overall results suggest that competitive framing makes peer rank information demotivating for

people who perform worst in the displayed rank. This is especially interesting because the Com-

petitive Rank treatment did not seek to prime negative thoughts or social judgements about poor

performance by the household � it actually had messaging encouraging low-performing households

to improve.

24

Table9:

First

Place

E�ects

First

Wave

SecondWave

ThirdWave

FourthWave

Overall

(1)

(2)

(3)

(4)

(5)

(6)

GPD

GPD

GPD

GPD

GPD

GPD

Rank

4.632

-1.273

-2.955

15.73∗

-10.16

∗∗2.580

(5.870)

(3.484)

(7.260)

(9.215)

(4.670)

(3.393)

Com

petitiveRank

-9.727

∗∗6.463

-8.703

22.69∗

∗-12.69

∗∗∗

1.008

(4.335)

(5.394)

(6.381)

(11.14)

(4.597)

(3.486)

Observations

687

664

671

627

2948

2649

R2

0.340

0.397

0.306

0.272

0.004

0.344

Mailers

SeenFixed

E�ects

N/A

N/A

N/A

N/A

No

Yes

WaterScore

Fixed

E�ects

Yes

Yes

Yes

Yes

No

Yes

ReadMonth

Fixed

E�ects

Yes

Yes

Yes

Yes

No

Yes

Dem

ographicControls

Yes

Yes

Yes

Yes

No

Yes

Notes:Standard

errors

inparentheses

(clustered

byhousehold

inmodels(5)and(6)).Models1-4

presentregressionresults

comparingthee�ectoftheRankandCompetitiveRanktreatm

entsfor'�rstplace'householdsonsubsequentwateruse

foreach

ofthefourmailer

roundsseparately.Theomittedgroupishouseholdsin

theControlgroupwhowould

havebeenin

�rstplace

intheirgroupshadthey

seen

arankingin

theirmailer.Models5and6presentsimilarregressionsbutuse

data

from

allmailer

rounds,both

withandwithoutvariouscontrolsand�xed

e�ects.

*p<0.10,**p<0.05,***p<0.01

25

Table10:LastPlace

E�ects

First

Wave

SecondWave

ThirdWave

FourthWave

Overall

(1)

(2)

(3)

(4)

(5)

(6)

GPD

GPD

GPD

GPD

GPD

GPD

Rank

12.01

-3.332

5.738

-1.280

9.297

1.347

(14.72)

(10.16)

(8.942)

(10.91)

(10.48)

(6.087)

Com

petitiveRank

-3.482

7.591

23.48∗

∗33.42∗

∗∗29.88∗

∗∗17.85∗

∗∗

(11.92)

(9.515)

(10.01)

(12.22)

(10.92)

(5.957)

Observations

660

680

691

696

3161

2727

R2

0.337

0.438

0.528

0.607

0.005

0.497

Mailers

SeenFixed

E�ects

N/A

N/A

N/A

N/A

No

Yes

WaterScore

Fixed

E�ects

Yes

Yes

Yes

Yes

No

Yes

ReadMonth

Fixed

E�ects

Yes

Yes

Yes

Yes

No

Yes

Dem

ographicControls

Yes

Yes

Yes

Yes

No

Yes

Notes:Standard

errors

inparentheses

(clustered

byhousehold

inmodels(5)and(6)).Models1-4

presentregressionresults

comparingthee�ectoftheRankandCompetitiveRanktreatm

ents

for'last

place'householdsonsubsequentwateruse

foreach

ofthefourmailer

roundsseparately.Theomittedgroupishouseholdsin

theControlgroupwhowould

havebeenin

last

place

intheirgroupshadthey

seen

arankingin

theirmailer.Models5and6presentsimilarregressionsbutuse

data

from

allmailer

rounds,both

withandwithoutvariouscontrolsand�xed

e�ects.

*p<0.10,**p<0.05,***p<0.01"

26

Figure 3: Coe�cients from Interactions of Treatment and Rank Position

The evidence for a comparable ��rst place e�ect� is not as compelling. As models (5) and (6) in

Table 9 show, the visible and bene�cial ��rst place e�ect� from rank information in a speci�cation

without controls disappears with the inclusion of controls.

For robustness, I use the approach outlined in section 4.2.2, restricting analysis to only those homes

in the middle third of water users. I estimate the e�ect of rank here by interacting treatment

and rank to determine if there was a di�erential response to rank position by treatment. Figure 3

provides a visual depiction of the coe�cients on the interaction terms by treatment and position,

with the full regression results reported in Table 1 in the Online Appendix. Since it is necessary to

omit a coe�cient, Control households in 3rd position are omitted. While the individual coe�cients

are not statistically signi�cant, the trend in the point estimates is clear: the Rank and Competitive

Rank treatments seem to consistently drive up water use for households in last place, while all other

rank positions seem to encourage less water use.

When these results are coupled with the earlier results showing that the Competitive Rank mailer

performed worst on aggregate, a clearer story emerges. The competitive framing on rank messaging

27

discouraged high water users, particularly those individuals who found themselves in �last place�

in the displayed rank. These individuals performed worse because of the competitive framing on

rank information. Simultaneously, the competitive frame had a small positive impact on low water

users. However, the detrimental e�ects of the competitive frame on high water users outweighed

the positive impacts on low water users, meaning that on aggregate the Competitive Rank mailer

performed worst of all mailer versions used.

6 Discussion and Conclusions

This experiment provides insights on some important underlying drivers of behavioral response to

social information and peer rank. Overall, the experiment �nds that the di�erent frames used

in the peer rank mailers had di�erent e�ects on water use, with the competitively-framed rank

mailer performing worst. However, this aggregate comparison of mailers masks more interesting

results on the underlying mechanisms of rank and response. The most robust results come from the

disaggregated analysis of treatment e�ects and from the analysis of speci�c rank e�ects. The analysis

shows that the display of a neutrally-framed peer rank relative to four similar homes caused a small

�boomerang e�ect� in water-e�cient households, increasing the households' water use relative to

the control. The small boomerang e�ect was, however, eliminated by the inclusion of a competitive

frame. The results together are supportive of a conclusion that high achievers thrive (or, at least,

do not struggle) when competition is primed, and may need a competitive motivation to avoid

boomerang e�ects from explicit rank information.

However, the competitive framing of rank information had large demotivational e�ects on water-

ine�cient households. These households responded poorly to the competitive framing, more than

o�setting any bene�cial e�ects of the competitive framing for high achievers. Furthermore, it

appears that rank e�ects played a signi�cant role as well. The results show that households who

�nished in �last place� in a competitively-framed peer ranking were demotivated, increasing their

water use relative to both the control and the neutrally-framed rank groups. Interestingly, this

implies that the adverse reaction was primarily driven by the competitive frame (rather than the

low ranking).

28

These results have direct implications for the competing theories related to peer rank and behav-

ioral response. This experiment �nds that when rankings are provided with competitive framing,

the theories on motivation and self-e�cacy seem more consistent with observed behavior, with top

performers holding steady while poor performers worsen. This �nding is consistent with the op-

positional reactions to peer information found by Beshears et al. (2015) in the savings context.

Furthermore, the �nding that competitive framing o�sets the small boomerang e�ect for top per-

formers is consistent with Garcia et al. (2006). However, without the competitive frame, simple

rank information seems to encourage a behavioral response more in line with social norms theories,

with small �boomerang e�ects� for low water users and small reductions in water use for high water

users. This set of results provides some structure to existing theories on rank and response, and

suggests that the framing of peer comparisons and rank information is an important factor in their

success or failure.

The implication of these �ndings for �rms, public policymakers, and �nudgers� seeking to use peer

ranking is mixed. In particular, the experiment reveals some potential downsides to providing such

information, namely that it can demotivate poor performers. This conclusion leads to important

follow up questions. What types of social information are best to motivate those who are performing

poorly? Why does a competitive frame prevent backsliding for top performers and to what extent

is this context-dependent? Further research is needed to better understand the observed e�ects and

what forms of social messaging are needed to negate those e�ects.

Follow up research could extend this work in a number of ways. First, the �last place e�ect� outlined

here could be tested in a randomized setting with a larger sample size to validate the �ndings here

and better explore its mechanisms. Indeed, there is growing academic interest in rank information

and how it might in�uence those at the bottom of the rank distribution (Barankay, 2012; Bursztyn

and Jensen, 2015; Gill et al., 2016). Understanding and better modeling the behavior of the low-

ranked is therefore a promising area for further work. Second, studies could explore alternative

methods for in�uencing conservation behavior in particular. Possibilities include increasing the

salience of costs, altering the framing of messaging around utility bills to increase its signi�cance

to households, or timing messaging to coincide with actual use. Third, future research needs to

explore how to promote major household behavior change in general. Serious water conservation

29

e�orts are part of a broader class of household behavioral phenomena that involve small upfront

transaction costs, but signi�cant long-run bene�ts (both through cost savings at the individual level

and social bene�ts through reduced water production costs). Present-biased individuals may balk

at such arrangements, even when they would make society better o� in the long run. While social

information interventions can help address these sorts of challenges, there is a need for more research

to inform and re�ne the techniques of such campaigns.

I would like to thank Richard Zeckhauser, Brigitte Madrian, and Michael Norton for their guidance

and advice. Additionally, I want to thank Alberto Abadie, Hunt Allcott, Dan Ariely, Gary Charness,

John List, Tim McCarthy, Duncan Simester, Monica Singhal, two anonymous reviewers, and sem-

inar participants at the UK Behavioral Insights Team, UCSD, and Harvard for their feedback. A

special thanks is also due to Ora Chaiken, Chad Haynes, and Peter Yolles of WaterSmart Software,

without whom this work would not be possible. Finally, I want to acknowledge Vivien Caetano, Peter

Hadar, Shahrukh Khan, Stephanie Kestelman, and Kate Musen for their excellent work as research

assistants.

30

References

Allcott, H. (2011). Social norms and energy conservation. Journal of Public Economics, 95(9):1082�

1095.

Allcott, H. and Rogers, T. (2014). The short-run and long-run e�ects of behavioral interventions:

Experimental evidence from energy conservation. American Economic Review, 104(10):3003�

3037.

Ayres, I., Raseman, S., and Shih, A. (2013). Evidence from two large �eld experiments that peer

comparison feedback can reduce residential energy usage. The Journal of Law, Economics, and

Organization, 29(5):992�1022.

Bandura, A. (1977). Self-e�cacy: Toward a unifying theory of behavioral change. Psychological

Review, 84(2):191�215.

Barankay, I. (2012). Rank incentives: Evidence from a randomized workplace experiment. Working

Paper.

Beshears, J., Choi, J. J., Laibson, D., Madrian, B. C., and Milkman, K. L. (2015). The e�ect of

providing peer information on retirement savings decisions. Journal of Finance, 70(3):1161�1201.

Brent, D., Cook, J., and Olsen, S. (2015). Social comparisons, household water use and participation

in utility conservation programs. Journal of the Association of Environmental and Resource

Economists, 2(4):597�627.

Bursztyn, L. and Jensen, R. (2015). How does peer pressure a�ect educational investments? The

Quarterly Journal of Economics, 130(3):1329�1367.

Clee, M. and Wicklund, R. (1980). Consumer behavior and psychological reactance. Journal of

Consumer Research, 6(4):389�405.

Corgnet, B., Gomez-Minambres, J., and Hernan-Gonzalez, R. (2015). Goal setting and monetary

incentives: When large stakes are not enough. Management Science, 61(12):2926�2944.

Doherty, D. and Adler, E. S. (2014). The persuasive e�ects of partisan campaign mailers. The

Political Research Quarterly, 67(3):562�573.

31

Eisenkopf, G. and Friehe, T. (2014). Stop watching and start listening! the impact of coaching and

peer observation in tournaments. Journal of Economic Psychology, 45:56�70.

Ferraro, P., Miranda, J. J., and Price, M. (2011). The persistence of treatment e�ects with norm-

based policy instruments: Evidence from a randomized environmental policy experiment. Amer-

ican Economic Review, 101(3):318�322.

Festinger, L. (1954). A theory of social comparison processes. Human Relations, 7(2):117�140.

Fischer, C. (2008). Feedback on household electricity consumption: A tool for saving energy?

Energy e�ciency, 1(1):79�104.

Garcia, S., Tor, A., and Gonzalez, R. (2006). Ranks and rivals: a theory of competition. Personality

and Social Psychology Bulletin, 32(7):970�82.

Gerber, A., Green, D., and Larimer, C. (2008). Social pressure and voter turnout: Evidence from

a large-scale �eld experiment. American Political Science Review, 102(1):33�48.

Gill, D., Kissova, Z., Lee, J., and Prowse, V. (2016). First-place loving and last-place loathing: How

rankin the distribution of performance a�ects e�ort provision. Working Paper.

Hagger, M., Chatzisarantis, N., and Biddle, S. (2002). A meta-analytic review of the theories of

reasoned action and planned behavior in physical activity: Predictive validity and the contribution

of additional variables. Journal of Sport and Exercise Psychology, 24(1):3�32.

Harding, M. and Hsiaw, A. (2014). Goal setting and energy conservation. Journal of Economic

Behavior and Organization, 107A:209�227.

John, L. K. and Norton, M. I. (2013). Converging to the lowest common denominator in physical

health. Health Psychology, 32(9):1023�1028.

Kast, F., Meier, S., and Pomeranz, D. (2014). Under-savers anonymous: Evidence on self-help

groups and peer pressure as a savings commitment device. Harvard Business School Working

Paper.

Kraft-Todd, G. T., Yoeli, E., Bhanot, S. P., and Rand, D. G. (2015). Promoting cooperation in the

�eld. Current Opinion in Behavioral Sciences, 3:96�101.

32

Lee, J. and Tanverakul, S. (2015). Price elasticity of residential water demand in california. Journal

of Water Supply, 64(2):211�218.

Olmstead, S. and Stavins, R. (2007). Managing water demand price vs. non-price conservation

programs. Pioneer Institute for Public Policy Research, (39).

Pajares, F. (1997). Advances in Motivation and Achievement, chapter Current Directions in Self-

E�cacy Research. JAI Press, Greenwich.

Schultz, W., Nolan, J., Cialdini, R., Goldstein, N., and Griskevicius, V. (2007). The constructive,

destructive, and reconstructive power of social norms. Psychological Science, 18(5).

Shelton, T. and Mahoney, M. (1978). The content and e�ect of "psyching-up" strategies in weight

lifters. Cognitive Therapy and Research, 2(3):275�284.

Sillers, A. (2015). California court rules against tiered payment system for water usage. PBS

Newshour.

Tran, A. and Zeckhauser, R. (2012). Rank as an inherent incentive: Evidence from a �eld experi-

ment. Journal of Public Economics, 96(9-10):645�650.

United States Environmental Protection Agency, O. O. G. W. and Water, D. (2009). Water on tap:

What you need to know.

33

Appendix A

Figure A.1: Experiment Location

Figure A.2: Pressure Zones

34

Appendix B

Figure B.1: Home Water Report

35

Appendix C

Chart C.1: Households by Irrigable Area and Number of Occupants

1 Occupant 2 Occupants 3 Occupants 4 Occupants 5+ Occupants Total

Small 273 654 1,044 426 187 2,584

Medium 93 209 457 243 101 1,103

Large 17 24 59 24 11 135

Extra Large 7 16 27 12 9 71

Total 390 903 1,587 705 308 3,893

Chart C.2: Total Households and Household Receiving All Four Treatment Mailers

Total Households Households receiving all

mailers

Control 1,308 1,091

Treatment #1: Rank 1,288 1,050

Treatment #2: Competitive Rank 1,300 1,068

Total 3,896 3,209

36

Appendix D

Figure D.1: Water Use Trends Over Time

(Note: vertical red lines mark the sending of the four mailers)

Figure D.2: Disaggregated Water Use Trends Over Time by Prior Use

(Note: vertical red lines mark the sending of the four mailers)

37

Appendix E

Table E.1: Comparison of Parametric and Non-Parametric Tests for Main Results

Result Competitive

Rank vs.

Control,

All Periods

Neutral

Rank vs.

Control for

Low Users,

First

Period

Competitive

Rank vs.

Neutral

Rank for

Low Users,

First

Period

Competitive

Rank vs.

Neutral

Rank for

Low Users,

All Periods

Competitive

Rank vs.

Neutral

Rank for

High Users,

All Periods

Competitive

Rank vs.

Control for

�Last

Place�, All

Periods

Competitive

Rank vs.

Neutral

Rank for

�Last

Place�, All

Periods

p-value for

Mann-

Whitney U

test

p= 0.481 p= 0.167 p = 0.046 p = 0.137 p = 0.225 p < 0.0001 p = 0.009

Preferred

parametric

speci�cation

Table 4,Model 2

Table 5,Model 3

Table 5,Model 5

Table 6,model 4

Table 8,Model 4

Table 10,Model 6

Not Shown inRegresionTables

p-value for

parametric

speci�cation

p = 0.078 p = 0.032 p= 0.009 p= 0.069 p = 0.063 p = 0.003 p = 0.006

38