NBER WORKING PAPER SERIES

IDENTITY VERIFICATION STANDARDS IN WELFARE PROGRAMS: EXPERIMENTAL EVIDENCE FROM INDIA

Karthik MuralidharanPaul Niehaus

Sandip Sukhtankar

Working Paper 26744http://www.nber.org/papers/w26744

NATIONAL BUREAU OF ECONOMIC RESEARCH1050 Massachusetts Avenue

Cambridge, MA 02138February 2020

We thank Prashant Bharadwaj, Lucie Gadenne, Siddharth George, Aprajit Mahajan, Ted Miguel, and participants in various seminars for comments and suggestions. This paper would not have been possible without the continuous efforts and inputs of the J-PAL/UCSD project team including Avantika Prabhakar, Burak Eskici, Frances Lu, Jianan Yang, Kartik Srivastava, Krutika Ravishankar, Mayank Sharma, Sabareesh Ramachandran, Simoni Jain, Soala Ekine, Xinyi Liu, and Vaibhav Rathi. Finally, we thank the Bill and Melinda Gates Foundation (especially Dan Radcliffe and Seth Garz) for the financial support that made this study possible. The views expressed herein are those of the authors and do not necessarily reflect the views of the National Bureau of Economic Research.

NBER working papers are circulated for discussion and comment purposes. They have not been peer-reviewed or been subject to the review by the NBER Board of Directors that accompanies official NBER publications.

© 2020 by Karthik Muralidharan, Paul Niehaus, and Sandip Sukhtankar. All rights reserved. Short sections of text, not to exceed two paragraphs, may be quoted without explicit permission provided that full credit, including © notice, is given to the source.

Identity Verification Standards in Welfare Programs: Experimental Evidence from IndiaKarthik Muralidharan, Paul Niehaus, and Sandip SukhtankarNBER Working Paper No. 26744February 2020JEL No. D73,H53,O30,Q18

ABSTRACT

How should recipients of publicly-provided goods and services prove their identity in order to access these benefits? The core design challenge is managing the tradeoff between Type-II errors of inclusion (including corruption) against Type-I errors of exclusion whereby legitimate beneficiaries are denied benefits. We use a large-scale experiment randomized across 15 million beneficiaries to evaluate the effects of more stringent ID requirements based on biometric authentication on the delivery of India's largest social protection program (subsidized food) in the state of Jharkhand. By itself, requiring biometric authentication to transact did not reduce leakage, slightly increased transaction costs for the average beneficiary, and reduced benefits received by the subset of beneficiaries who had not previously registered an ID by 10%. Subsequent reforms that made use of authenticated transaction data to determine allocations to the program coincided with large reductions in leakage, but also significant reductions in benefits received. Our results highlight that attempts to reduce corruption in welfare programs can also generate non-trivial costs in terms of exclusion and inconvenience to genuine beneficiaries.

Karthik MuralidharanDepartment of Economics, 0508University of California, San Diego9500 Gilman DriveLa Jolla, CA 92093-0508and NBERkamurali@ucsd.edu

Paul NiehausDepartment of EconomicsUniversity of California, San Diego9500 Gilman Drive #0508La Jolla, CA 92093and NBERpniehaus@ucsd.edu

Sandip SukhtankarDepartment of EconomicsUniversity of VirginiaCharlottesville, VA 22904 srs8yk@virginia.edu

A randomized controlled trials registry entry is available at https://www.socialscienceregistry.org/trials/1620

1 Introduction

How should recipients of publicly provided goods and services prove their identity in order to access

these benefits? From accessing welfare benefits to obtaining a driver’s license to casting a vote, how

stringent ID requirements should be is a perennially controversial question around the world. The

core design issue is how to manage the tradeoff between Type-II errors of inclusion whereby benefits

are paid to non-eligible or “ghost” recipients against Type-I errors of exclusion whereby legitimate

beneficiaries are denied benefits to which they are entitled. While there is a large literature on how

to target people to be put on program beneficiary lists (Alatas et al., 2012, 2016; Niehaus et al.,

2013), there is much less evidence on the distinct question of how onerous it should be for citizens

to prove their identity at the point of receiving benefits.

This question is particularly salient in developing countries. Historically, states have invested

in the ability to better identify their citizens as they develop (Scott, 1998). During the past

two decades in particular “the number of national identification and similar programs has grown

exponentially. . . to the point where almost all developing countries have at least one such program”

(Gelb and Metz, 2018). Around two-thirds of these use biometric technology, reflecting the view that

this provides more reliable authentication than alternatives, particularly in settings with low levels

of literacy and numeracy.1 A leading case is India, where the government has now issued unique

identification (“Aadhaar”) numbers linked to biometric records to over 1.24 billion people and

is gradually integrating Aadhaar-based authentication into a range of applications. The extent to

which authentication should be mandated to receive welfare benefits has been a highly controversial

issue, contested all the way to the Supreme Court. Proponents argue that this is necessary to

prevent fraud, while critics argue that the requirement denies people their legal entitlements and in

doing so “undermines the right to life” (Khera, 2017). In a September 2018 ruling, India’s Supreme

Court allowed the government to mandate the use of Aadhaar for accessing social programs, making

it all the more urgent to understand how doing so affects errors of inclusion and exclusion.

This paper reports results from the first (to our knowledge) experimental evaluation of intro-

ducing Aadhaar as a requirement to collect welfare benefits. Specifically, we examine how this

introduction shifted the tradeoff between errors of inclusion and exclusion in the Public Distri-

bution System (PDS), India’s largest welfare program, accounting for roughly 1% of GDP. The

PDS is the primary policy instrument for providing food security to the poor in India, which has

the largest number of malnourished people in the world (FAO et al., 2019). In principle, PDS

beneficiaries are entitled to purchase fixed monthly quantities of grain and other commodities at

a highly-subsidized price from a government-run fair-price ration shop (FPS). In practice, the re-

sulting dual-price system creates strong incentives for corrupt intermediaries to divert grains to

the open market, with nation-wide estimated rates of leakage at 42% as of 2011-2012 (Dreze and

Khera, 2015). The Government of India aimed to reduce this leakage by requiring beneficiaries

1National biometric ID systems have recently been rolled out in Malawi, Senegal, and Uganda, amongst others,while Ghana, Kenya, and Tanzania are currently enrolling citizens. The World Bank has a dedicated initiative -ID4D - to help countries “realize identification systems fit for the digital age.” (https://id4d.worldbank.org/).

1

https://id4d.worldbank.org/

to obtain an Aadhaar number for at least one member of their household, link (or “seed”) it to

their PDS account, and then authenticate their identity by scanning the fingerprints of a seeded

household member each time they transacted at a ration shop.

To evaluate the impact of introducing Aadhaar in the PDS, we worked with the government of

the state of Jharkhand to randomize the order in which biometric authentication was introduced

across 132 sub-districts in 10 districts in the state of Jharkhand. Our evaluation sample is represen-

tative by design of 15.1 million beneficiaries in 17 of Jharkhand’s 24 districts, and representative

on observables of the rest of the state. Further, the integration of Aadhaar into the PDS was

implemented by the Government of Jharkhand (GoJH) as part of a full-scale deployment that was

being rolled out across the country. Thus, our study design allows us to directly estimate the

policy-relevant parameters of interest.

The GoJH implemented this reform in two phases. In the first phase, electronic Point-of-Sale

(ePoS) machines were installed in PDS shops in treated areas (as well as the rest of the state in

non-study districts) and beneficiaries were required to use Aadhaar-based Biometric Authentication

(ABBA) to collect their rations. The control group continued with the default of authentication

based on presenting a paper “ration card” to collect benefits, where it was much more difficult for

the government to verify if beneficiaries had in fact collected their rations. In the second phase

(“reconciliation”), which started 11 months later after stabilizing the implementation of the ABBA

system, the GoJH also started using the electronic records of authenticated transactions from

ePoS machines to adjust the amount of grain that was disbursed to PDS shops, simultaneously in

treatment and control areas.2 This was done after deploying the ePoS machines to control areas in

the two months prior to the onset of reconciliation. We therefore present experimental estimates of

the impact of requiring ABBA to collect benefits, and non-experimental estimates of the impact of

reconciliation using a pre-specified event study framework (that also uses unreconciled commodities

as a control group).

We conduct our analysis with a combination of administrative and survey data. The former

include data on disbursals of commodities by month across treatment and control blocks. The

latter include baseline and three follow up surveys of 3,840 PDS beneficiaries to measure receipts of

benefits as well as transaction costs in accessing them. One follow up survey was conducted before

the onset of reconciliation and before the rollout of ePoS machines to control areas, and we use

this to study the effects of ABBA alone. Two further follow-up surveys were conducted to study

the effects of reconciliation.

We first examine program implementation. Jharkhand is a relatively “low capacity” state (see

Section 2 for details) which one might expect to struggle with the sheer logistics of a major reform

such as Aadhaar integration. Yet we find the opposite: by our first follow-up survey, which measured

beneficiary experiences 6 to 8 months after treatment onset, 97% of beneficiary households in

2Specifically, prior to reconciliation, the government would send each FPS the full amount of grain needed to satisfythe entitlements of all beneficiaries registered to the FPS each month. After reconciliation started, the governmentwould only disburse the amount of grain that was shown to have been distributed by the FPS as per the ePoS recordsassuming that the FPS dealer still had the undistributed grains in stock.

2

treated areas had at least one member with an Aadhaar number seeded to the PDS account, and

90% reported that transactions at their FPS were being authenticated. Overall, implementation of

the reform was thorough and quick.

We then examine how this affected the transfer of value from the government to beneficiaries:

the value of goods disbursed by the former (from administrative data), received by the latter (from

household surveys), and the difference between these (i.e. leakage). The reform was framed by

the central government as part of a broader effort to achieve fiscal savings by reducing fraud,

with the possibility but no commitment that such savings would be fed back into the PDS or

other safety net programs in the future. Ex ante, the impact was ambiguous, with the extent of

leakage reductions depending on how exactly the government would choose to use the data on

authenticated transactions to determine grain disbursals. Further, ABBA could potentially help to

mitigate “identity fraud”, whereby benefits were collected by others. On the other hand, ABBA

could potentially increase exclusion error or add to transaction costs due to authentication failures.

Moreover, it was not designed to address the problem of “quantity fraud” (where PDS dealers give

beneficiaries less than their entitlement). As one official put it: “Even if someone uses his thumb

and gets 2 kg of rice instead of 5 kg, what can you do?”3 The design of the reform contrasts

in this sense with the related biometric “Smartcards” reform we previously studied in Andhra

Pradesh (Muralidharan et al., 2016), where the government focused more on improving the end

user experience rather than on fiscal savings (a contrast to which we return below).

In practice, we find that the impacts of ABBA on beneficiaries were small on average and, where

significantly different from zero, negative. Consistent with not using the data on authenticated

transactions for adjusting grain disbursal in the first phase of the reform, ABBA did not decrease

(and if anything slightly increased) government spending. It also did not increase mean value

received by beneficiaries substantially or significantly, and had no impact on leakage. We can reject

changes in value received by beneficiaries outside of [−4.3%, 3.9%] of value disbursed, and changesin leakage outside of [−1.7%, 6.5%]. We see no meaningful changes in measures of the quality ofgoods received, of their market prices, or of beneficiaries’ food security. Beneficiaries, however, did

incur a 17% higher transaction costs to collect their benefits (a Rs. 7 increase on a base of Rs. 41),

driven mainly by an increase in the number of unsuccessful trips made to the ration shop, and the

opportunity cost of doing so.

While average benefits received may not have decreased, more stringent ID requirements could

have excluded a vulnerable minority who were unable to meet the new identification standards.

Extensive margin effects are consistent with this possibility, as treatment increased the probability

that a beneficiary received no commodities at all in any given month by 2.4 percentage points

(p = 0.099). To examine this further, we focus on the 23% of households who did not have at least

one member’s Aadhaar number seeded to their PDS account at baseline, and were thus at greater

risk of being excluded. Unseeded households tend to be poorer and less educated than their seeded

3“Death by Digital Exclusion,” The Hindu, https://www.thehindu.com/news/national/other-states/death-by-digital-exclusion/article28414768.ece, accessed 13 July 2019.

3

https://www.thehindu.com/news/national/other-states/death-by-digital-exclusion/article28414768.ecehttps://www.thehindu.com/news/national/other-states/death-by-digital-exclusion/article28414768.ece

peers. Among unseeded households, exclusion errors increased significantly: the mean value of rice

and wheat received fell by Rs. 49, or 8.4% of value disbursed, and the probability of receiving none

of these commodities increased by 10 percentage points (a 50% increase on a base of 20%).

Overall, these results are consistent with the critique that biometrically authenticating transac-

tions per se caused at least some “pain without gain” (Dreze et al., 2017). A potential counterar-

gument, however, is that authenticating transactions was a necessary first step towards the second

phase of the reform which the government subsequently introduced. After rolling out Aadhaar and

ePoS devices in both treatment and control subdistricts, the GoJH began in July 2017 to reconcile

its monthly shipments of grain to each FPS against authenticated transaction data for previous

months, sending less grain to dealers it believed - based on the data - should have had more stock

remaining from their previous months’ allotment.

The data suggest that reconciliation initially had a substantial impact. Focusing first on the

control group, which began reconciliation shortly after beginning ABBA itself, we find that the

introduction of reconciliation coincided with an 18% (Rs. 92 per ration card per month) fall

in the value of reconciled commodities disbursed by the government. This drop is specific to

the two commodities (rice and wheat) subject to reconciliation and does not appear for the three

unreconciled commodities (sugar, salt, and kerosene). Of the drop, we estimate that 22% represents

a reduction in value received by beneficiaries (based on household-survey data). Thus, of the total

reduction in disbursals, 78% represents a genuine reduction in leakage.

In treatment areas, effects of reconciliation were more pronounced overall, and the tradeoff be-

tween errors of exclusion and inclusion somewhat less advantageous. Disbursements fell initially

by 36% (Rs. 182 per ration card per month), of which 34% represents a drop in value received

by beneficiaries and 66% a reduction in leakage. The former effect was largely felt on the exten-

sive margin: the probability that a household received no reconciled commodities increased by 10

percentage points, enough to account fully for the drop in average receipts. The larger effect in

treated areas reflects the fact that, dealers in the treated areas had been implementing ABBA for

roughly nine months longer. Thus, based on the ABBA records maintained on the ePoS machines,

treated areas held significantly greater undisbursed stocks of grain, and thus received significantly

less grain from the government. In practice, dealers may not have had these stocks of grain (likely

due to having diverted undisbursed stocks to the open market), and thus the larger reduction of

grain disbursement in treated areas may have been passed on to beneficiaries since they may not

had grains left to distribute.

Interestingly, the reduction in leakage is also consistent with the expectations of PDS dealers:

those treated early reported a 72% lower expected future bribe price for FPS licenses, suggesting

that they expected a substantial fall in the potential for extracting rents from an FPS. However, in

cases where the dealers had already diverted or not properly stored the undisbursed grain in prior

months, the reduction in disbursal from the government may have been passed on to beneficiaries,

as seen in our results.

Overall the reconciliation policy was unpopular, drawing complaints from both dealers and ben-

4

eficiaries and demands for waivers and exemptions. Effects on both leakage and beneficiaries were

short-lived and largely attenuated within three months, after which the GoJH temporarily rescinded

the policy. One interpretation of this outcome, and especially of the difference in the effects of rec-

onciliation between experimental arms, is that holding dealers accountable for past diversion may

have contributed to the increased exclusion we find after reconciliation. The government might

have obtained a more favorable tradeoff between inclusion and exclusion errors had it instead given

all dealers a “clean slate” when introducing reconciliation. We examine this by exploiting the ex-

perimentally induced variation in opening balances to extrapolate to a “clean slate” scenario, and

estimate that this would have reduced leakage while if anything weakly increasing value received

by beneficiaries.

A longer-run analogue to reconciliation is to delete ration cards that have not been seeded,

thereby removing them from the eligibility list entirely. We find that the rate of card deletions

increased after the onset of ABBA and reconciliation, and that deleted cards included both true

“ghosts” as well as non-ghost recipients (based on data on the subset of households we sampled

and surveyed). While purely descriptive, these results highlights another margin along which the

rollout of Aadhaar likely cut leakage at the cost of some exclusion.

Overall, biometric authentication in Jharkhand’s ePOS was not a free lunch: depending on how

it was used, it either did not reduce errors of inclusion or leakage or did so at the cost of increased

exclusion error. While the combination of ABBA and reconciliation did reduce leakage, around

22-34% of the reduced disbursals represented a reduction in beneficiary receipts. If we consider the

conservative case of reconciliation in the control group, a planner would need to value marginal

revenues at at least 28% (i.e. 22%/78%) of the value placed on transfers to marginal households in

order to prefer such a policy to the status quo. In this specific case, the benefits of reduced disbursal

may have been even lower as the savings were only notional, yielding an increased stock of grain in

public warehouses as opposed to reduced spending.4 In the conclusion, we discuss several practical

ways to reduce the likelihood of exclusion errors while still achieving leakage reductions.

Our most important contribution is to provide the first experimental evidence on the trade-off

between Type I and Type II errors from introducing stricter ID requirements for receiving welfare

benefits, and to do so using an at-scale experiment across 15 million people in the context of the

largest welfare program (PDS) in the country with the largest biometric ID program in the world

(India). Our results showing that reductions in leakage came at the cost of increased exclusion

errors and inconvenience to beneficiaries are directly relevant to policy discussions regarding the

use of more stringent ID requirements to access public services in India and other countries.5 More

broadly, they add to the evidence base on how transaction costs affect the incidence of welfare

benefits (e.g. Currie (2004) and more recently Alatas et al. (2016)). As predicted by Kleven and

Kopczuk (2011), they illustrate how increasing the complexity of the process of obtaining benefits

4Over time, fiscal savings may be possible by reducing the amount of grain procured from farmers, but no suchpolicy change has been announced during the period of introducing ABBA and reconciliation into the PDS.

5They also provide a counterpoint, for example, to recent panel-data evidence that voter ID requirements havehad surprisingly little effect on voter participation in the United States (Cantoni and Pons, 2019).

5

can affect their overall incidence by decreasing takeup among eligible households. “Complexity”

does not appear to have been an effective screening device in this case, however, as the households

excluded generally appear less well off on socioeconomic measures.6

Second, our results illustrate the potential “shadow costs” of controlling corruption. Following

early theoretical debate about the costs and benefits of corruption (Leff, 1964; Huntington, 1968;

Shleifer and Vishny, 1993), the last decade or so of micro-empirical work has (as we read it) typically

found that corruption is harmful and that there exist methods of reducing it that are cost-effective,

at least in the sense that the benefits are large compared to the direct costs of intervention, such

as the cost of hiring auditors (see for example Olken (2007) or Duflo et al. (2013), among others).

However, in many settings the indirect costs may also matter. Rigid procurement procedures, for

example, may limit the scope for graft but also slow down decision-making and make it hard to act

on “soft information” (Wilson, 1989). In our setting, the cost of controlling corruption was not just

the direct cost of issuing Aadhaar numbers and biometric readers, but included the (considerable)

indirect cost of excluding marginalized households from their legally entitled benefits.7

Finally, our findings are also relevant to ongoing discussions about “external validity” in program

evaluation – that is, how best to predict the effects of reforms from the results of past experiments.

In prior work (Muralidharan et al., 2016), we found that introducing biometric payments in rural

welfare programs in the state of Andhra Pradesh (AP) both reduced leakage and improved the

payment experience. However, the impacts of ABBA and reconciliation in the PDS in Jharkhand

were quite different. Our results, and in particular the fact that the rollout in Jharkhand was much

faster than that in AP, suggest that this was less because of differences in state or implementation

capacity, but more because of key differences in program design and emphasis. Leakage fell in

both cases, but the rules in AP aimed to enhance the beneficiary experience (and thus passed on

the benefits of reduced leakage to beneficiaries, while achieving no fiscal savings), while those in

Jharkhand focused on achieving fiscal savings (resulting in reduced disbursals and leakage, but also

passing on some pain to beneficiaries). Overall, the results highlight the importance of construct

as well as context in extrapolating results from one case to another. They also caution against a

simplistic attempt to characterize the effects of new technologies such as biometric authentication

without paying careful attention to design details and to the beneficiary experience.8

The rest of the paper is organized as follows. Section 2 describes the context and intervention.

Section 3 presents the research design including data collection and estimation strategy. Section

4 describes results of point-of-sale authentication, Section 5 describes results of reconciliation and

ration card deletion, and Section 6 offers a concluding summary.

6In addition to the tradeoffs we discuss here, implementing large-scale biometric ID schemes typically involvestradeoffs between state capacity and privacy. See Gelb and Metz (2018) for further discussion.

7In related work, Lichand and Fernandes (2019) find that the threat of audits reduced corruption but also displacedspending on services such as public health care in Brazilian municipalities, and that this led to worsening of somelocal public health outcomes.

8The point that the impacts of technology-based interventions depend crucially on design details is also seen in theliterature on education technology, where impacts on learning outcomes vary widely as a function of how effectivelytechnology is (or is not) used to improve pedagogy (see Muralidharan et al. (2019a) for a review).

6

2 Context and intervention

Malnutrition remains a serious problem today in India, which ranked 102 of 117 countries in the

most recent Global Hunger Index Rankings (Grebmer et al., 2019) and had an estimated 38% of

children stunted and 36% underweight as of 2015-2016 (UNICEF et al., 2017). The Public Dis-

tribution System (PDS) is a central piece of the government’s efforts to provide food security to

the poor. Through a network of over 527,000 ration shops known as “Fair Price Shops” (FPS), it

disburses subsidized wheat and rice to targeted households on a monthly basis, and other commodi-

ties such as sugar, salt, and kerosene on an occasional basis. Under the National Food Security

Act of 2013, the government has a mandate to include 75% (50%) of the rural (urban) population

as beneficiaries. Individual states administer targeting and distribution within their boundaries.

Overall, the PDS costs roughly 1% of GDP to operate.9

Because it creates a dual-price system, distributing commodities at prices well below their market

prices, the PDS has historically suffered from various forms of diversion. Commodities “leak” from

the warehouses and trucking networks meant to deliver them to the Fair Price Shops, or from the

shops themselves; dealers adulterate commodities or over-charge for them. Historically estimated

leakage rates have been high; Dreze and Khera (2015) estimate that 42% of foodgrains nationwide

and 44% in Jharkhand were diverted in 2011-2012, which is itself an improvement on the estimate

of 73% by the Planning Commission in 2003 (The Programme Evaluation Organisation, 2005).

Various reforms meant to address these challenges are underway, including several grouped un-

der the broad heading of “PDS computerization.” We focus on one of the major components

of computerization: the introduction of electronic point-of-sale (ePOS) devices to process and

record transactions between dealers and beneficiaries. As we describe below, these devices enabled

Aadhaar-based biometric authentication (ABBA) as well as the creation of a digital transaction

ledger. Rollout of these devices was well underway elsewhere in India by the time the GoJH began

its deployment; as of July 2016 an estimated 23% of India’s FPSs had received devices, rising to

54% by December 201710 with the rollout ongoing.11

ePOS devices perform biometric authentication using Aadhaar, India’s landmark unique ID sys-

tem. The Government of India launched Aadhaar in 2009 with the goal of issuing an identification

number linked to biometric information for every resident of the country. As of June 2019, it had

issued Aadhaar numbers to 1.24B people, or 91% of the country’s population.12 Investments in ID

9The PDS is enabled in part by India’s policy of a Minimum Support Price for essential commodities like riceand wheat combined with public procurement of these commodities from farmers. The resulting stocks of foodgrainwith the government are then distributed to the poor through the PDS. In this way, Indian agriculture and foodpolicy intervenes in both the production and distribution side of the market. For PDS expenditures, see http://www.indiabudget.gov.in/ub2018-19/eb/stat7.pdf. For GDP estimates, see https://dbie.rbi.org.in/DBIE/dbie.rbi?site=statistics. Both sources accessed on 5 March, 2018

10For July 2016 statistics, see http://164.100.47.190/loksabhaquestions/annex/9/AS26.pdf/. For December2017 statistics, see http://pib.nic.in/PressReleseDetail.aspx?PRID=1512902. Both sources accessed 5 March2018.

11Other PDS computerization initiatives included digitization of beneficiary databases, computerization of supply-chain management, and creation of grievance redressal mechanisms and online transparency portals.

12For statistic on number of Aadhaar UIDs generated, see https://uidai.gov.in/aadhaar_dashboard/india.php.

7

http://www.indiabudget.gov.in/ub2018-19/eb/stat7.pdfhttp://www.indiabudget.gov.in/ub2018-19/eb/stat7.pdfhttps://dbie.rbi.org.in/DBIE/dbie.rbi?site=statisticshttps://dbie.rbi.org.in/DBIE/dbie.rbi?site=statisticshttp://164.100.47.190/loksabhaquestions/annex/9/AS26.pdfhttp://pib.nic.in/PressReleseDetail.aspx?PRID=1512902https://uidai.gov.in/aadhaar_dashboard/india.php

could be particularly important in India given its historically unusual situation as a country with

a substantial welfare state at relatively low levels of per capita income, and indeed the government

has touted Aadhaar as an enabling technology which will support reforms to the implementation

of a wide range of government schemes – “a game changer for governance,” as the Finance Minister

at the time put it (Harris, 2013). Abraham et al. (2017) estimate that it was being applied to at

least 558 use cases as of 2017. Government claims regarding the fiscal savings achieved by intro-

ducing Aadhaar have at times been met with skepticism (Khera, 2016), however, in part because

they did not differentiate between real reductions in leakage and increased exclusion of legitimate

beneficiaries. To our knowledge, however, there has been no experimental evidence to date on the

impacts of an Aadhaar deployment in any welfare program.

Jharkhand is a relatively challenging environment in which to roll out an ambitious reform

such as ABBA. In terms of state capacity, it ranked 17th of 19 major states on the most recent

Governance Performance Index (Mundle et al., 2012), well below 3rd-ranked Andhra Pradesh in

which our previous evaluation of biometric authentication was set. As one concrete example, it had

the highest rate of teacher absence among all Indian states in both 2003 and 2010 (Muralidharan et

al., 2017). Jharkhand also rated relatively low in terms of key pieces of enabling infrastructure such

as rural teledensity (40 telephone or mobile phone connections per 100 people in rural Jharkhand

as of 31 October 2017, ranked 19 out of 19 reported states) and at the middle of the pack for

Aadhaar penetration (93% penetration as of 31 December 2017, ranked 17th of 36 states).13

2.1 The intervention

In August 2016, the GoJH introduced ePoS machines in FPSs to authenticate beneficiaries when

they came to collect their rations (Figure 2 provides the rollout timeline). In August 2016, the

GoJH introduced ePoS machines in FPSs to authenticate beneficiaries when they came to collect

their rations (Figure 2 provides the rollout timeline). Prior to the intervention, authentication

in the Jharkhand PDS was relatively informal. Each beneficiary was assigned to a unique FPS

and issued a ration card listing members of the household and displaying a photograph of the

household head. To collect benefits, any one of these listed household members was required to

appear in person with the ration card at the assigned FPS. Anecdotally it was not uncommon for

neighbors or friends to collect benefits on their behalf, or for dealers to hold on to beneficiaries’

ration cards themselves. Dealers were expected to record transactions both on ration cards and in

their own ledgers; ledgers were typically not audited, and anecdotally there was wide variation in

record-keeping practices.14

The reform modified authentication and record-keeping processes. The state gave each dealer

For total population statistics, see https://data.worldbank.org/indicator/SP.POP.TOTL.13For rural teledensity statistics, see http://164.100.47.190/loksabhaquestions/annex/13/AU2751.pdf, ac-

cessed March 5, 2018. For Aadhaar penetration statistics, see https://uidai.gov.in/enrolment-update/ecosystem-partners/state-wise-aadhaar-saturation.html, accessed January 31, 2018.

14One common practice is to keep separate “official” and “unofficial” ledgers, where the unofficial ledgers accountedfor actual transactions including leakage while official clean ledgers would be produced in case of a government audit.

8

https://data.worldbank.org/indicator/SP.POP.TOTLhttp://164.100.47.190/loksabhaquestions/annex/13/AU2751.pdfhttps://uidai.gov.in/enrolment-update/ecosystem-partners/state-wise-aadhaar-saturation.htmlhttps://uidai.gov.in/enrolment-update/ecosystem-partners/state-wise-aadhaar-saturation.html

an ePOS device configured to authenticate beneficiaries in one of three modes: online, offline, and

partially online.15

In online mode, the device required the operator to input a ration card number. It then displayed

a list of all individuals who were both (i) listed as beneficiaries on the relevant ration card, and

(ii) had an Aadhaar number linked (“seeded”) to the card. The dealer selected the beneficiary

present, and the device then prompted him/her to place a finger of choice on the device’s scanner

to be authenticated against the central Aadhaar database. If fingerprint authentication failed on

three consecutive attempts, the beneficiary could opt to receive a one-time password texted to their

mobile phone number as a fallback method of authentication.16

In offline mode, the device simply captured and stored fingerprint information for the person

collecting benefits but performed no authentication checks. However, transaction logs were meant

to be synchronized with a server periodically (as explained below).

In partially online mode, the device functioned as in online mode if it detected a network con-

nection and in offline mode otherwise. Dealers did not have discretion to select modes (but could

potentially have tried to force the device to operate in offline mode by disrupting connectivity).

The government varied the mode assigned to each FPS in an effort to balance the risks of inclusion

and exclusion error: it sought to enforce relatively strict authentication requirements in areas where

connectivity was strong enough to provide a reliable connection to the central Aadhaar database,

but not deny benefits to legitimate beneficiaries in areas where connectivity was weaker.17 In

our experimental design assignment to receive a machine was random but assignment to machine

mode was not, so that the effects we report represent an average of mode-specific effects given

the assignment policy described here. We also report a non-experimental decomposition assuming

that, had they been treated early, control Fair Price Shops would have been treated with the same

machine mode to which they were subsequently assigned.

ePOS devices also enabled digital record-keeping. After authentication, the device would display

any previously uncollected commodity balances to which the beneficiary was entitled, including the

current months’ entitlement and any uncollected balance from the previous month (but not balances

from two or more months previous). After completing a transaction the dealer would record the

amount of each commodity purchased in the device, which would print a paper receipt and also

voice the transaction details in Hindi. Dealers were instructed to give the receipt to the recipient

as well as recording the transaction in their ration card. In practice, recipients often reported

not receiving receipts or that these faded quickly. In any case, the digital ledger maintained in

the device became the source of truth for balance information from the government’s perspective,

15Most FPS were assigned to the online mode (81% of shops), with 15% offline and only 4% partially online onaverage prior to August 2017. In August 2017 the government ended the use of partially online mode after which88% of FPSs operated in online mode with the remaining 12% offline.

16Some officials claimed that at least initially if neither method of authentication succeeded there was an “override”option available allowing the dealer to authenticate a beneficiary without using Aadhaar, but officially no such optionwas meant to exist.

17In data collected by our survey team, the proportion of FPS at which no cellular signal could be detected was5% for shops with online devices, 10% for shops with partially online devices, and 58% for shops with offline devices.

9

though dealers were of course free to maintain their own parallel paper records if they wished.

The government accessed transaction data by synchronizing (“syncing”) regularly with each

device. Online devices synced their records with a central government server automatically in real

time. Dealers using partially online and offline devices were instructed to sync data within 48 hours

of a transaction, but did not face any obvious repercussions if they did not. Instead their binding

constraint appeared to be monthly: devices would not authorize new transactions in a given month

until the previous month’s transactions had been synced.

The process of seeding Aadhaar numbers to ration cards was ongoing during the period we study.

To seed their ration card, a household first needed to have at least one of the members listed on

the ration card obtain an Aadhaar number, either at camps organized specially for this purpose or

subsequently by applying at the local block or district office. It then needed to link this Aadhaar

number to its ration card, again either at camps organized for this purpose during NFSA enrollment

or by applying at the block or district office. As of May 2016, 76.5% of ration cards in areas assigned

to treatment and 79.9% of those in areas assigned to control had been seeded with at least one

Aadhaar number. These figures had risen to 94.5% and 92.7%, respectively, by October of 2016.

Finally, by May 2018, these figures had risen further to 99.8% and 99.5%, respectively. The seeding

process could itself have affected errors of inclusion and exclusion, e.g. if the government choose to

delete ration cards that had not been seeded after some interval in an effort to eliminate ghosts.

The GoJH’s stated policy was not to do so, but anecdotes circulated of cases in which this occurred.

We examine this further in Section 5.3 below.

2.2 Reconciliation

Prior to the introduction of Aadhaar-based biometric authentication using ePOS devices, the GoJH

rarely (if ever) reconciled balances with FPS dealers. For example, if the grain needed to serve

all PDS beneficiaries assigned to a given FPS was 100kg of rice per month, it was GoJH policy

to ship 100kg of rice to that FPS each month irrespective of how much rice it had distributed to

beneficiaries in previous months. This reflected in part the simple fact that the government had

no timely and reliable data on transactions at the shops.

By June of 2017, ePoS devices were actively in use for authentication in 93% of Fair Price

Shops in our study area, including those in control blocks, where they were rolled out during

April and May. Starting in July, therefore, the government introduced a second reform, reconciling

its disbursements of rice and wheat, though not of sugar, salt or kerosene. The full formula the

government used to determine disbursements under this regime is in Appendix C. To summarize,

the government’s new policy was to calculate (a) the amount each dealer would need to meet claims

by beneficiaries against the current month’s entitlements, as well as any outstanding claims on the

preceding one month’s entitlements, and (b) the amount the dealer should have in stock given

the full history of deliveries and transactions (starting from the time the FPS first used an ePoS

device), and then disburse the difference between these quantities.

From a dealer’s perspective, this reform (if implemented by the book) had two effects. First, it

10

had a retrospective effect, reducing the amount of rice and wheat received starting in July: dealers

who had not distributed the full amounts disbursed to them in previous months (as recorded by

the ePoS machines) received less. We would expect this effect to be larger for dealers in treatment

blocks, since as of July they had been using devices for 11 months as opposed to 1-2 months for

dealers in control blocks. As we discuss below, many dealers had “opening balances” at the onset

of reconciliation equivalent to over a month of entitlement (based on the amount of grain they had

received in previous months against which no authenticated transaction log existed). This implied

that by rule they should have received no incremental grain at all, since they were supposed to

be holding enough stocks of grains to make all program-required disbursals in July. Second, the

reform prospectively affected dealers’ marginal incentives to report via the ePOS devices that they

had distributed grain to ration card holders, since reporting less than full distribution would reduce

the amount they received the next month. The legitimate incentive to do so was the commission

of 1 rupee per kilogram of grain they received for distributing commodities. In addition, receiving

more grains would also make it easy to divert some while still providing beneficiaries with a given

level of benefits.

From a beneficiary’s perspective the consequences of reconciliation are unclear. On one hand,

dealers might pass on some share of the reduction in grains disbursed to them, reducing in turn

the amounts distributed to beneficiaries. On the other hand, dealers might distribute more grain

to beneficiaries in order to increase future disbursements. Note, however, that strictly speaking

the reform created incentives for dealers to report that they had distributed grain, not to actually

distribute it. They needed beneficiaries to appear and scan their fingerprints to do so, but did not

need to give beneficiaries the amount of physical grain that they recorded. Anecdotally, some dealers

told beneficiaries that they would enter the full amounts into the devices even while distributing

less or none, since otherwise there would be less grain to distribute in the subsequent month. To

summarize, using the terminology in Niehaus and Sukhtankar (2013), ABBA and reconciliation

likely made it more difficult for dealers to divert grains through “over-reporting” the number of

beneficiaries (including making up fake or ghost beneficiaries), but it may not have altered their

incentives for “under-payment” of benefits to genuine beneficiaries.

2.3 Summary

Overall, the reforms introduced by the GoJH were representative of the way in which the Gov-

ernment of India has envisioned using Aadhaar to reform program administration. In particular,

they made possession of an Aadhaar number effectively mandatory for the receipt of PDS benefits

(despite a 2013 Supreme Court ruling prohibiting this).18 A priori one would thus expect it to have

both strong potential to reduce errors of inclusion, and a high risk of generating additional errors

of exclusion. Media criticism has argued that it has done exactly that, leading in some cases to

18http://judis.nic.in/temp/494201232392013p.txt

11

http://judis.nic.in/temp/494201232392013p.txt

preventable starvation deaths –“death by digital exclusion,” as one headline put it.19

Compared to the Smartcards reform we previously studied in Andhra Pradesh (AP) (Muralid-

haran et al., 2016), the reform in Jharkhand exhibits some similarities and some differences. Both

interventions introduced biometric authentication, but the implementation was generally stricter

in Jharkhand. Many devices in Jharkhand operated in online mode and required connectivity

to function, while all devices in AP operated offline; devices in AP featured a manual override

mechanism for use if biometric authentication failed, while we have no evidence to suggest such a

mechanism was used in Jharkhand. Meanwhile, (food) balances in Jharkhand were not reconciled

initially, while (cash) balances in AP were reconciled from the outset. Finally, in AP the location

where program participants collected their benefits was moved (from post offices to customer ser-

vice providers within each village), while in Jharkhand the location was held constant at the Fair

Price Shop.

3 Research design

Our research design follows a pair of pre-specified and pre-registered analysis plans, one for the

evaluation of Aadhaar-based authentication itself and another for the analysis of reconciliation.20

Appendix D provides a comprehensive list of analysis reported in addition to what was pre-specified.

3.1 Randomization

To obtain policy-relevant estimates, we sought to design an evaluation that was “at scale” in each

of the three senses identified by (Muralidharan and Niehaus, 2017). These include conducting our

study in a sample that is representative of the (larger) population of interest, studying the effects

of implementation at large scale, and having large units of randomization to capture general equi-

librium or other spillover effects such as changes in the market prices of subsidized commodities.21

We first sampled study districts. Of Jharkhand’s 24 districts, we excluded 1 in which the in-

tervention rollout had already begun and 6 in which a related reform (of Direct Benefit Transfers

for kerosene) was being rolled out. From the remaining 17 districts, home to 24M people and

15.1M PDS beneficiaries, we randomly sampled 10 within which to randomize the rollout of the

intervention.22 This design ensures representativeness of the 17 districts in our frame. In practice

19“Death by Digital Exclusion.” The Hindi, 13 July 2019. https://www.thehindu.com/news/national/other-states/death-by-digital-exclusion/article28414768.ece, accessed 13 July 2019.

20https://www.socialscienceregistry.org/versions/39275/docs/version/document and https://www.socialscienceregistry.org/versions/39274/docs/version/document respectively.

21Each of these three design choices helps to improve external validity. Conducting experimental evaluations innear-representative samples helps by reducing the risk of site-selection bias (Allcott, 2015). Evaluating a large-scale implementation helps because effect sizes have been shown to decline with size of implementation (Vivalt,forthcoming), Finally, randomizing large units into treatment and control status helps produce estimates that areinclusive of spillovers, which have been shown to be salient for policy in several studies including Cunha et al. (2018),Muralidharan et al. (2016), and Egger et al. (2019).

22We used stratified random sampling, stratifying on three variables related to geography and socio-economicstatus. We used these 3 binary variables to classify the 17 available districts into 8 (2x2x2) distinct cate-gories. We then sampled half of the districts in each category, rounding down to the nearest integer and us-

12

https://www.thehindu.com/news/national/other-states/death-by-digital-exclusion/article28414768.ecehttps://www.thehindu.com/news/national/other-states/death-by-digital-exclusion/article28414768.ecehttps://www.socialscienceregistry.org/versions/39275/docs/version/documenthttps://www.socialscienceregistry.org/versions/39274/docs/version/documenthttps://www.socialscienceregistry.org/versions/39274/docs/version/document

our 10 study districts appear comparable on major demographic and socio-economic indicators to

the 14 remaining districts of Jharkhand (Table 1). Our frame is thus arguably representative of

the full population of 5.6M PDS households and 26M PDS beneficiaries in the state.

The evaluation was conducted within the context of a full-scale rollout, as the GoJH deployed

ePOS devices to 36,000 ration shops covering the entire population of 26M PDS beneficiaries in the

state. This deployment involved a major effort by the government and was the stated top priority

of the Department of Food and Civil Supplies for the year and (anecdotally) the single largest use

to which they put staff time. We thus measure the effects of implementation at full scale by a

bureaucratic machinery fully committed to the reform, which are the effects of interest for policy

purposes.

Finally, we assigned treatment to large units. We randomized the rollout at the level of the sub-

district (“block”), which on average covers 73 Fair Price Shops and 96,000 people. Figure 1 maps

treated and control blocks and illustrates their geographic balance and coverage of the state. We

allocated 132 blocks into a treatment arm of 87 blocks and a control arm of 45 blocks, reflecting the

government’s preference to delay treatment in as few blocks as possible.23 Treatment and control

blocks are similar in terms of demographic and program characteristics, as one would expect (Table

2, Panel A). Of 12 characteristics we examine, one is marginally significant at the 10% level.

The GoJH complied closely and quickly with the treatment assignment. By the time of our follow-

up survey, households in treated blocks reported that 96% of dealers in treated blocks possessed an

ePOS device and 91% were using it to process transactions (Table 2, Panel B).24 ePOS utilization

was stable at 90-91% in treated blocks during January-March 2017, which increases our confidence

that we are estimating steady state impacts and not transitional dynamics. In control blocks, on

the other hand, 5% of dealers possessed a device and 6% were using it process transactions, largely

reflecting early rollout in one control block.25 Overall these figures suggest that it is sensible to

estimate intent-to-treat effects and to interpret them as fairly close approximations of the overall

average treatment effect.

ing probability proportional to size (measured as number of Fair Price Shops) sampling, and lastly sampled ad-ditional districts without stratification to reach our target of 10. Full details in the Pre-Analysis Plan: https://www.socialscienceregistry.org/versions/39275/docs/version/document.

23Within each district, we assigned blocks to treatment status as follows: We first divided blocks into rural and urbansamples, then stratified them into groups of three by ordering them on the first principal component of three variablesrelated to household size and benefit category. Within each group of 3 blocks we randomly assign 2 to treatment and1 to control. Full details, including how we dealt with districts with residual strata of fewer than 3 blocks, in thePre-Analysis Plan: https://www.socialscienceregistry.org/versions/39275/docs/version/document.

24This rollout was substantially faster than the Smartcards rollout in Andhra Pradesh, for example, which took2 years to cover 50% of transactions. The rollout of (non-Aadhaar) biometric authentication in Andhra Pradeshis analogous to the one we study here took six months to complete, according to government claims. See http://dfpd.nic.in/1sGbO2W68mUlunCgKmpnLF5WHm/mergedoriginal.pdf, accessed 27 July 2018. The difference in speedof execution reflects a combination of the priority placed on implementation by the national Department of Food andPublic Distribution as well as the fact that Aadhaar was more prevalent across citizens prior to the integration withthe PDS compared to Smartcards where enrollment had to be done from scratch.

25Of the 31 control households that report a dealer using an ePOS device, 24 are in one block. The remaining 7are scattered across 6 other blocks and most likely reflect reporting errors.

13

https://www.socialscienceregistry.org/versions/39275/docs/version/documenthttps://www.socialscienceregistry.org/versions/39275/docs/version/documenthttps://www.socialscienceregistry.org/versions/39275/docs/version/documenthttp://dfpd.nic.in/1sGbO2W68mUlunCgKmpnLF5WHm/mergedoriginal.pdfhttp://dfpd.nic.in/1sGbO2W68mUlunCgKmpnLF5WHm/mergedoriginal.pdf

3.2 Sampling and Data Collection

Our data collection focused on measuring three core concepts: the value of commodities disbursed

by the government, the value of commodities received by beneficiaries (both net of price paid), and

the real transaction costs incurred by dealers and beneficiaries to implement/obtain this transfer

of value. Leakage in this framework is simply the difference between value disbursed and value

received. Our concept of transaction costs includes a number of costs incurred by beneficiaries such

as the amount of time required to collect rations.

To measure these quantities we begin with administrative records. These include information

on monthly quantities of commodities disbursed to individual Fair Price Shops, which we obtained

from the National Informatics Commission.26 We obtained this data for all FPS shops in study

blocks and we use this administrative data as our primary source of data on disbursals (as opposed

to that reported by the sample of dealers we surveyed).

We next used the administrative database of eligible PDS beneficiaries and their assignment to

Fair Price Shops to draw samples of dealers and households to survey, and attempted to survey

them four times – once at baseline and then at three subsequent follow-ups. We sampled as follows:

from administrative records we drew a sample of 3 Fair Price Shops (FPS) in each study block, for

a total of 396 shops.27 We successfully interviewed the dealers operating 367 (93%) of these shops

at baseline, and 373 (94%) of them in the endline. Dealer surveys covered measures of the quantity

of commodities received by the shop each month, their operating costs, the dealers perceived value

of FPS licenses and interest in continuing to operate a ration shop, and stated preferences for the

reform as opposed to the status quo system. Enumerators also measured by hand the strength of

the four major cellular networks at the shop in order to capture connectivity.28.

For each sampled ration shop we sampled 10 households from the government’s list of PDS

beneficiaries,29 which had been created as part of a targeting exercise conducted in 2015 to comply

with the National Food Security Act of 2013. This generated a target sample of 3,960 households.

26In some cases we were also able to obtain and digitize disbursement records directly from District Supply Officers,Market Supply Officers, Block Development Officers, and godowns run by the Food Corporation of India and thestate of Jharkhand. These records generally correlated strongly (from 0.87 to 0.95 for various commodity × monthpairs) but not perfectly with the NIC records. We use the NIC records to ensure representative coverage, but obtainqualitatively similar results if we use the hand-captured ones instead.

27We sampled a fixed number of shops per block as we had previously selected study districts using PPS sampling.28In follow-up surveys, we expanded the number of dealers surveyed, as a few (7.9%) of our sampled households

had been re-assigned to new dealers in the normal course of operations during the 10 months since baseline. Wereport results for both the original and augmented dealer samples, as the reassignment rate of households is balancedacross treatment and control, and the incremental dealers are not statistically distinguishable from the original oneson measured characteristics (Table A.1). Note also that the reassignment of households to other shops does not affectour ITT estimates because we follow the originally sampled households. It also does not affect the first-stage or theinterpretation of our results because the reassignment was to other FPS in the same block, with the same treatmentstatus (which is another advantage of randomizing at the block level)

29We define household here as those individuals belonging to a single ration card. We first sampled one villagefrom the catchment area of each FPS using PPS sampling, with “size” defined as the number of ration cards in thevillage assigned to that FPS. We sampled ration cards using stratified random sampling, with strata including themethod by which the household became eligible for the PDS and the benefit category to which the cardholder isentitled. Full details in the Pre-Analysis Plan, https://www.socialscienceregistry.org/versions/39275/docs/version/document.

14

https://www.socialscienceregistry.org/versions/39275/docs/version/documenthttps://www.socialscienceregistry.org/versions/39275/docs/version/document

We attempted to interview these households for baseline and three follow-up surveys to create a

household-level panel. 30

Of the 3,960 ration cards we sampled, we identified and interviewed the corresponding household

at least once in 97% of cases.31 Overall, we estimate that at most 3% of beneficiaries were ghosts

(see Figure A.1 for a more detailed categorization of households). This is noteworthy as it suggests

that the scope for eliminating leakage by removing ghosts (or non-existent households) from the

beneficiary list was relatively limited in this setting.

We timed follow-up surveys and their associated recall periods to obtain continuous monthly

data on beneficiaries’ experiences with PDS from January through November of 2017. Figure

2 illustrates the recall window covered by each survey. We use data from follow-up 1, covering

January through March, to measures the impacts of ABBA, and use data from all three follow-ups

to examine the impacts of reconciliation. Topical coverage varied across surveys; follow-up 1 was

most comprehensive, while follow-ups 2 and 3 measured a subset of outcomes (e.g. for households,

the quantities of each commodity received). In particular, we did not measure market prices in

follow-ups 2 and 3 and so do not examine price effects of reconciliation.

3.3 Estimation strategy: Aadhaar-based biometric authentication

To examine the impacts of ABBA we estimate intent-to-treat specifications of the form

Y thfbs = α+ βTreatedbs + γY0hfbs + δs + �

thfbs (1)

where Y is an outcome measured for household h assigned to Fair Price Shop f in block b of

stratum s.32 Regressors include an indicator T for whether that block was assigned to treatment,

the baseline value Y 0hfbs of the dependent variable, and a stratum fixed effect δs. Where we observe

baseline values for multiple months we take their average. Where the baseline value is missing we

set it equal to the overall mean value, and include an indicator for baseline missingness. When

using survey data we weight specifications by (inverse) sampling probabilities to obtain results that

are representative of the sample frame.33 We use analogous specifications for outcomes measured at

the level of the Fair Price Shop or block. We pool observations for January-March 2017, following

our pre-specified plan for dealing with the possibility of non-stationary treatment effects.34

30Because our frame is the universe of households previously deemed eligible for the program, our sample is notsuited to examine errors of inclusion and exclusion in the process of determining who is eligible for the PDS, as inthe extensive literature on poverty targeting. Our focus here is rather on studying changes in inclusion and exclusionresulting from increased stringency in verifying the identity of those previously deemed eligible for benefits.

31We successfully interviewed 3,410 (86%) of these households at baseline and 3,583 (90%), 3,618 (91%), and 3,562(90%) at follow-ups 1, 2 and 3, respectively.

32Because the randomization algorithm created 6 strata (3 urban and 3 rural) of size 1, we create a single fixedeffect δs for each of these two groups.

33Variation in sampling probabilities was driven largely by field logistics constraints, e.g. the need to plan tointerview a fixed number of households per village rather than a fixed proportion.

34Specifically, we pre-specified that we would (i) estimate models for each month individually, pooled models, andpooled models with a linear interaction between treatment and month, and then (ii) choose which specification toprivilege based on the overall tendency of the trend terms to be significant predictors of primary outcomes. We

15

Each regression table below reports the percent of the original sample for which data were non-

missing and included in the estimation. In Tables A.2 and A.3 we examine missingness by treatment

status and generally do not find evidence of imbalance, with 9% of differences significant at the

10% level. We impute zeros when calculating quantities and value received for verified “ghost”

ration cards (which account for 1.6% of sampled households and do not differ across treatment and

control groups).

3.4 Estimation strategy: reconciliation

To examine the effects of reconciliation we examine time series variation in value disbursed and

received using the following pre-specified model:

Yhfbst = αhfbs + γt+ βRRt + βRtRt(t− t∗) + Pt + �hfbst (2)

where Rt is an indicator equal to one if disbursements for month t were calculated using the

reconciliation formula (i.e. for July through October), t∗ is the first month of reconciliation (i.e.

July), and Pt is an indicator for the one post-reconciliation month in our data (i.e. November).

We estimate the model separately for treated and control blocks. To compare the two, we pool

the data and interact (2) with an indicator for treatment. We report standard errors clustered

by FPS. In most cases, we have a single well-defined summary measure of outcomes such as value

disbursed or received. We adjust for multiple-hypothesis testing when reporting outcomes at the

individual commodity level, reporting both standard p-values and q-values adjusted to control the

false discovery rate.

This specification embodies several substantive assumptions. First, we assume the effect of

reconciliation is identified once we control for a linear pre-trend. This is a strong assumption,

but the best that is realistic with 6 months of pre-treatment data (and as it turns out yields an

excellent fit). Second, because we include a distinct indicator for November we do not impose

that outcomes immediately revert to what they would have been absent the intervention. While

the latter assumption would significantly improve power if true, we find it implausible. Third,

we model the potential for (linear) time variation in the treatment effect. This reduces power and

increases the risk of overfitting if the treatment effect is in fact time-invariant, but seems appropriate

given both that (a) theory suggests reconciliation should generate transitional dynamics, and (b)

anecdotes suggest that the government granted many waivers to the reconciliation policy, and these

may vary over time. Finally, we present results using time-series variation in value disbursed and

received for both reconciled commodities (rice and wheat), and unreconciled ones (salt, sugar, and

kerosene). The latter commodities provide a plausible contemporaneous control group to examine

the effects of reconciliation.

generally do not observe any evidence of trends, and therefore privilege the pooled estimators. This is consistent withthe fact discussed above that program implementation also appeared to have stabilized by the time of our follow-up.For completeness we report the other estimators in Appendix B.

16

4 Results: Biometric authentication

4.1 Value transfer

We measure value (V ) as the sum across commodities of quantity (Q) multiplied by the difference

between the local market price (pm) of that commodity and the statutory ration shop price (ps).35

Formally,

Vht =∑c

Qcht(pmht − psht) (3)

In total, ration card holders are entitled to a meaningful monthly amount. The quantity of com-

modities each household can purchase is capped at levels that depend on the category of ration

card it holds and the size of the household; the mean value of these entitlements evaluated using

Equation 3 is Rs. 595 per month, equivalent to 14% of the national rural poverty line for an average

household in our sample.36

In practice, households receive less than their entitlement. The mean value received in the control

group at follow-up was Rs. 463 per month, or 78% of the mean entitlement. This was largely not

because the government failed to disburse commodities, at least according to its own records, as

it disbursed commodities worth an average of Rs. 584 per month, or 98% of mean value entitled.

Rather, it reflects the fact that roughly 21% by value of the commodities the government did

disburse did not reach beneficiaries.

4.1.1 Value disbursed

Table 3 summarizes impacts on value transfer during January-March 2017, beginning in Panel A

with value disbursed by the government. Note that we observe this outcome for the universe of

FPS’s in our study area and therefore use all of these data, with outcomes expressed per rationcard

× month. We expect no meaningful changes to disbursements, as the government’s policy duringthis period was to disburse to each FPS in each month the full amount to which households

assigned to that shop were entitled. We find this is largely the case, though we do find some

modest substitution away from wheat and towards rice which results in a small but significant

increase in total value disbursed of Rs. 12 per ration-card month, or around 2%. This may reflect

adjustments in treated areas to an informal policy the government maintained of accommodating

regional differences in preferences for rice as opposed to wheat while keeping the total quantity of

foodgrains fixed at their entitled value.37 In any case, there is no evidence that ABBA saved the

35We find very little evidence of over-charging (below), and hence our results are essentially the same if we useactual as opposed to statutory ration shop prices. We obtained data on local market prices for equivalent commoditiesas those provided by the ration shops. Even if the prices reflected higher quality of market grains, that would notaffect our leakage calculations because we use the same price to estimate both value disbursed and value received.

36An average household in our sample had 4.4 members, and the national rural expenditure poverty line was Rs.972 / person / month (Commission, 2014). The poverty line had not been updated since 2014; if we adjust it upwardsfor changes in the rural consumer price index from 2014-2017, then the mean entitlement was 13% of the poverty linefor an average household.

37Specifically, government policy was to provide rice to rural blocks while providing rice and wheat in 3:2 proportionsto urban blocks and to exceptional rural blocks that expressed a desire for wheat. Given preferences for rice in

17

government money.

4.1.2 Value received

Panel B reports effects on value received by households using survey data. We see some directional

evidence of the shift from wheat to rice noted above, but no significant change in overall value

received. A 95% confidence interval for this effect is Rs. [−25.2, 22.8], letting us rule out a decreasegreater than 4.3% of value disbursed or an increase greater than 3.9% of value disbursed. Any

effects on value received by the average household were thus small in economic terms.

We also examine whether our null effect on total value masks offsetting changes in the underlying

prices and quantities. If the intervention reduces quantities flowing into rural markets which in turn

raises market prices, we might see no overall effect even though recipient welfare had changed. As

Panel B of Table A.4 reports, we see no significant changes in the mean quantity of any commodity

received, though directionally there appears to be a shift from wheat to rice as noted above. The

market prices households faced for these commodities also did not change significantly, with the

possible exception of a fall in the price of sugar which is marginally significant after adjusting for

multiple testing (Table A.5, Panel A).38

Our quantity-based measure of value received does not account for potential variation in the

quality of commodities. Allegedly, PDS dealers sometimes adulterate the goods they sell (e.g.

by adding sand or stones to wheat) or sell spoiled goods (e.g. rotten grains). We address this

in two ways. First, we asked respondents who had completed purchases whether they received

adulterated or low-quality goods. Generally speaking, few beneficiaries report experiencing these

issues and rates are unaffected by treatment (Table 4). In the control group, reported adulteration

rates range from 1% to 9% and none change significantly with treatment. Reported rates of quality

issues are similarly low with the exception that 38% of control households report receiving low-

quality salt; this rate is 6% lower among treated households, with the difference significant before

but not after adjusting for multiple comparisons.39

Second, we asked respondents what amount of money they would have been willing to accept

in lieu of the bundle of goods they purchased at the ration shop in each month. This metric has

important limitations; it measures stated as opposed to revealed preferences, and requires asking

subjects a series of questions which they often find confusing.40 On the other hand, it has the

Jharkhand, it is possible that more exceptions were made in treated areas. However, these exceptions were made onan ad-hoc basis and were not recorded.

38Interestingly, dealers do report facing lower prices, notably for rice (Table A.5, Panel B). We view these datacautiously as (i) unlike the household reports they are not based on actual transactions, and may not reflect thepricing that is relevant to the beneficiaries whose welfare we wish to examine, and (ii) only the effect on the price ofrice survives the adjustment for multiple testing.

39Response rates for these outcomes are relatively low as we observe them only for households that purchased apositive quantity.

40For households that purchased PDS commodities in a given month we elicited their stated willingness-to-accept(WTA) by asking for a series of values ranging from Rs. 100 to Rs. 2,000 whether they would have preferred to receivethat amount of money to the opportunity to purchase the commodities they obtained. We define their WTA as thesmallest amount of money for which they answered “yes.” For households that did not purchase any PDS commoditiesin a given month we define WTA as zero. An unusually high proportion (48%) of respondents answered “yes” for a

18

advantage of capturing all aspects of both quantity and quality as perceived by the beneficiaries.

When we replace our default measure of total value received with this alternative measure, we

estimate an insignificant reduction in value received of Rs. 11 per month, equal to 1% of the

control group mean. A 95% confidence interval for the treatment effect is [−5.5%, 3.6%], againletting us reject substantial changes in value received in either direction.

4.1.3 Leakage

Given that value disbursed increased slightly while value received as unchanged, we do not expect

to find reductions in leakage. Panel C of Table 3 tests this directly. We use a Seemingly Unrelated

Regressions framework with the ration card × month as the unit of analysis and with (i) valuereceived as reported by the household, and (ii) value per ration card disbursed to the corresponding

block as the dependent variables, and then report the difference between the estimated treatment

effects on these variables.41

We estimate that leakage increased insignificantly by Rs. 14 per ration card × month. We canreject large decreases in leakage: a 95% confidence interval is [−10, 38] which lets us reject changesin the share of value lost outside of [−1.7%, 6.5%].

Because the value figures in Table 3 are based on the difference between market and statutory

ration shop prices, they pick up leakage on the quantity margin (e.g. the diversion of food grains)

but may not pick up leakage due to overcharging by the FPS dealer. We examine this separately in

Panel D of Table A.5. Overall, we estimate that the average control group household overpaid by

Rs. 8 for the bundle of commodities it purchased, representing a small share (less than 2%) of total

value received. Treatment reduced overcharging by a statistically insignificant Rs. 2.6. This makes

sense given that the intervention did not directly change marginal (dis)incentives for over-charging.

4.2 Transaction costs

The transaction costs incurred in order to transfer value from the government to beneficiaries

include the government’s cost to store and ship commodities, the FPS dealer’s cost to receive,

store, and retail them, and the beneficiaries cost to travel and collect them.

Using budgetary records, we calculate that the Government of Jharkhand spends an average of

Rs. 127 per ration card per month operating the PDS. The government paid around Rs. 1,600

per month per e-PoS machine to an IT provider inclusive of equipment rental, maintenance, and

training. The average FPS in our data has 257 households, yielding an estimated cost of ePoS

deployment of Rs. 6.2 per ration card per month. While, it is possible that some administrative

costs associated with paper-based record keeping were reduced (including time taken to do so),

range of values but then subsequently answered “no” for at least one higher value, likely reflecting confusion aboutthe nature of exercise. We believe that the lowest “yes” coding is the most reasonable way to interpret the data, butclearly they should be treated with caution.

41This approach lets us take advantage of the fact that we observe the universe of disbursements while also exploitingpotential efficiency gains due to covariance in the error terms in the two equations.

19

these savings were not reported in any official spending records. Thus, we treat the costs of ePoS

deployment as the change in administrative cost in treatment areas, which was a 5% increase.

Using dealer survey data, we estimate that PDS dealers spend Rs. 7 per ration card × monthon the direct costs of transporting and storing PDS commodities. This is a lower bound on the

total cost of distributing PDS commodities as it does not reflect costs that are shared between PDS

and other activities – for example, rent paid by a shopkeeper who uses space both to distribute

PDS commodities and to retail other commercial products. With that caveat in mind, we reject

economically meaningful treatment effects on the portion of dealer costs we do observe (Table 5,

Columns 2-3), which is what we would expect given the lack of an impact on quantities.

Using household survey data, we estimate that the average control group household spent the

monetary equivalent of Rs. 41, or 9% of mean value received, in order to collect its benefits in

March 2017. We calculate this using information on the individual trips they took to the ration

shop, whether each trip succeeded, the time each trip took, and any money costs incurred (e.g.

bus fare), as well as information on the opportunity cost of time of the household member who

made the trip.42 Treatment increased these transaction costs by a small but significant amount:

Rs. 7, or around 1.5% of value received (Table 5, Column 1). In Table A.6 we examine impacts

on the variables that feed into our total cost measure; the cost increase appears to be due to (i)

a significant increase in the number of trips that were unsuccessful in the sense that they did not

result in any purchases, which more than doubled from 0.1 per household per month to 0.23, and

(ii) an increase in the opportunity cost of time of the household member who collected benefits,

consistent with the idea that the reform reduced households’ flexibility to send whoever could be

spared from other work.

4.3 Food security

Given the results above, we would not expect to see impacts on food security outcomes.43 Table 6

confirms this. We examine two measures of a household’s food security: a food consumption score

that follows standard World Food Program methodology to calculate a nutrient-weighted sum of

the number of times a household consumed items from each of a set of food groups in the last

week, and a simple food diversity score defined as the number of groups from which the household

consumed any items in the past week.44 We see a tightly estimated null effect of treatment, with

42In our survey we asked both for the number of unsuccessful trips made by each individual on the household rosterand for the total number of unsuccessful trips taken. When the latter exceeds the sum of the former we attributethe stated total number of trips to household members in proportion to their stated individual number of trips. Theresults are not sensitive to alternative approaches, e.g. simply using individuals’ trip counts.

43It is possible that there could have been indirect effects on food security if the incentive to register for Aadhaarcreated by treatment affected households’ access to other, non-PDS benefits which also required Aadhaar. At the timeof our follow-up Aadhaar was being used in some form to control access to wage payments under the National RuralEmployment Guarantee Scheme, pension payments, and conditional cash transfers to mothers under the PradhanMantri Matru Vandana Yojana. Note however that any effects through access to these benefits would likely be smallgiven that the difference in Aadhaar registration rates between treatment (96%) and control (92%) was only 4%.

44For more details on these methods including the weights for each food group, which are defined based onthe group’s nutrient density, see http://documents.wfp.org/stellent/groups/public/documents/manual_guide_proced/wfp197216.pdf?_ga=1.115126021.300736218.1470519489

20

http://documents.wfp.org/stellent/groups/public/documents/manual_guide_proced/wfp197216.pdf?_ga=1.115126021.300736218.1470519489http://documents.wfp.org/stellent/groups/public/documents/manual_guide_proced/wfp197216.pdf?_ga=1.115126021.300736218.1470519489

95% confidence intervals expressed in control group standard deviations of [−0.11σ, 0.12σ] and[−0.11σ, 0.09σ] respectively.

4.4 Distributional and heterogeneous effects

The notion that stricter ID requirements should trade of reductions in errors of inclusion (including,

broadly defined, leakage) against increases in errors of exclusion itself suggests that effects are likely

to be heterogeneous: many may be unaffected, or affected only to the extent that transaction costs

change, while the main risk is that some lose access to most or all of their benefits.

The distributional effects of treatment suggest this was the case. Figure 3 plots the CDFs of

value received in the treatment and control groups separately; these track each other closely except

for values close to zero, where there is more mass in the treatment group. The probability that

a treated household received zero value is 2.4 percentage points higher than a control household

(Table 7, Column 1), significant at the 10% level.

For a sharper test, we examine how impacts differed for the household we would expect to be most

likely to lose access to benefits, namely the 23% of households that were “unseeded” at baseline

in the sense that they did not have at least one member whose Aadhaar number had been seeded

to their ration card. Figure A.2 plots the distributions of household income and mean years of

schooling completed for the two most educated household members, separately by seeded status.

Unseeded households tend to have lower values on both metrics, and in both cases a Kolmogorov-

Smirnov test rejects equality of distributions. Unseeded households are also 5% less likely be upper

caste (p < 0.01).

Losses in value received are concentrated among unseeded households. Table 8 reports estimated

treatment effects split by this variable. The reform lowered value received by Rs. 49 per month

for unseeded households, equivalent to 12.6% of the control group mean for this category. This

is significantly different from zero as well as from the mean effect among seeded households. On

the extensive margin, treatment lowered the probability that unseeded households received any

benefit by 10 percentage points, also significantly higher than the (insignificant) impact on seeded

ones. While we cannot of course identify specific households that counterfactually would not have

been excluded, this decrease fully accounts in an arithmetic sense for the overall decrease in the

fraction of households reporting receiving any benefits in a given month. Treatment effects on

stated willingness to accept are also significantly lower for unseeded households, though not in this

case significantly different from zero. Transaction costs, on the other hand, increase the most for

seeded households, consistent with the idea that they are the ones able to continue transacting with

the system, albeit at a higher cost (and that unseeded households may not have bothered making

multiple trips). Overall, this suggests that the reform did cause a significant reduction in value

received for the households least ready for the reform, likely driven by the total loss of benefits of

a subset of these households.

We also examine heterogeneity along several additional pre-specified dimensions, including (i)

characteristics likely to matter for understanding the distributional and political consequences of

21

the reform such as caste, education level, and income level, and (ii) characteristics of the location

likely to predict heterogeneity in the implementation of the reform such as rural status, cellular

network signal strength, and the device mode (online, partially online, or offline).45 Note that to

examine heterogeneity by machine mode we need to infer the mode that a given control FPS would

have been using had it been treated at the time. We do so by assuming that it would have used

the same machine type it was ultimately assigned to use once the control blocks were treated.46

In general we find limited evidence of heterogeneity along these dimensions (Tables A.7, A.8, and

A.9.). There is some evidence that wealthier and better-educated households receive differentially

more value and that wealthier households incur larger increases in transaction costs.

4.5 Stakeholder preferences and perceptions

We next examine beneficiaries’ and dealers’ stated preferences for the reform relative to the status

quo. These preferences provide both a cross-check on any inferences about well-being we draw from

the experimental estimates, and insight into the longer-term political viability of the reform.

Overall, views on the reform were sharply divided (Table 9). Fifty-three percent of households

and 51% of dealers preferred the reform to the status quo method of authenticating. Even among

unseeded households, which were most hurt as a group, 50% of households prefer the reform to the

status quo. Views are quite polarized, with 89% (87%) of households (dealers) holding a strong

as opposed to a weak view one way or the other. One interpretation is that respondents view the

question as being as much about their political allegiance as ab