DECRG CoursePoverty and Inequality Analysis

Module 6: Poverty Impacts of Policies and Programs

Ex-Ante Evaluation of Policy Reforms Using Behavioral Models

Francisco H. G. Ferreira

I. What are “ex-ante evaluations”?

1. An “ex-ante evaluation” is a comparison of a (counterfactual) prediction of the outcomes of a program, with the (actual) absence of the program.

a. Compare how an ex-post evaluation approximates the counterfactual...

b. With how an ex-ante evaluation does it:

0,1,)0()1(: PCjyEPTiyEPyPyy jiiiiiTi

01)0()1(: PyPyPyPyyii

ssiiiiTi

I. What are “ex-ante evaluations”?

2. In the absence of actual data on program participants, ys must be simulated.

3. Simulation requires a model• Simple arithmetic simulations• Behavioral (partial equilibrium) simulations• General equilibrium models

4. Models may also be structural or reduced-form, provided reduced form does not change under the new policy and one is not interested in the deep structural parameters.

II. What are they for?

1. To provide an estimate (or prediction) of the impact of programs that do not yet exist.

2. To compare predicted impacts (and costs) for various alternative designs (even for an existing program).

3. They are complementary to (not substitute for) ex-post evaluations.

III. Implementation: A Five-Step Process

• Step 1: Identify a well-defined, tractable policy reform question.

• Step 2: Write the simplest economic model able to capture the factors that are likely to determine the agents’ response to the policy reform.

• Step 3: Find a data set that contains reliable information on the variables that need to be included in the model.

• Step 4: Estimate the model on the data set.

• Step 5: Simulate the policy reform using the empirical estimate of the model.

IV. Two applications from developing countries

• Conditional Cash Transfers: Brazil’s Bolsa Escola Program (Bourguignon, Ferreira, Leite WBER 2003)

• Employment Guarantee Scheme in India (Murgai, Ravallion 2005, World Bank Policy Research w.p.)

Bolsa Escola Program in Brazil

(Bourguignon, Ferreira, Leite)

1. The policy reform question

• How would the introduction of a conditional cash transfer perform with respect to its twin stated objectives: the reduction of current and future poverty?

1. Are the school enrollment incentives built into CCTs effective? (Do households change their behavior in response to the program?)

2. What is the impact of the program on current poverty and/or inequality?

1. The policy reform question: The Bolsa Escola Program

• Means-test: income per capita less than R$90 (50% of the 1999 minimum wage)

• Conditionality : 6-15 year-olds must – Be enrolled in school.

– Attend at least 85% of classes.

• Transfer : R$15 per child in school

• Limit : R$45 per household

• Monitoring at the local and federal levels

• Introduced in July 2001. No ex-post evaluations by 2003.

2. The Model

• For simplicity, we make four key simplifying assumptions

– Gloss over debate on who makes the child’s occupational decision.

– Adult behaviour unaffected by child-level variables

– Sibling interactions ignored– Household composition exogenous.

2. The Model

• Child’s occupational choice: (0) Not going to school (paid or unpaid work); (1) Going to school and paid work; (2) Going to school and no paid work

Ui(0) = Zi.0 + 0.(Y-i + yj0) + vi0

Ui(1) = Zi.1 + 1.(Y-i + yj1) + vi1

Ui(2) = Zi.2 + 2.(Y-i + yj2) + vi2

2. The Model

• Child i’s Contribution to Household Income in state j = 0, 1 or 2:

yi0 = wi ; yi1 = Mwi ; yi2 = Dwi

with wi = market earnings:

Log wi = Xi . + m*Ind(Sj=1) + ui

and M = Exp (m)

2. The Model

Household (kid) i chooses the alternative j that yields the highest utility Ui(j):

Ui(0) = Zi.0 + 0.Y-i + 0.wi + vi0

Ui(1) = Zi.1 + 1.Y-i + 1.wi + vi1

Ui(2) = Zi.2 + 2.Y-i + 2.wi + vi2

with 0 = 0 ; 1 = 1M; 2 = 2 D

3. The Data• Pesquisa Nacional por Amostra de Domicílios

(PNAD, 1999).• Approx. 60,000 households.• Representative of country, except for rural areas of

Acre, Amazonas, Pará, Rondônia and Roraima.• Labor status questions asked if age >= 10.• Enrolment (but no attendance) questions.• Reasons to suspect income variables in rural areas

(see FLN, 2003)… but staple hh survey in Brazil.• Bolsa Escola Program not in effect!

4. Estimation (by age)

• If vij are i.i.d. and have a double exponential distribution, then this discrete choice model can be estimated as a multinomial logit:

• Earnings estimation:

2

1000

000

)().().(1

)().().(

jjijiji

jijijiij

wYZExp

wYZExpp

Log wi = Xi . + m*Ind(Sj=1) + ui

4. Descriptive Statistics and Estimation Results

10 11 12 13 14 15 TotalNot going to school 2.6% 2.3% 3.4% 5.9% 8.5% 13.6% 6.1%Going to school and working 8.0% 11.0% 14.0% 18.3% 22.5% 27.1% 16.8%Going to school and not working 89.4% 86.7% 82.6% 75.8% 69.0% 59.3% 77.1%

Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%

Source: PNAD/IBGE 1999 and author's calculation

Table 1: School enrollment and occupation of children by age (10-15 years old)

5. The Simulation Stage

Introducing the State-ConditionalTransfer:

YYifvwYZU

YYifvwTYZU

YMwYifvwYZU

YMwYifvwTYZU

vwYZU

Iiiii

Iiiii

iIiiii

iIiiii

iiii

22122

22122

11111

11111

00100

.)2(

)(.)2(

.)1(

)(.)1(

.)0(

5. Simulation Results.

Not going to school Going to school and working Going to school and not working TotalNot going to school 60.7% 14.0% 25.3% 6.0%Going to school and working - 97.8% 2.2% 16.9%Going to school and not working - - 100.0% 77.1%

Total 3.7% 17.3% 79.0% 100.0%

Not going to school Going to school and working Going to school and not working TotalNot going to school 41.3% 21.7% 37.0% 8.9%Going to school and working - 98.9% 1.1% 23.1%Going to school and not working - - 100.0% 68.1%

Total 3.7% 24.7% 71.6% 100.0%Source: PNAD/IBGE 1999 and author's calculation

All Households

Poor Households

Table 6: Simulated effect of Bolsa Escola on schooling and working status (all children 10-15 years old)

•40% currently not enrolled would have the incentive to change status and enroll•Impact on children currently working is smaller•Impacts stronger for the poor (means test)

Original Bolsa escola's program Scenario 1 Scenario 2 Scenario 3 Scenario 4 Scenario 5Not going to school 6.0% 3.7% 2.9% 2.2% 2.8% 3.2% 6.0%Going to school and working 16.9% 17.3% 17.4% 17.4% 17.4% 17.5% 16.8%Going to school and not working 77.1% 79.0% 79.7% 80.3% 79.8% 79.3% 77.2%

Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%

Original Bolsa escola's program Scenario 1 Scenario 2 Scenario 3 Scenario 4 Scenario 5Not going to school 8.9% 3.7% 1.9% 0.6% 1.8% 3.6% 8.9%Going to school and working 23.1% 24.7% 25.1% 25.4% 25.2% 24.9% 23.0%Going to school and not working 68.1% 71.6% 72.9% 74.0% 73.0% 71.4% 68.2%

Total 100.0% 100.0% 100.0% 100.0% 100.0% 100.0% 100.0%Source: PNAD/IBGE 1999 and author's calculation

note: Scenario 1: transfer equal R$30, maximum per household R$90 and means test R$90 Scenario 2: transfer equal R$60, maximum per household R$180 and means test R$90 Scenario 3: diferent values for each age, no household ceiling and means test R$90 Scenario 4: transfer equal R$15, maximum per household R$45 and means test R$120 Scenario 5: Bolsa escola without conditionality

Table 7: Simulated effect on schooling and working status of alternative specifications of conditional cash transfer program (all children 10-15 years old)All Households

Poor Households

•No conditionality vs. Bolsa Escola: conditionality is key •Impact quite sensitive to changes in T, the transfer amount•Impact less sensitive to changes in the means test Y0

Original Bolsa escola's program Scenario 1 Scenario 2 Scenario 3 Scenario 4 Scenario 5Mean Income per capita 254.2 255.4 256.5 258.8 256.4 255.6 255.3

Inequality measuresGini coefficient 0.591 0.586 0.581 0.570 0.581 0.585 0.586Mean of logarithmic deviation 0.692 0.659 0.636 0.601 0.639 0.658 0.660Theil index 0.704 0.693 0.682 0.663 0.684 0.691 0.693Square coeffcient of variation 1.591 1.573 1.556 1.522 1.558 1.570 1.574

Poverty measuresPoverty headcount 30.1% 28.8% 27.5% 24.6% 27.7% 28.8% 28.9%Poverty gap 13.2% 11.9% 10.8% 8.8% 10.9% 11.9% 12.0%Total square deviation from poverty line 7.9% 6.8% 5.9% 4.6% 6.0% 6.8% 6.8%

Annual cost of the program (million Reais) 2076 4201 8487 3905 2549 2009Source: PNAD/IBGE 1999 and author's calculation

note: Scenario 1: transfer equal R$30, maximum per household R$90 and means test R$90 Scenario 2: transfer equal R$60, maximum per household R$180 and means test R$90 Scenario 3: diferent values for each age, no household ceiling and means test R$90 Scenario 4: transfer equal R$15, maximum per household R$45 and means test R$120 Scenario 5: Bolsa escola without conditionality

Table 8. Simulated distributional effect of alternative specifications of the conditional cash transfer program

Conclusions

1. Bolsa Escola was rather effective in inducing additional enrollment (60% of poor kids out of school enroll in simulation)

2. Role of conditionality is key: substitution rather than income effect.

3. Likely to INCREASE number of children studying and working. (School duration?)

4. The program’s impact on current poverty and inequality is modest, due largely to low transfer amounts.

Conclusions (ctd.)

4. Impact on poverty appears to be elastic w.r.t. transfer amount, but less w.r.t. level of means-test.

5. Dynamics: See Attanasio, Meghir, Santiago (2005), IFS, and Todd, Wolpin (AER, 2006) for dynamic simulations using Progresa in Mexico.

– Use the randomized experiment to validate dynamic structural models and simulate alternative designs

Guaranteed Living Wage in India

(Murgai, Ravallion)

Setting:

• The agricultural wage rate is known to be a significant predictor of the rural poverty rate.

• Is a “living wage” a feasible anti-poverty program?

– Enforcement problems with minimum wages in an underdeveloped rural economy.

– Government as an employer of last resort?

• In 2005, an Employment Guarantee Scheme was under consideration by the Government of India.

1. Policy questions

How much impact on poverty can be achieved by a guaranteed wage sufficient to reach the poverty line?

Would such a policy be cost effective, relative to an untargeted allocation of the same public expenditure?

2. Model for casual labor

• If employment by the scheme is guaranteed, the EGS effectively establishes a lower bound to the wage distribution.

• Impact would then be found amongst workers whose: – (actual or imputed) wage rate is below the

EGS wage floor, and – who have labor market characteristics

consistent with doing casual labor

•Assume a perfectly inelastic labor demand (matters for predicting program costs.)•Ignore benefits from any assets created by scheme.

*w

EGSw

LD

LS

2. Underlying assumptions:

2. Model for casual labor

(reduced form, parametric)

Wages for casual labor:

Selection into casual labor:

correct for selection through standard Heckman selection correction

• Impute predicted wages for those not working• Account also for the possibility of unemployment in

the absence of the EGS

Probit for unemployment:

Expected wage rate in casual labor:

iii Zw ln

)( ii XP

)ˆ1(ˆ ii Uw )( ii XU

2. Gains from a guaranteed wage rate

• For those already working as casual workers:

• For those induced to switch to casual work by EGS (those with a predicted prob. of participation>1/2):

]0),ˆ1(max[ iiEGS

i Uwwg

]0),ˆ1(ˆmax[).5.0ˆ( iiEGS

ii UwwPIg

2. Impacts on poverty

Post-EGS counterfactual consumption of household j is:

Compare pre- and post-ESG distributions, under different wage schemes.

jn

ijijj HigYY

1

*

),...,,( 21 myyyP ),...,,( **2

*1 myyyP

2. Cost for the Government

• Supply of casual labor

• Cost to the government

s is the scheme’s labor share of the total cost

)0,()0()(1

i

n

ii

EGS giNgIwS

sLwSwwC DEGSEGSEGS /])([)(

3. Data

• Schedule 10, 55th round NSS (1999-00) • 61,000 households; 178,000 adults (15

to 59 years) from the 15 major states • Potential beneficiaries: all adults who

are likely to gain from an increase in casual labor market wages, – directly (as EGS participants), or – indirectly due to an increase in wages paid

in the market as a whole.• The beneficiaries need not be currently

in the labor force: – 65% sample adults in the labor force, 24%

in casual wage labor

4. Simulated impact of 2 ESG wages:

1. Anchor the EGS wage rate to the official poverty line (Daily wage rate per working adult around 40 would allow to reach poverty line)

2. Existing (State-level) minimum wages (adjusted for average working hours) around 50

4. Compare it to alternative anti-poverty scheme:

Uniform (untargeted) allocation of the same gross budget, net of administrative costs.

• Per-capita consumption of each household rises by:

• k share of admin. Cost in the budget• m population size

mwCk EGS /)()1(

5. Results: incidence of gains

• Estimated wage elasticity using different EGS wage rates: overall estimate 0.19

• Individual most likely to gain are:•Those already employed as casual laborers•Those induced to switch likely to be less educated, more likely to be from SC/ST

• Given the self-targeting mechanism: gains larger for the poorest quintile (1/2 likely to have gain > 0)

5. results: simulated absolute gains (ATE) from guaranteed minimum wage

01

23

45

Per

cap

ita g

ain

(Rs/

day)

0 .2 .4 .6 .8 1Per Capita Expenditure Rank

incid_cum_30 incid_cum_40

incid_cum_50 incid_cum_55

5. Minimum wage scheme vs. uniform transfers: % gains largest for the poor

01

02

03

04

0%

ga

in i

n p

er

cap

ita

exp

en

ditu

re

0 .2 .4 .6 .8 1Per Capita Expenditure Rank

propgain_EGS40 propgain_unif40

EGS

Uniform unconditional transfer

5. What about impact on poverty?ESG

Headcount index (%)

Poverty gap Index (x100)

Squared poverty gap index (x100)

Pre-EGS 34.0 7.1 2.2 25 31.9 6.4 1.9 30 29.8 5.8 1.7 35 27.2 5.1 1.5 40 24.6 4.5 1.3 45 22.2 4.1 1.2 50 20.5 3.8 1.1 55 19.1 3.5 1.0

wage around 40:• Cost estimated to be

around 3.7% GDP

wage around 50:• Cost estimated to

be around 5% GDP

5. What about impact on poverty?budget-neutral uniform transfer

EGS wage Headcount Poverty gap Squared rate (Rs/day) index (%) index (x100) poverty gap index (x100) I. Gross EGS cost

25 24.3 4.0 1.0 30 20.7 3.1 0.7 35 16.0 2.2 0.5 40 12.5 1.5 0.3 45 9.4 1.0 0.2 50 7.2 0.7 0.1 55 5.2 0.5 0.1

II. EGS labor cost 25 28.1 5.1 1.4 30 25.8 4.5 1.2 35 23.1 3.7 0.9 40 20.5 3.1 0.7 45 18.1 2.6 0.6 50 16.0 2.2 0.5 55 14.2 1.8 0.4

Impacts on poverty are larger!

ESG better targeted (higher % gains for the poor), but comes with extra costs

• forgone income of participants• non-wage costs government (supervision, materials)

Uniform transfer gives same amount to everybody.Untargeted, but lower efficiency costs

• Simulated the effect of an employment guarantee scheme, under different wage levels, assuming government absorbs all excess supply at the EGS wage.– With rationing, most likely lose spillover effects to non-

participants, assignment more prone to bureaucratic manipulation

• Assumed that work requirement binding for participants. Caveat on local capacity on monitoring

• Cost implications sensitive to assumptions on LD

• Did not consider way program is financed (taxation, cutting other programs). Different welfare implications for the poor.

5. Results: underlying assumptions/caveats

Conclusions

• Ex-ante evaluations: combine micro-econometric estimation of economic models and simulation procedures.

• Tool to predict impact of hypothetical programs, or to simulate the likely effect of alternative program designs.

• Complementary to ex-post evaluations– Need for testing / benchmarking models.