Alexandria HastingsAli Bayraktar

Dalia Zileviciute-MirchandaniKumiko Shibata

Marshiella Pandji

What is the Impact of Bolsa Familia on Children’s Schooling in Brazil?

Outline

● Motivation

● Overview of Bolsa Familia Program

● Existing Research Designs

● Our Proposal

○ Data

○ Identification

○ Methodology

● Expected Results, Persisting Challenges & Further Policy Questions

Motivation

Conditional Cash Transfers:

Poverty reducing transfers with a condition to invest in human capital like education and health

WHY BOLSA FAMILIA?

● Oldest and most influential CCT in South America

● 26% of population (12 million households)

● No impact evaluation process planned

CCT programs are "...the world's favorite new anti-poverty device" (The Economist, 2010)

● Main objectives :

○ Reduce short-term poverty through direct cash transfers

○ Address long-term poverty (by investing in human capital) through CCTs:

■ Health provisions: child vaccination, pre- and post-natal check-ups

■ Educational provision: school attendance (85%)

● Administration:

○ Registration System: Cadastro Único

○ Beneficiaries/transfer information: Portal da Transparência

○ Compliance monitoring: appropriate government ministries

● Method:

○ Voluntary registration

○ Non-randomized, phase-in

○ Municipality-level quotas and prioritization of eligibility criteria

Overview of Bolsa Familia Program

Existing Research Designs

Our Proposal

● Outcomes of interest: children’s schooling (investment in human capital)

○ Likelihood of being enrolled

○ Likelihood of progressing from or repeating the previous grade level

○ Likelihood of dropping out

● Approach: propensity score matching (PSM) combined with difference-in-differences (DiD)

estimator

○ Robust counterfactual (PSM)

○ Allows for individual and time fixed effects (DiD)

○ Improved efficiency (potentially big variation in PSM countered by parametric regression

using DiD)

○ Sufficient sample size to meet data requirement imposed by PSM

Data

● Baseline survey: household- and municipality-level characteristics, possibly from the

national statistics agency or social ministry. Examples (more on Appendix):

○ Household-level:

■ Household size

■ Housing quality index

■ Log of per capita monthly expenditure (food + nonfood)

○ Municipality-level:

■ Percent of population working in agricultural sector

■ Percent of households with access to piped water

■ Number of public schools per capita

● Pre- and post-treatment survey to compare outcomes of interest

● Trends on outcomes of interest, possibly from the education ministry

○ 3-5 years prior to program

Identification

● Unit of analysis: household, conditioned on (voluntary) Cadastro registration to avoid

selection bias

● Time dimension:

○ t = 1 → pre-treatment year

○ t = 2 → post-treatment year

● Treatment group: households that received transfers by t = 2, but not at t = 1

● Control group: households that did not receive transfers at both t = 1 and t = 2

● This identification is possible due to the variation caused by municipality-level quotas and

prioritization of eligibility criteria

★ Very similar households in similar but distinct municipalities are likely to have different

Bolsa Familia recipient status!

Methodology

● Generate propensity score (p-score):

○ Probit/logit regression of treatment dummy (T) on the set of characteristics (Xs)

○ For each observation (household), record the predicted probability of treatment (p-score),

T^i (Stata command: predict)

● Graph the density of p-score for both treatment and control group

● Restrict sample to observations for which there is common support in p-score distribution

● Match using nearest neighbor/radius matching (due to continuous variables)

Schnitzer and Azzari (2013)

Methodology

● Regress:

yit = b0 + b1(T)i + b2(post)t + b3(T*post)it + uityit = outcome of interestTi = treatment dummy variable, 1 if observation gets treated, 0 otherwisepostt = time dummy variable, 1 if observation from period after treatment,

0 otherwise

● b3 = DiD estimator of the program effect. Interpretation:

★ If yit = whether children in household i are enrolled in school at times t, b3 is the average

difference in the likelihood of children being enrolled between the treatment and

control group

● Alternative: probit/logit regression of yit on Ti, postt, and Ti*postt, look for the marginal

effect of b3 (Stata command: dprobit)

Expected Results, Persisting Challenges &Further Policy Questions

● Expected results:

○ Heterogenous impacts with regard to age, gender, location (urban/rural)

○ Estimates that are more precise and less prone to bias due to: i) sample restriction to

the common support area, and ii) parametric fit

● Persisting challenges:

○ Interpretation beyond the common support group

○ Unobserved factors that are time-variant

○ Anticipation effect

○ Attrition problems

○ Time and resources to collect baseline data

● Further policy questions:

○ Long-term effects

○ Other important outcomes, e.g. interaction with quality of supply of schooling

○ Unpacking CCT effects, e.g. is it the cash or the conditions that makes a difference?

References

● Attanasio, Orazio, Emla Fitzsimons, Ana Gomez, Diana Lopez, Costas Meghir, and Alice Mesnard. 2008. “Child Education and Work Choices in the Presence of A Conditional Cash Transfer Programme in Rural Colombia.” Working Paper WP06/13, The Institute for Fiscal Studies, London.

● Bryson, Alex, Richard Dorsett, and Susan Purdon. 2002. “The Use of Propensity Score Matching in the Evaluation of Active Labour Market Policies.” Working Paper No. 4, Department of Work and Pensions, London.

● De Brauw, Alan, Daniel Gilligan, John Hoddinott, and Shalini Roy. “The Impact of Bolsa Familia on Education and Health Outcomes in Brazil.” Presentation by Daniel Gilligan (IFPRI) and Anna Fruttero (LCSHD) at the Second Generation of CCTs Evaluations Conference, The World Bank, October 24, 2011.

● De Brauw, Alan, Daniel O. Gilligan, John Hoddinott, and Shalini Roy. 2015. “The Impact of Bolsa Familia on Schooling.” World Development 70: 303-316.

● Fischer, Greg. “Difference-in-Differences Estimation.” Lecture for EC455: Quantitative Approaches and Policy Analysis at the London School of Economics, Lent Term 2015.

● Fischer, Greg. “Matching Estimators”. Lecture for EC455: Quantitative Approaches and Policy Analysis at the London School of Economics, Lent Term 2015.

● Fiszbein, Ariel, Norbert Schady, Francisco H. G. Ferreira, Margaret Grosh, Niall Keleher, Pedro Olinto, and Emmanuel Skoufias. Conditional Cash Transfers: Reducing Present and Future Poverty. The World Bank: Washington, D.C.

● Schnitzer, Pascale and Carlo Azzari. “Evaluation in Africa RISING.” Presentation at Africa RISING-CSISA Joint Monitoring and Evaluation Meeting, Addis Ababa, Ethiopia, November 11-13, 2013.

AppendixExamples of characteristics assessed to construct propensity score:

De Brauw, et al. (2015)