FROM EVIDENCE TO ACTION: The Story of Cash Transfers & Impact Evaluation in Sub-Saharan Africa

Ms. Jenn Yablonski, UNICEFDr. Sudhanshu Handa, University of North

CarolinaOn behalf of all the editors & authors

8th SPIAC-B MeetingSeptember 22, 2016

I. Introduction to the Transfer Project

II. From Evidence to Action: Addressing myths, findings and impacts

Outline

SDG 1.3: Implement nationally appropriate social protection systems & measures for all, including floors, and by 2030 achieve substantial coverage of the poor & the vulnerable.

Wide range of social & economic

outcomes

Universal coverage & access to

social protection

Key features of the African ‘Model’ of cash transfers

• Programs tend to be unconditional (or with ‘soft’ conditions)

• Targeting tends to be based on poverty & vulnerability OVC, labor-constrained, elderly

• Important community involvement in targeting process• Payments tend to be manual, ‘pulling’ participants to

pay-points Opportunity to deliver complementary services

Transfer Project countries

Ethiopia, Ghana, Kenya, Lesotho, Malawi, Madagascar, South Africa, Tanzania, Zambia and Zimbabwe

Transfer Project approach

Stage I. Design of Impact Evaluation

Stage II. Implementation & Analysis

Stage III. Use of Results & Dissemination

• Focus on supporting impact evaluations of national programmes & research-policy interface• Close relationship between all national stakeholders

• Impact evaluations as part of broader evidence/learning agendas & policy processes at national & regional level

Transfer Project approach

• Innovative research components: Mixed methods: quantitative, qualitative & local

economy impacts simulation Youth transitions to adulthood & HIV risk (UNICEF &

UNC) Productive impacts, local economy effects FAO (PtoP)

Country Quantitative Qualitative Lewie Other analysis

Ethiopia Non-experimental X X Targeting, payment process

Ghana Non-experimental X X Transfer payments

Kenya Experimental X X Operational effectiveness

Lesotho Experimental X X Rapid appraisal, targeting, costing & fiscal sustainability

Malawi (incl. Mchinji pilot)

Experimental X Not on Mchinji pilot

Targeting, operational effectiveness, transfer payments

South Africa Non-experimental X No Take up rate, targeting

Zambia (CG & MCTG) Experimental Not on

MCTGNot on MCTG

Impact comparisons across programme, targeting

Zimbabwe Non-experimental X XInstitutional capacity assessment rapid assessment, MIS analysis,

process evaluation

Methods used by the Transfer Project

Contribution to strengthening the evidence-based case for promoting social protection

as a poverty reduction instrument

Impact of Transfer Project:Global level

Generation of evidence on the broad range impacts of social cash transfers• Poverty impacts: child & household level• Social impacts: education, access to health, nutrition-sensitive

indicators, food security• Addressing economic & social determinants of HIV risk:

adolescent wellbeing• Building the economic case: economic & productive impacts

at household level; Impacts to beneficiaries & to local economy

Social cash transfers can work in low-income contexts, including SSA; can be affordable; are a worthwhile investment

Impact of Transfer Project:Regional level

• Strong evidence base on impact of cash transfers now available in SSA

• Context-specific design and implementation (home grown models, community participation, unconditional transfers, etc)

• Strengthen evidence base to feed to important regional processes (AU commitments, etc)

• Contribution to changing the discourse: SP as an investment, not a cost

Results from impact evaluations have influenced design of programs and

contributed to strategic policy decisions

Impact of Transfer Project:Country level

• Adjustment to program design & implementation (targeting, transfer size)

• Moving from cash to cash+ (specifically in terms of nutrition, agriculture & HIV/AIDS)- cash is important, but not sufficient

• Contribute to build & strengthen the case for scale-up & expansion:

Impact evaluations instrumental in strengthening reputation of social cash transfer programs, & confidence with which policymakers decide scale up

Economic & productive impacts: addressing concerns regarding dependency & contribution of the poor to inclusive growth

• Ghana – Gov triples transfer size after baseline simulations show level too low

• Kenya – Increase transfer size based on 4-year follow-up; Gov able to respond to criticisms w/ rigorous data

• Lesotho – Scale up after secondary analysis/ large impacts seen on the ground• Malawi – Ghana & Zambia lessons on predictability ensured payments not

skipped• Zimbabwe – Revised tageting system after positive comparison to more mature

programs• Zambia – Massive scale up: Gov contribution jumped from $4m to $35m (2014)

Continues to increase; campaign issue in recent elections

Research informing policy: Examples of scale-up

The highlights

Domain of impact Evidence Food securityAlcohol & tobaccoSubjective well-beingProductive activitySecondary school enrollmentSpending on school inputs (uniforms, shoes, clothes)Health, reduced morbidityHealth, seeking careSpending on healthNutritional statusIncreased fertility

Common criticisms or doubts that we hear on the ground

• Cash will be wasted: Will be spent on alcohol and other bads’• Its just a ‘hand-out’, not used for productive activities,

cannot contribute to development• Causes dependency, laziness• Leads to inflation, disrupts local economy• For child focused grants, increases fertility

Wasted? Across-the-board impacts on food security

Ethiopia SCTP

Ghana LEAP

Kenya CT-OVC

Lesotho CGP

Malawi SCTP

Zambia MCTG

Zambia CGP

ZIM HSCT

Spending on food & quantities consumed

Per capita food expenditures ü ü ü ü ü ü ü üPer capita expenditure, food items ü ü ü ü ü üKilocalories per capita ü üFrequency & diversity of food consumption

Number of meals per day ü ü üDietary diversity/Nutrient rich food ü ü ü ü ü üFood consumption behaviours

Coping strategies adults/children ü ü ü üFood insecurity access scale ü ü ü

Green check marks represent significant impact, black are insignificant and empty is indicator not collected

Wasted? No evidence cash is ‘wasted’ on alcohol & tobacco• Alcohol/tobacco represent 1% of budget share• Across 7 countries, no positive impacts found on

alcohol/tobacco• Data comes from detailed consumption modules covering

over 250 individual items• In Lesotho negative impacts on alcohol consumption

(possible decrease through decrease in poverty-related stress?)• Alternative measurement approaches yield same result:• “Has alcohol consumption increased in this community over the last

year?”• “Is alcohol consumption a problem in your community?”

Claim: Its just a ‘hand-out’, not used for productive activities, cannot contribute to development

School enrollment impacts (secondary age children): Same range as those from CCTs in Latin America

02468

101214161820

8

3

7 8

15

8 9

12

6

9

6

10

Primary enrollment already high, impacts at secondary level. Ethiopia is all children age 6-16.Bars represent percentage point impacts

Significant increase in share of households who spend on school-age children’s uniforms, shoes and other clothing

Ghana (LEAP) Lesotho (CGP) Malawi (SCTP) Zambia (MCTG) Zambia (CGP) Zim (HSCT) small hh

Zim (HSCT) large hh

0

5

10

15

20

25

30

35

11

26

30

23

32

11

5

Solid bars represent significant impact, shaded not significant. Lesotho includes shoes and school uniforms only, Ghana is schooling expenditures for ages 13-17. Other countries are shoes, change of clothes, blanket

ages 5-17.

Percentage point increase

Grade 3 math test – Serenje District, ZambiaMore kids in school but school quality still a challenge

Productive impacts positive but vary by transfer size, other operational features

Zambia Ethiopia Malawi ZIM Lesotho Kenya Ghana

Agricultural inputs +++ +++ +++ NS ++ NS +++

Agricultural tools +++ +++ +++ +++ NS NS NS

Agricultural production

+++ +++ +++ NS ++ NS NS

Livestock ownership

+++ +++ +++ +++ ++ Small hhlds

NS

Non farm enterprise +++ NS +++ +++ NS +FHH NS

Stronger impact Mixed impact Less impactNS=not significant+++=positive, significant---=negative, significant

Claim: Leads to lazinessWe find reduction in casual wage labor, shift to on farm and more productive activities

Zambia Kenya Malawi Ethiopia ZIM Lesotho Ghana

Agricultural/casual wage labor

- - - - - - - - - --- --- -- NS

Family farm +++ +++ NS +++ NS NS +++

Non farm business (NFE)

+++ +++ ++ NS NS NS NS

Non agricultural wage labor

+++ NS +++ NS NS NS

Shift from casual wage labor to family business consistently reported in qualitative fieldwork

Claim: Leads to laziness

“I used to be a slave to ganyu (labour) but now I’m a bit free.” -elderly beneficiary, Malawi

0.0

1.0

2.0

3.0

4D

ensi

ty

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100age

Claim: Lead to inflation, disrupts local economy

• In six countries, tested for inflation in intervention versus control communities using basket of goods• No inflationary effects found

• Why not?• Beneficiaries small share of community, typically 15-20 percent• Poorest households, low purchasing power, don’t buy enough to affect

market prices• Sufficient supply response to meet demand

In fact, cash transfers lead to positive multiplier effects in local economy!!

Kenya (Nyanza)

Ethiopia (Abi_adi)

ZIM Zambia Kenya (Garissa)

Lesotho Ghana Ethiopia (Hintalo)

0

0.5

1

1.5

2

2.5

3

Multiplier: Amount generated in local economy by every $1 transferred

Claim: Cash transfers cannot contribute to developmentMultiplier effects of cash transfers in Zambia & Malawi Zambia (ZMK) Malawi (MK)

MCP CGP SCTPAnnual value of transfer (A) 720 720 26,169Savings 33 41 381Loan repayment 3 2 916Consumption 1021 767 41,520Livestock & productive assets 138 66 124Non agricultural assets 163Total spending (consumption + spending) (B) 1195 876 44,282

Estimated multiplier (B/A) 1.66 1.22 1.69

Impacts are based on econometric results and averaged across all follow-up surveys.Estimates for productive tools and livestock derived by multiplying average increase (numbers) by market price. Only statistically significant impacts are considered.

Claim: In child focused programs, increases fertilityEvidence suggests the opposite, if anything • Zambia Child Grant Programme

No impacts on total fertility or whether currently pregnant Some indication of improved birth outcomes (fewer pregnancy

complications)

Kenya Cash Transfer for Orphans & Vulnerable ChildrenReduction in early pregnancy among young women age 15-24 by 6 ppNo increase in number of children living in household

• South Africa Child Support Grant Reduction in early pregnancy by 11 percentage points

Where is evidence the weakest in terms of impact?Young child health and morbidityRegular impacts on morbidity, but less consistency on care seeking

Ghana LEAP

Kenya CT-OVC

Lesotho CGP

Malawi SCTP

Zambia CGP

Zimbabwe HSCT

Proportion of children who suffered from an illness/Frequency of illnesses ü ü ü ü ü üPreventive care ü ü üCurative care ü ü ü üEnrollment into the National Health Insurance Scheme üVitamin A supplementation ü

Supply of services typically much lower than for education sector.More consistent impacts on health expenditure (increases)

Green check marks represent positive protective impacts, black are insignificant and red is risk factor impact. Empty is indicator not collected

Where is evidence the weakest in terms of impact?No impacts on young child nutritional status (anthropometry)• Evidence based on Kenya CT-OVC, South Africa CSG, Zambia CGP,

Malawi SCTP, Zimbabwe HSCT However, Zambia CGP 13pp increase in IYCF 6-24 months

• Some heterogeneous impacts If mother has higher education (Zambia CGP and South Africa CSG) or if protected water source in home (Zambia CGP)

• Possible explanations…Determinants of nutrition complex - involve care, sanitation, water, disease environment & food

Weak health infrastructure in deep rural areasFew children 0-59 months in typical OVC or labor-constrained household

Meanwhile, emerging evidence that transfers enable safe-transition of adolescents into adulthood: Impacts on sexual debut among youth

Kenya (N=1,443) Malawi (N=1684) Zimbabwe (N=787) South Africa, girls (N = 440)

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

36%

27%

17%

11%

44%

32%28%

Treat Control

-6 pp impact**

-7 pp impact**

-13 pp impact***

Kenya and Zimbabwe impacts driven by girls, Malawi driven by boys. Zambia no impacts.

-11 pp impact***

How to make cash work better? Impacts depend on transfer size‘Rule of thumb’ of 20 percent of consumption

Ghana 2010

Kenya CT-OVC

(big)

Burkina TASAF 2012

Kenya CT-OVC

RSA CSG

Malawi 2014

Lesotho CGP

(2010)

Ghana 2015

Kenya CT-OVC (small)

Zim (HSCT)

Zambia CGP

Zambia MCP

Malawi 2007

0

5

10

15

20

25

30

35

40Widespread impact

Selective impact

% o

r per

cap

ita c

onsu

mpti

on

How to make cash work better? Transfers must be predictable and regular!

Regular and predictable transfers facilitate planning, consumption smoothing and investment

Sep-10

Nov-10

Jan-1

1

Mar-11

May-11

Jul-1

1

Sep-11

Nov-11

Jan-1

2

Mar-12

May-12

Jul-1

2

Sep-12

0

1

Zambia CGP

# of

pay

men

ts

Apr-10

Jun-1

0

Aug-10

Oct-10

Dec-10

Feb-11

Apr-11

Jun-1

1

Aug-11

Oct-11

Dec-11

Feb-12

Apr-12

0

1

2

3

4

5

6

Ghana LEAP

# of

pay

men

ts

Regular and predictableLumpy and irregular

From Evidence to Action: Key messages

• Social cash transfers can be transformative for children, families & communities

Wide range of impacts across many social & economic domains — but depends on implementation & other factors (cash transfers are not a magic bullet)

• Impact evaluations have helped build credibility of social protection sector in SSA

• Evidence debunks myths, e.g. cash transfers do not create dependency

• Transfer Project utilizes an innovative approach: the ‘how’ matters• Evaluations & learning not only to ‘assess’ results, but to inform national policy &

progressively strengthen program design & implementation

THANK YOU!Website: www.cpc.unc.edu/projects/transfer

Facebook: https://www.facebook.com/TransferProject

Twitter: @TransferProjct

0.0

1.0

2.0

3.0

4D

ensi

ty

0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100age

0.0

1.0

2.0

3.0

4D

ensi

ty0 20 40 60 80 100

age

0.0

1.0

2.0

3.0

4D

ensi

ty

0 20 40 60 80age at baseline

Zambia SCT (Monze Evaluation)

0.0

2.0

4.0

6D

ensi

ty

0 20 40 60 80 100Age in Wave 1

Kenya CT-OVC

Malawi SCT Zimbabwe HSCT

Unique demographic structure of recipient householdsin OVC and labor-constrained models (missing prime-ages)

How much do programs pay? Benefit structure and level in selected programs (US$)

# members Ghana LEAP

Malawi SCT

MOZ PSA

Zimbabwe HSCT

Kenya CT-OVC

Zambia SCT

1 person 8 2.83 7 10 15 flat 12 flat

2 9.50 3.66 9 15

3 11 4.83 11 20

4+ 13.25 6.17 13.50 25

Beneficiary consumption pp per day

0.62 0.34 0.50 0.85 0.70 0.30

ZAM: $24 if disabled memberMLW: 0.83 and 1.67 top-up per child in primary and secondary school respectively