EPAG Study Team(World Bank / MoGD/ Subah-Belleh Associates)

Making Cents / Youth Economic Opportunities Conference12 September 2012

EPAG Impact Evaluation: Preliminary Midline Results

• Countries: Liberia, Nepal, South Sudan, Afghanistan, Rwanda, Jordan, Laos, Haiti

The Adolescent Girls Initiative

Liberia Rwanda

South Sudan

Afghanistan

NepalHaiti

Jordan

Laos

• Objective: Vocational and entrepreneurship training for young women

• Design Features:– Private sector/NGO training providers competitively

selected to provide market-relevant skills– Vocational Skills for wage or self employment– Life skills training to address girls’ vulnerabilities– Financial literacy and business development– Stipend (Liberia, Rwanda, Afghanistan)– Job placement through performance-based

contracts

The Adolescent Girls Initiative

• Cross-country learning– Core evaluation team works across countries– Reports shared across countries– International workshops at different stages of project

to share lessons learned• Expansion: Potential to influence AGI programs in

new countries• Sustainability: By comparing outcomes, we can learn

what methods work best. These lessons can inform design of future programs

and the scale-up of these pilot projects

Global Perspective of the AGI

• Implemented by Ministry of Gender and Development• Design Features:

– 4 service providers competitively selected (plus 4 more sub-contracted)

– Training delivered in 2 rounds in 9 communities– In Round 1: Job Skills (35%) and Business Development

Services (65%)– Wide variety of job skills areas: painting, hotel/

restaurant work, driving, etc.– Coverage: 2500 girls in Greater Monrovia and Kakata

(1191 in Round 1; 1300 in Round 2) – Six month follow-up period for job placement

AGI in Liberia (EPAG)

• Like other evaluations, IE focuses on outcomes– What is the effect of a specific program on specific

outcomes?• But unlike other evaluations:

1. IE starts before the project begins2. IE compares the beneficiary group with a similar group of

individuals who do NOT receive the project• This method allows us to attribute causality:

– How much better off are beneficiaries because of the program?

– How can we know that the outcomes we see are due to the program, rather than other factors?

Impact Evaluation

• Objective: To measure the impact of the program on the well-being of participants and their families

• Methodology: Randomized selection to treatment and comparison groups

• Data collection using a series of household and individual surveys: – Baseline: 2010– Midline: 2011– Endline: 2012 (in the field now)

Research Design

What we will learn:• Does the program improve the economic well-being of young

women who participate? What is the impact on employment, earnings, investment, savings, borrowing, and lending?

• What is the impact of the program on a wide range of socioeconomic behaviors and outcomes, such as reproductive health, time management, experience of gender-based and other violence, and attitudes toward risk?

• Does the program promote the empowerment of participants, as measured by proxies such as decision-making, aspirations for the future, and control over household resources?

• How do the program impacts vary according to the demographic and personal characteristics of the participants?

Research Questions

Research Design

Sample size2106

Originally recruited

116Never started

training

1273Assigned to

round 1

769Control group

39Re-assigned to

round 1

808Assigned to

round 2

1157Started training

1191Entered round 1

25Very pregnant

(assigned to round 2)

34Started training

1131Completed round 1

Response Rates

Table 1. Response rates for baseline and midline surveys

Midline

Not interviewed Interviewed Total

Baseline

Not interviewed 56 56 112

Interviewed 314 1680 1994

Total 370 1736 2106

• Benchmark the socioeconomic status and demographic characteristics of the young women who would be participating in the EPAG program to provide a basis for comparison in the future

• Validate whether the treatment and control groups are statistically viable comparison groups for the impact evaluation

• Should not be used to make generalizations about adolescent girls and young women in Liberia overall– EPAG girls are more educated than the “average” girl– EPAG girls live in more urban areas than the “average” girl

Baseline Survey

Results: Employment

Baseline Midline0.30

0.35

0.40

0.45

0.50

0.55

0.60

0.65

0.70

0.38

0.45

0.39

0.67

Likelihood of working

Control Treatment

Perc

ent w

ith a

t eas

t 1 IG

A

Notes:• The treatment group in the graph includes only those who actually started the

training.• The overall midline employment rate for the treatment group is 67%. This includes

the Job Skills (52%) and BDS (76%) tracks.

Results: Earnings

Table 2. Impact on Weekly Earnings (LD)

Indicator

Treatment Effect

T-statistic

Number of Observations

(1) Entire sample 947.88 2.66*** 3608

(2) Those who took part in program 1,019.40 2.79*** 3467

(3) Those working 902.56 1.14 1648

(4) Those who took part in program and are working

977.83 1.21 1596

*** significant at 1%

Note: An increase of 1,109 LD per week is roughly equal to 58 USD per month. The average earnings at baseline was 43 USD per month.

Results: Savings

Table 3. Impact on Savings

Treatment Effect T- Statistic

Number of Observations

(1) Any Savings? (Yes/No) 0.458 14.90*** 3591

(2) Amount saved at home (USD) -1.681 0.79 3582

(3) Amount saved at bank (USD) 30.188 7.35*** 3590

(4) Amount saved at credit group( USD) 4.289 1.80* 3608

(5) Amount saved at susu( USD) 6.201 2.09** 3607

(6) Amount saved at Nigerian susu (USD) 4.487 2.97*** 3608

(7) Amount saved total (USD) 44.526 6.81*** 3563* significant at 10%; ** significant at 5%; *** significant at 1%

Note: EPAG participants were given small stipends and a $25 bonus for completing the course.

Results: Self-Confidence

Table 4. CHANGES IN ATTITUDES OVER THE PAST YEAR (only asked at midline)Treatment

effect T- statisticNumber of

observations

(1) I feel more able to work well with people now than a year ago 0.178 4.45*** 1666

(2) I feel more comfortable with who I am now than a year ago 0.243 6.37*** 1670

(3) I feel more in control of my life now than a year ago 0.170 4.08*** 1670

(4) I feel more able to call upon my friends for support than I was a year ago

0.164 3.04*** 1666

(5) I am more able to help my friends now than I was a year ago 0.288 5.98*** 1668

(6) I am more comfortable in situations now with people I do not know than I was a year ago

0.201 4.82*** 1665

(7) I am more outgoing now than I was a year ago 0.167 2.98*** 1650

Note: A higher number indicates more likely to agree or strongly agree with the statement.* significant at 10%; ** significant at 5%; *** significant at 1%

Summary

What have we learned?• Large increases in employment (55% increase)

and earnings (115% increase)• Positive impacts on savings and self-

confidence• Stronger effects for Business Skills trainees

than for Job Skills trainees– But the business skills income is total enterprise

revenue, not earnings or profits.

Summary

Next Steps: • Examine impact on households• Investigate heterogeneous impacts: by

community, type of training, age, etc.• (After completion of endline survey) Look at

longer-term outcomes: do these positive effects persist, grow, or weaken over time?

• Understand profits and incomes of businesses.

Thank you