MEASURING OUTCOMES

1

JANUARY 2020

HOW DO YOU THINK ABOUT IMPACT?

LOOK BEYOND THE OUTREACH

The “What” … So What?

• Did lives improve?• Do the youth get jobs?• Do these new jobs pay more?• Has health improved?• How are lives transformed?

Our mission is to create dignified jobs for the poorest of the

poor by building scalable businesses with investment and consulting support. 4

17 BUSINESSES | 14,500+ JOBS

Inputs

What do you exist to do?

What are the inputs you bring to bear?

BE SYSTEMATIC: USE A LOGIC MODEL

6

Activities

What do you doto carry out your mission?

if / then

Outputs

What happens as a directresult of those activities?

if / then

Outcomes

What happens indirectly, as a longer term consequence of those activities?

What is outside your direct sphere of influence?

if / then

Impact

What outcomes are proven to be attributed to your actions?

What is the causal connection?

if / then

Mission

Our mission is to create dignified jobs for the poorest of the poor by building scalable businesses with investment and consulting support

Activities

Screen and select

Patient equity

Accelerator and deeper support

Outputs

Profitable, self-sustaining and scalable enterprises

New job creation

Outcomes

Stabilized and increased income for jobholders

Jobholder progress out of poverty including:• Improved housing

quality• Improved food

security• Increase in the

number of household assets

UPAYA’S LOGIC MODEL

7

Mission

Our mission is to create dignified jobs for the poorest of the poor by building scalable businesses with investment and consulting support

Activities

Screen and select

Patient equity

Accelerator and deeper support

Outputs

Profitable, self-sustaining and scalable enterprises

New job creation

Outcomes

Stabilized and increased income for jobholders

Jobholder progress out of poverty

UPAYA’S LOGIC MODEL

8

• # of channels• # of applicants• # of businesses accelerated• Amount of capital committed for

investment• # of investments

• Annual revenue growth• % that are profitable• Amount of follow-on capital attracted• # of jobs created• Average % income increase• Change in poverty scores (PPI™)

• % moving to more secure homes• Average % increase in food

expenditure• # and type of new assets procured

after the job

Resulting Metrics

MEASURING THE QUALITY OF A JOB

• Track at household level• Gather all previous income sources • Gather all current income sources• Standardize for same time period (daily)• Do not overlook discontinued income sourcesà Calculate % change in incomeà Calculate what % is from partner business

INCREASE IN INCOME

Rs 200 / week Rs 10,000 / season Rs 1,600 / month

Previous

Rs 1,000 / week

Current

IS INCOME MORE STABLE?

OVERALL JOB SATISFACTION?

ASSETS PURCHASED AFTER HIRE

LONG TERM ASPIRATIONS

10

MEASURING THE QUALITY OF A JOB

11

80% Ave Income Increase($4.15 to $7.50)

MEASURING THE QUALITY OF A JOB

12

This Photo by Unknown Author is licensed under CC BY-NC-SA

MEASURING THE QUALITY OF A JOB

Challenges in Impact Management

13

• Selecting the most appropriate metricsConsider industry standards (IRIS+, Poverty Probability Index, etc.)

• Lack of resources to support IM activitiesFunders must make IM a priorityOrganizations should include costs in their program budgets

• Systems designed for mission-driven organizationsData storageAlignment with industry standardsAnalysis and visualizationReport generation

Tips for Collecting Beneficiary-level Data

14

• Select most critical metricsA number between 15-20 is ideal What indicators can your organization directly / indirectly effect? Can you act on the feedback?

• Design surveyWord questions to prevent biased answer(Ex: Are you satisfied? VS Would you recommend this to your friends?)(Ex: Has this improved your life? VS What are your goals with this new income?)Consider multiple choice (vs free form) à Ex: Asset buckets (comfort, household, productive)Administer within 10 minutes

• Work with enumeratorsSelect third-party, objective groupTraining on questions, flow, and testing very importantProvide interpretation guide to accompany survey à what is the goal for every question?

Designing Metrics: Considerations

15

• What is objective / observable?Ex: Assets / housing condition as indicators of poverty

• Do not discount respondent’s own perceptionsEx: Is your income more stable than before?

• Methods to assess “impact”Track same respondents over timeCompare cohorts of similar respondents (ex: compare by 1-year versus 3-year tenure)Focus group discussions before survey design à create metrics around themes that emerge

16

POVERTY PROBABILITY INDEX (“PPI™”)

0-432% < $1.90/day96% < $3.80/day

45-492% < $1.90/day38% < $3.80/day

80-840% < $1.90/day3% < $3.80/day

95-1000% < $1.90/day0% < $3.80/day

Please visit povertyindex.org

for more information