May 6th Presentation OI

8/7/2019 May 6th Presentation OI

1/29

Developing Poverty Targeting andSocial Impact Monitoring Tools for the Opportunity Network

Brian Beard - Opportunity International

SEEP - PAWG MeetingMay 6, 2004


2/29

BackgroundOpportunity International Network:

40 Implementing Partner MFIs in 25countries487,105 clients as of December 31, 200387% womenAverage loan size = $226/ $128 outside EEOperational Sustainability = 106%


3/29

Exit RateClient

SatisfactionMeasures

Poverty

TargetingMeasures ImpactIndicators


4/29

L imitation as an Impact ToolBecause a monitoring system must be keptsimple and cost-effective, its primary valueis as a management tool serves to raisethe red flags and to let us know whichareas need further research

Does NOT replace:Qualitative impact researchIssue-specific impact research


5/29

Poverty Measurement - CIMS4 indicators for Poverty Index to betested/validated by comparing national indicators

Income and Dependency Ratio:Business Income as a proxy for Household IncomeHousehold income/ poverty line or


6/29

Current Research:Collaborating with CGAP and P L P to test CIMS

poverty indicators in 4 countries.In Colombia, the Philippines, Ghana and Albania,

300 clients and non-clients are being interviewedwith a combined tool: CIMS along with theexpenditure and consumption component of theL SMS or national government survey for

assessing poverty.Use data to adjust poverty assessment componentof CIMS, as necessary for each country(weighting and/or indicatorsi.e., EE housing)


7/29

Current Research (continued)Implement certified tool, and further test thevalidity of the poverty assessment indicators inother countries as CIMS is implementedthroughout our Network.In addition to testing relative weights andfinalizing a numerical poverty scale, will also

compare our indicators with standard povertymetrics available from MIS (avg. loan size,median loan size, avg. outstanding loan balance,etc.)


8/29

I. IntroductionWhat were the primary reasons behind the

organization choosing to develop and/or use thetool? For poverty targeting purposes, for measurement of outreach goals, in order to better

track impact over time, a donor requirement,market research, internal mission and vision, etc.?Opportunity International has developed its Client

Impact Monitoring System tool for all of the above

reasons.Mission/Outreach: Our mission stipulates that weaim to help the poorest of the economically activepoor, so this tool will help facilitate depth of

outreach;


9/29

Imp act: The tool includes indicators to measureimpact over time; combining poverty assessment withimpact assessment will hopefully make the tool morecost-effective and useful for management decision-makingD onors: The data we collect will help satisfy thedesires and requirements of our donors, both fromindividual donors and from internationalorganizations;Market Research: The client satisfaction, impact andexit data we collect will provide important marketinginformation to allow us to better serve and keep our present clients and also increase our client base; andT argeting: We hope our poverty indicators willbecome certified by USAID to allow us to fulfill their requirement for poverty targeting.


10/29

What type of poverty distinctions are possiblewith the tool? Does it only distinguishbetween poor and not-poor, or is it able tohelp distinguish between very poor, poor,and not-poor, etc.?

STAY TUNED..


11/29

II. Time and Cost P lease provide an estimate of the timeand cost requirements for:

Staff to implement the tool Management to adapt, oversee, and usedata from the tool Clients time during implementation


12/29

Estimated Cost w/ 6,800 clients

Minutes/ Client 5 20 10 10 2 47

Monthly Salary $105 $105 $141 $141 $37$867

Cost er Client$0.05 $0.22 $0.15 $0.15 $0.08 $0.65

Assumin6,800 clients; 460 interviews ever 4 months,or 1,380 interviews er ear , then

Hours per Qtr. 5 19 10 10 2 46Hours per Year 20 76 40 40 8 184

Cost per Qtr.$25

$101

$67

$67

$36 $29

6Cost per Year $100 $404 $268 $268 $144 $1,184

otal

Decision-Making by

Senior Mgmt.

Data Entry, Analysis &

Reporting by MIS

Sampling byCIMS

Coordinator

Quality Controlby L.O.

supervisor ,etc.

DataCollection byLoan Officer


13/29

I f the tool is not administered as part of routine operations, what is the cost (to the

institution and client) of applying the tool?Does this significantly increase the cost (to the organization and the client) of implementation?CIMS is a monitoring system, and as such

will be part of routine operations. Thus,the cost estimates for implementing it arealready discussed above.


14/29

H ave you had any challenges finding staff and assigning a particular staff member with theright skills to manage, coordinate and implement the whole process? This refers tolocal staff in the organization, not from thesupporting international NGO experts. (SomePA WG participants have noted that they haveobserved a real shortage of qualified staff at thelocal NGO level and find it very hard to sparesome of the qualified staff because they areliterally drawn into every task that is urgent at the time.)


15/29

Y es, we have had difficulties in getting localstaff who are qualified to be able tocontinually oversee the CIMSimplementation without being drawn into

other projects.L ydia Opoku in GhanaMitzi Machado in Colombia


16/29

We are taking the following steps to solvethis dilemma:Insist that each of our partners have abackup CIMS coordinator who is fullytrained in all of the duties required for the

job.Ensure that there is buy-in from the

Board and all levels of management, andthat they are involved in and accountablefor the process.

Create Regional CIMS Coordinator role


17/29

III. Data collection and analysis-Frequency: H ow often do you carry out data

gathering and what is the length or duration of one cycle?We want each loan officer to interviewone client from each Trust Bank (villagebank) at the beginning of every loan

cycle. Alternate clients to reduce fatigue, soeach client in sample interviewed every

other loan cycle, until they leave.


18/29

-Who analyses the data you collect?:Describe this process and its complexity.H ave you had to create or adapt an M I S to deal with this data?

Currently, by BrianBuild local capacityFramework for Quarterly Reporting in Excelformat with up to 8 graphs per tabTemplate in Epi Info while new software is

being developed.


19/29

-Feedback: H ow do you give feedback to theclients and staff? H ow are they involved beyond the data gathering?Regular Management MeetingsPosted on bulletin boards at each branchSent via email to staff Loan officers report to clients at meetings


20/29

-Decision-making: H ow do you integrate the data or results into decision making? Who takes theresponsibility? What role does the management

play and how are they involved (realistically sincethey are busy with day to day operations and other strategic decisions)?Key indices on impact, outreach, satisfaction and exit rate given equal consideration as portfolio

data at management meetingsQuarterly Report I ncentiveMore in-depth analysis by Networkred flags


21/29

IV. Misreporting or data manipulationWhen you administer the tool, is it done

during a routine visit with the client, either at her home or when she would routinely come to the office? I f so, might this timing affect her answers? H ow so?After loan disbursement

Same location as group meetingExtra travel time will undermine efficiency


22/29

I s your tool susceptible to misreporting by respondents (e.g., clients might give ananswer they think you want to hear inorder to get more funding or services rather than the truthful answer)? P lease

provide details on how it is susceptible tothis risk.Y es. Because the loan officer is the

person asking the questions.L earning from ClientsL .O. can sometimes tell if client is lyingInterviewing skill and rapport with clientessential


23/29

What does your organization do to overcome arisk of misreporting? What incentives are in

place to encourage clients to provide accuratedata?

Triangulation on some questions, with loanapplication data from MISMake sure L.O.s explain each time that

answers wont affect future loanSpot check/ QC by supervisor and CIMSCoordinator


24/29

What incentives are in place to encourage accuratecollection of data? Are those collecting the datainvolved with adapting the tool or do they understand how to effectively use the data?H ave you made any overt strides towards

achieving buy-in of either line staff or management in implementing this tool? I f so, doyou believe this has helped in using the tool more effectively or efficiently? I f not, why not?

Spot check/ QC by supervisor and CIMSCoordinator No incentives for Impact or SatisfactionExperience shows clients willing to give negative

feedback, except about loan officer


25/29

H ave you put in place any mechanisms todouble-check the accuracy of the datacollected? For example, do any senior staff spot check how the data is recorded or reported?As mentioned previously, there will beseveral layers of QC, including occasionalspot checks by senior management to look for inconsistencies.


26/29

V. AdaptationI f your tool is being implemented in more than

one country, please provide an overview of how you adapted it between countries, and the time and cost of doing this.

Adapted for Eastern Europe becauseresponses for housing ownership, materialsand utilities questions dont apply there; Also, in terms of impact, adapted in Africa to

find out what is impact of HIV / AI DS awareness, prevention, testing training givenduring weekly meetings;Most adaptation done at regional trainings, but

testing of new questions will incur a cost.


27/29

Did you make any adjustments to your tool since you began to use it to make it cheaper or easier to use? P lease providedetails of why and how this was done.

Centralizing data analysis function makesit cheaper and easier to use at the localPartner level;

Number of indicators reducedChanged sampling plan a few times toreduce cost/time


28/29

I s your tool specific to a microfinance setting or could it be implemented in any type of development program?Majority of questions could either be used, as is,or adapted to another type of development

program.Poverty level indicators

Impact, if another program wants the same kinds of impactExit rateSatisfaction questions could be easily adapted


29/29

Points to Consider Institutionalize the Process Buy-inCost-Benefit considerations for

methodology (sampling, interviewers, etc.)Standardize vs. CustomizeMinimizing bias vs. maximizing learning

Quality Control and IncentivesL imited effectiveness of measuringcommunity and spiritual impact

KISS (time, ease of analysis, utility of data)

Documents

May 6th Presentation OI