View
1.533
Download
0
Embed Size (px)
DESCRIPTION
Citation preview
Overlooked Links in the Results Chain
Vinod Thomas
Director-General and Senior Vice President
Xubei Luo
Senior Economist
Independent Evaluation Group
World Bank Group
June 2nd, 2011
When vital links are overlooked…
• Focusing only on relief and rehabilitation and not on prevention costs millions of lost lives and livelihood
• School access without learning hurts millions of children and their success in the marketplace
• Misplaced emphasis on liberalization and self-regulation without regulatory frameworks contributed to the global financial crisis
• Repeating successful water projects in the face of groundwater and coastal zone crises drowns development results
Crucial links in the results chain
►Focus on the right results
►Measure results right
►Use findings creatively to improve results
Focus on the right results
1. The urgent can drive out the important►One-off natural disaster responses limit
capacity for reacting in the future
• Almost half of the countries borrowing from the World Bank for disaster response did not mention disaster prevention in their development plans
►Mitigation and prevention can greatly reduce loss of life and damages
• The benefit-cost ratios of prevention range from 1.5 to 5.7
2. Project- and country-level results differ
► Projects and country programs are rated against different objectives
• No fixed relationship between results frameworks at the project- and country-level
• Outcome objectives at the project level are not the same as those at the country level
► Satisfactory project outcomes do not add up to satisfactory country program outcomes
• Chad-Cameroon pipeline met project objectives but not Chad’s country objectives of poverty reduction and capacity building
3. Results are linked across sectors
► Infrastructure
• Better roads contribute to higher school enrollment and better health outcomes
• Rural electrification improves life quality
► Health
• Mother’s education affects her children’s health
• Poor sanitation and hygiene can wipe out benefits of health projects
► Public-private partnership
• Public and private sector inputs combine to improve crop production
Measure results right
4. Composite indicators may mislead
► Indicators should be built on sound premises
• Emphasizing costs but not benefits of regulations can lead to the wrong prescriptions for regulatory reform
► Clusters and weights can have a critical effect
• Countries with better governance ratings may not gain from a larger weight on governance in IDA resource allocation
► Adding, rating, or ranking can over simplify reality
• Adding indices of different dimensions can obscure actual results
• Rigor can be compromised by rescaling from cardinal to ordinal values
5. Intermediate outcomes do not ensure desired final results
►Program achievement will be hard to assess if the desired results are not spelled out
► The results chain is complex from school enrollment to learning• Tanzania: increase in enrollment vs. decline in secondary
education learning outcomes
• India: increase in teacher attendance vs. same test scores
► The new WB education strategy rightly focuses on accountability and results
6. Averages can mask targeted outcomes
►Rural electrification• Bangladesh: the poorest 40% rural households accounted for 17% of
total electrified rural households
►Nutrition• Ethiopia: free food distribution improved children’s weight-for-height z
score in richer households but not in poorer households
►Microfinance• Philippines: microfinance benefited men and richer entrepreneurs
more than women and the poor
► Social funds and community-driven development• Benin: community contribution requirement created hardship for the
poor
Use findings creatively to improve results
7. Applying lessons for the future
► Repeating successful projects may not be enough in a changing environment• Water accessibility vs. water scarcity concerns
– China: large-scale irrigation projects
• Road investment vs. environmental concerns – Bangladesh: three-wheeled taxis
► Innovative strategies are required to meet future needs• Water: coastal zone management, pollution reduction, and
groundwater conservation
• Transport: programmatic, cross-cutting, and multi-sectoral approaches
8. Missed opportunities
► Resolve apparent conflicts among policy objectives• Energy subsidies
• Protected forest areas
► Overcome institutional constraints and limitations• MIGA’s Convention on risk insurance
• WBG’s safeguard policies
► Reduce information constraints• Cost-Benefit Analysis
9. Timing can make a big difference
►Learn faster what works and focus on
results at the right time
• Mexico: evaluation of Progresa, Oportunidades
• Philippines: early childhood development program
evaluation
►Translate evaluative lessons into
development results • Tailored messages in various formats to reach the
audience
• Collaboration with clients and stakeholders
Conclusions
► Enormous value can be added when evaluators recognize and emphasize crucial factors
► Pay-offs can be high when evaluation frameworks encourage innovation and risk taking and when findings and recommendations are followed up
Thank you!
Improving Development Results Through Excellence in Evaluation
www.ieg.worldbankgroup.org