Intro Slide

April 11, 2013

Participatory

Impact Pathways

Analysis Sophie Alvarez

International Center for

Tropical Agriculture

BIO: Tony Pryor

Tony Pryor

Tony Pryor is the Senior Programming

Advisor at USAID's Bureau for Policy,

Planning and Learning (PPL). He started his

career at USAID in 1983 in energy and

natural resources in Africa, and then joined

the Policy Planning and Coordination Bureau

(PPC), involved with program reforms. From

2001-2010, he was in charge of the

Knowledge Management and Training

portfolios for the International Resources

Group. He returned to USAID in 2011 to

PPL and is responsible for innovation, change

management, and learning in the Office of

Strategic and Program Planning.

The Society for International Development (SID), a not-for-profit

organization founded in Washington, D.C. in 1957, and headquartered

in Rome, is a global forum of individuals and institutions concerned

with sustainable economic, social, and political development. SID-

Washington has a thriving workgroup community to share

development successes, best practices and lessons learned, and to

informally exchange ideas.

BIO: Larry Garber

Larry Garber

Larry Garber is Deputy Assistant

Administrator in the Bureau of Policy

Planning and Learning (PPL) at USAID, where

he supervises the Offices of Science &

Technology and Learning, Evaluation, and

Research. Previously, Garber was detailed to

the Africa Bureau, where he served as the

Agency point person for the Referendum on

Southern Sudan independence. Garber was

also a senior policy-maker for USAID in

Washington, including nine months as the

Acting Assistant Administrator for the Bureau

of Policy and Program Coordination.

BIO: Sophie Alvarez

Sophie Alvarez

Sophie Alvarez is a Monitoring and

Evaluation Consultant at the International

Center for Tropical Agriculture (CIAT),

specializing in Knowledge Management and

Capacity Strengthening . Her work in

agriculture focuses on social network

analysis and participatory research. Ms.

Alvarez has conducted research in planning,

monitoring and evaluation, capacity building,

learning, participatory research, and

knowledge management for projects in

Africa, Asia, and Latin America and the

Caribbean.

Agenda

1. Introductions (to speaker, to seminar)

2. Discussion questions

3. What is PIPA?

4. Utility of PIPA: Some current context of Ag. Research programs

5. Highlight: use of PIPA for planning and M&E of learning

6. Q&A

Discussion Questions

• Are methodologies such as PIPA useful for the inclusion of

learning (programmatic and for next and end users of

research products) in the design and M&E of agricultural

research-for- development interventions?

• What elements are missing / what are the main hurdles to

make a shift to “systems/ outcomes thinking” in R4D

projects?

Change Models

…if you can improve these theories you can

improve the practice, making impact more likely

People plan and implement projects

on the basis of their change models -

their implicit theories about how

the world works…

Methodology

An ex-ante monitoring and

evaluation methodology for

…description and quantification

of results, and for mapping the

potential impact of a project,

program or intervention

… description of theory of change from multiple perspectives,

involving stakeholders of a project in M&E and learning

activities throughout the whole project cycle

How PIPA helps

Making practitioners’ and other stakeholders’ theories

explicit about how they will achieve adoption and impact

(impact pathways, program theory, theory of change);

Contributing to the improvement of these theories;

Preparing models / frameworks for M&E and impact

assessment, for learning and for project and program

“adaptive management”

PIPA helps by:

“Stakeholders' implicit theories are not likely to be systematically and

explicitly articulated, and so it is up to evaluators to help stakeholders

elaborate their ideas.”

Impact Pathways

Two complementary conceptualizations of a project, a program or

an organization’s impact pathways:

A visual description of the causal chain of events and outcomes

that link outputs to the goal (logic model); and

Activities Research Outputs

Research Outcomes

Development Outcomes

Impact

Network maps that

show the evolving

relationships necessary to

achieve the goal District_Assemblies

Ministry_of_Food_and_Ag.

NGOs

Presidential_Special_Initiative_on_Cassava

Savanna_Agricultural_Research_Institute

Seed_Companies

Ultimate_Beneficiaries

Water_Research_Institute

Winrock_International

Foundations – Impact Pathways Matter

Foundations – Project control x time

Foundations – Bennett’s Hierarchy

Workshop Roadmap

Problem

Diagnostics

Problem Tree Graphic

Chart

Falta de sostenibilidad ensistemas intensivos de

producción de cerezas enla Patagonia sur

Bajarentabilidad

Alto riesgo eincertidumbredel sistemaproductivo

Problemas decomercialización

Altos costos,falta de

disponibilidad ycalificación de lamano de obra

Baja y variableproductividad

Baja densidaddemográfica y

competencia conotras actividades

y subsidios

Monocultivo

Altos costos decomercialización

y empaque

Falta dedesarrollo dealternativas

agro-industriales

Contratos tipo

Competencia conotras actividades

y subsidios

Bajadisponibilidad de

técnicoscalificados

Demanda demano de obra

muyconcentrada,

escasacalificación

Baja eficacia enlas distintasetapas delproceso

Insuficientecantidad y

formación derecursoshumanos

Curso deproducción decereza para

técnicos

Capacitaciones aproductores y

operarios

Formación endesarrollo rural

Insuficienteadopción detecnologíasdisponible einsuficiente

desarrollo detecnologíasadaptadas

Transferencia detecnología

inapropiada(variedades)

Insuficientessistemas demonitoreo

Desconocimientode los objetivos

de losproductores

Sistema degeneración y

transferencia detecnologíainapropiado

Diseño eimplementaciónde sistemas de

monitoreo

Identificación delos objetivos delos productores

Nuevo sistema degeneración ytransferenciabasado en laco-innovación

Protocolo paraanálisis de

calidad y puntoscríticos

Informalidad enlas ventas

Bajos volúmenesde

comercialización

Desarrollo dealternativas

agro-industriales

Productoresnucleados en

cámaras yasociaciones

Falta de planesde desarrollo

para actividadesalternativas(políticasactivas)

Falta deinformación

sobre sistemasproductivos

diversificados

Alternativasproductivasrelevadas y

cuantificadas

Degradación delos recursos

naturales

Salinización

Pérdida de labiodiversidad

Erosión eólica

Diseño desistemas demonitoreo

ambientales

Falta deinfraestructura e

incorporacióntecnológica

Políticascrediticias no

adaptadas a laactividad y poca

accesibilidad a lasmismas

Alto costo deinversiones

Desconocimientode la actividad ysus necesidadespor parte de las

entidadesfinancieras

Poca flexibilidaden los requisitosde las entidades

financieras

Estrategias parapromover la

conexión entrelos formuladoresde créditos y losconocedores de

la actividad

Productos

Problemas que el casono tratará

Vision of Success

By 2020 next users will:

- Engage in public, private and civil society

collaboration

- Better target the vulnerable

- Use scientifically proven tools

- Support ex-ante interventions,

particularly government

- Adopt climate resilient technologies

(water storage, water use, inputs, etc.)

- Develop policies that provide incentives

for best management practices (tax

breaks, credit schemes, etc.)

- Have country 5-year plans influenced by

climate smart practices

Network/Stakeholders Mapping

The human eye is an analytic tool of remarkable power, and “eyeballing” pictures of networks is an excellent way to gain an understanding of their structure

Network/Stakeholders Mapping

Network/Stakeholders Mapping

Outcomes “Logic Model”/Table

Final Products

Products Used by groups of actors

Changes in

KAS in that

group

Local extension

workers know how

to use and

communicate the

guidelines

National level

planners can use the

benefit studies to

propose

interventions and

policy

Final Products

Products

Used by groups of

actors

Changes in

KAS in that

group

Local extension

workers know

how to use and

communicate the

guidelines

National level

planners can use

the benefit studies

to propose

interventions and

policy

Changes in

Practice and

Behavior

Local extension

workers using

guidelines

Policy makers

incorporate climate

science into NAPAs

Regulatory agencies

using rapid ID kits at

borders

Development Outcomes

Impact

Improved soils 250.00 improved nutrition

from RTBs in country

Poverty reduction in the millions

Improved nutrition & health in the millions

Final Products

M&E Indicators

Communication of Project IP Logic

Farmers usingdrought probabilitymap and droughttolerant varieties

Improved utility ofcommunity dugouts

Majority ofcommunities in

Northern Ghana haveconstructed and are

using domestic waterharvesting systems

More time forincome

generatingactivities for

women

Improvedincome for rural

households

Improved foodsecurity and

rurallivelihoods

Improved croppingsystems in Northern

Ghana

Farmers plant toavoid crop loss dueto draught, majority

have intensifiedcropping systems

Farmers routinelygenerate organic

matter , e.g.composting andcover cropping

Farmers usingappropriate

tillage methodsto conserve

soil moisture

Changes tohousing

structure tomeet waterharvesting

needs

More wateravailable for

domestic needs

High labourproductivity

High land andwater

productivity

Communitydugouts

efficientlyutilized for fish

production

Effective managementof community water

resources

Adequate watersupply for dry

seasonagriculture

Improved soilfertility

Project Goals

9

8

11

10

11

Drought probabilitymap

Drought tolerantvarieties developed

Crop productionguides or manuals for

MoFA

Best-bet soil and waterconservation and

management optionsmanuals

5

Scaling up

Dugoutsenhanced toretain water

Communitieshave knowledge

of low-costdomestic water-

harvestingsystems

Communitiestrained on

efficient fishproductiontechniques

Manuals on fish culture indugouts and dugout

maintenance

Manuals onappropriate water

harvesting systems

Reduction in waterrelated diseases

4

Methods developed toinstitutionalize dialogueabout water use among

multiple users

7

Crop Related Outputs

Water Related Outputs

Crop Related Outcomes

Water Related Outcomes

Water UsersAssociations formed

and strengthened

Higher cropyields

Wider adoption of project outputs beyondpilot sites

Adoption of project outputs by MoFA forextension after project finishes

National variety releasecommittee releases

varieties

6

Scaling Up

3Project Activities

carried out in PilotSites with boundary

partners andultimate

beneficiaries

1ScalingOut

ScalingOut

Soil and waterconservation improved

in farmlands in N.Ghana

Improvedknowledge ofstakeholdersat pilot sites

Changes instakeholdersattitudes andperceptions

Stakeholdersmodify andinnovate

Adoption oftechnologiesand changesin practice

Iterations oflearning cycle

2

Communication of Project IP Logic

Impact Narrative

Impact Narrative

• Text description of the project impact pathways

• Achieves the integration between the logic and

network models

• Helps with colligation (tracing of logical steps,

Roberts, 1996), making hidden assumptions

explicit

• Helps with the plausibility of ex-ante impact

assessment

Current R4d-CGIAR

IFPRI

CIMMYT

CIAT

CIP

IITA

IWMI

CIFOR

IRRI ICRISAT

WORLD FISH

BIOVERSITY

ICARDA

ILRI

AFRICA RICE

WORLD AGROFORESTRY

CENTRE

CGIAR is a global research partnership for a food secure future. Its science is

carried out by the 15 research centers of the CGIAR Consortium in collaboration

with hundreds of partner organizations

Overarching Themes Graphic

A results-based program management in the CGIAR shifts the

focus from outputs or products into how these products will be

used, and by whom, to ultimately contribute to CGIAR’s overall

goals of poverty and hunger alleviation.

This has implications in the way WE:

•Plan projects and initiatives,

•Implement them, especially regarding clarity on who we have to

work with and who we work for, and strategies to do it,

•Learn & document and report, and

•Measure performance

CGIAR

Generic Impact P/way + Actors in AgR4D

Activities Research Outputs

Research Outcomes

Development Outcomes

Impact

Implementers Next users End users

Final Products

Sustainability

Food

security

Nutrition

& health

IDOs CRP – IDOs System Outputs

Impact- SLOs Research outcomes

Beneficiaries Millions 1000s 10.000s 100,000

Poverty

reduction

CCAFS Theme 1

Impact Pathway

IDOs

Adoption

National policy change

International instl.

change

Local institutional

change

Gender

NBDC Impact Pathway

Emphasis on Learning of Implementers,

Next and End users Of Implementers, Next and End Users

Strategies (of capacity building, knowledge

sharing, communication, extension

(standalone or accompanying products),

knowledge products

Achieve the changes in knowledge

and skills necessary for the practice

changes to happen

Emphasis on Learning of Implementers,

Next and End Users

Research Outcomes

Development Outcomes

Implementers

Next and end users

Capacity change (changes in knowledge,

attitudes, and skills)

Key practice/ behavioral changes,

practice change*

Direct benefits to beneficiaries

Changes in the enabling environment

that underpin or support…..

Emphasis on Learning

Programmatic

• Monitoring and evaluating progress along impact pathways – Regularly revisiting the indicators, outcomes logic model and

network maps

– Use of Most Significant Change to pick up unexpected consequences

– Provides the information needed for “adaptive management”

• Impact Pathways Evaluation = Action research – Is publishable; raises the status of M&E

Cases

Past Work (since 2005)

• CPWF-supported, CIAT-led impact assessment project in 9

river basins ($900,000)

• EU-funded, Wageningen-led “eco-system approach for co-

innovation of farm livelihoods” project

• Phase II of Knowledge Sharing for Research Project (with

Simone Staiger)

• PRGA INIS Project (CIAT and CIP led)

• Use in the Cambio Andino (DFID project) in South America

• Application in several CG centers/ Programs- APRs, MTPs

Lessons Learnt

• The workshop really needs the 3 days to be meaningful- and

the matter of timing.

• Results become more meaningful with identifiable and not

generic actors- thus better in a geographically determined

space, too. It’s key to have clarity on the limits of the system to

which you apply IPs.

• There is also a value in the development of generic IPs for

thinking through theory of change and for adaptation to local

context

• Trying to simplify the complex is complicated

PIPA is useful to…

Provide systematic information to

facilitate learning amongst peers,

decision making and production of

reports during project implementation

Improve integration with sub-projects

or related projects

Establish a common explicit plan and a

feeling of shared purpose among

implementing partners

PIPA is useful for…

Getting the participation of the users of results in determining the form and

content of research and knowledge products

Establishing a framework for examining critical change processes that a

project hopes to initiate and sustain

Involving actors and partnerships needed for a joint process towards achieving

outcomes and impact

Communicating to donors and partners the desired and achieved outcomes

and impact

Cases

Current Work

CIAT-participation CRPs

Convenios Colombia

See www.impactpathways.pbwiki.com

Further Reading

Douthwaite, B., Alvarez, B.S., Cook, S., Davies, R., George, P.,

Howell, J and Mackay, R. 2006. The Impact Pathways

Approach: A Practical Application of Program Theory in

Research-for-Development.

See www.impactpathways.pbwiki.com

https://sites.google.com/a/cpwf.info/m-e-guide/home

Thanks

To: Boru Douthwaite, Graham Thiele, Ronald Mackay

All the projects whose materials we use freely…

For your attention, questions, ideas for improvement

Discussion Questions

• Are methodologies such as PIPA useful for the inclusion of

learning (programmatic and for next and end users of

research products) in the design and M&E of agricultural

research-for- development interventions?

• What elements are missing / what are the main hurdles to

make a shift to “systems/ outcomes thinking” in R4D

projects?

End Slide

Share Feedback

Please take our 3 minute

survey:

http://sgiz.mobi/s3/5df22d18ac16

You can also visit the event page to

post comments & questions.

Thank you for joining us!

Stay In Touch

Contact:

Sophie Alvarez

PPL/LER

SID-Washington

Visit:

USAID’s Learning Lab

SID Website