SEA Change webinar "Monitoring and Evaluation of Climate Change Adaptation Interventions: What can we learn from existing CCA M&E Frameworks and Principles?" - presentation slides

7/30/2019 SEA Change webinar "Monitoring and Evaluation of Climate Change Adaptation Interventions: What can we learn from existing CCA M&E Frameworks and Principles?" - presen

1/34

With financial support provided by theRockefeller Foundation and technical and

logistical support from Pact.


2/34

http://www.seachangecop.org

11th

SEA Change webinar 1st

webinar in the 2013 series!

Monitoring and Evaluation of Climate Change Adaptation:

What can we learn from existing

CCA M&E Frameworks / Principles?

BY DENNIS BOURS, TEAM LEADEROFTHE SEA CHANGE COMMUNITYOF PRACTICE,

15TH

OF FEBRUARY 2013 - 16:00 BANGKOK/JAKARTATIME (ICT / UTC+7), 09:00 LONDON (UTC)
http://www.seachangecop.org/http://www.seachangecop.org/


3/34


11th




What can we learn from existing CCA M&E Frameworks / Principles?


15TH


http://www.seachangecop.org/node/1480
http://www.seachangecop.org/http://www.seachangecop.org/node/1480http://www.seachangecop.org/node/1480http://www.seachangecop.org/node/1480http://www.seachangecop.org/


4/34


11th






15TH


Dennis Bours

([email protected])

Team Leader SEA Change CoP


5/34


11th






15TH


Paul Bacon

([email protected])

M&E Expert SEA Change CoP


6/34


11th






15TH


Logistics of this webinar session

40 to 45 minutes of presentation

20 to 30 minutes of Q&A session

http://www.seachangecop.org/http://www.seachangecop.org/node/1480http://www.seachangecop.org/node/1480http://www.seachangecop.org/


7/34


* Webinar facilities will unmute microphones for

attendees who have raised their hand and would like to

ask their question verbally.

During the Q&A, please click on the

Raise Hand icon

or simply type the question(s) in the

Question window

The Q&A session


8/34


11

th

SEA Change webinar 1

st



What can we learn from existing

CCA M&E Frameworks / Principles?


15TH



9/34


UNDP Proposed Framework for Monitoring Adaptation to

Climate Change (UNDP, 2007-2008)



1
http://www.seachangecop.org/http://www.seachangecop.org/node/139http://www.seachangecop.org/node/140http://www.seachangecop.org/node/140http://www.seachangecop.org/node/139http://www.seachangecop.org/


10/34


UNDP Proposed Framework for Monitoring Adaptation to

Climate Change (UNDP, 2007-2008)



Advantages Disadvantages

Well established framework, used by

UNDP since 2007 (but still unpublished)

Good consideration of particular CCAI

contexts, situations and policy/program

levels

Follows LFA, including MDG goals Indicators provided good specific

examples that can be readily applied

Framework doesnt follow a clear step-by-

step approach (i.e. where the user would

start and finish etc)

Not much mention of possible timeframes

for applying the evaluation framework

Specific Applicability

Focusing on National Adaptation Programmes of Action (NAPAs) under Special Climate

Change Fund (SCCF) and resilience under the Least Developed Countries Fund (LDCF); Could apply to broad, well established development multi/bi-lateral programs in various

sectors that need to consider some candidate indicators for CCAI (although of course

these should be carefully considered and not just cosmetically bolted on)

1


11/34


IDS Sussex GEF DFID: Evaluation of Adaptation to Climate Change from

a Development Perspective (IDS Sussex, GEF-EO, DFID, 2008)



2


12/34


IDS Sussex GEF DFID: Evaluation of Adaptation to Climate Change from

a Development Perspective (IDS Sussex, GEF-EO, DFID, 2008)




Good explanation of adaptation theory

(including maladaptation) and synthesis

of other frameworks

Guides users through a full consideration

process of important aspects of CCAI

Useful pyramid diagram of internationalto household levels with accompanying

candidate indicators for these levels

Actual framework only drops out at the

end of the document, and is not really

clear on the steps to be followed by the

user

Indicators are given for all intervention

levels, but their number is quite limited


The descriptions for community and household levels could be useful for application atthese levels (as these are not seemingly included to such detail in some of the other

frameworks)

2


13/34


Synthesis report on efforts undertaken to monitor and evaluate the

implementation of adaptation projects, policies and programmes and

the costs and effectiveness of completed projects, policies and

programmes, and views on lessons learned, good practices, gaps andneeds (UNFCCC, 2010)

3



14/34


3 Synthesis report on efforts undertaken to monitor and evaluate theimplementation of adaptation projects, policies and programmes and

the costs and effectiveness of completed projects, policies and

programmes, and views on lessons learned, good practices, gaps andneeds (UNFCCC, 2010)


Gives guidance on how to develop and

use adaptation indicators

Recognizes different policy/programlevels

Links to the work of the GEF and UKCIP

on M&E and the UNDP on VRA

Not a distinct framework and no step-by-

step approach towards implementation

Very limited number of high-levelprocess-indicators given

Lessons learned and good practice page

comes to some conclusions that leave

room for misinterpretation, and no real

new lessons learned are introduced


Good to scan through, but ideas and guidance presented can be very broadly applied.



15/34



Tracking progress for effective action: A framework for monitoring

and evaluating adaptation to climate change(GEF Climate-Eval, 2011)4


16/34



Tracking progress for effective action: A framework for monitoring

and evaluating adaptation to climate change(GEF Climate-Eval, 2011)


Strong background document with good

synthesis and consideration of previous

frameworks (including UNDP, UNFCCC,

IDS Sussex)

A good overview of most of the

indicators presented in previous reports Presents a number of guiding questions

to develop a foundation for a CCAI M&E

framework

Useful set of nine national climate

change adaptation frameworks / NAPAs

Not a distinct framework/methodology

with steps as such


An essential pre-requisite read before starting to develop your own CCAI framework

4


17/34


Learning to ADAPT: monitoring and evaluation approaches in CCA

and DRR challenges, gaps and ways forward(SCR, 2011)



5


18/34


Learning to ADAPT: monitoring and evaluation approaches in CCA

and DRR challenges, gaps and ways forward(SCR, 2011)




A lot of conceptual thinking drawn out by

following the framework

Interesting recap of current approaches,

split by M&E methodology

Interesting approach with the ADAPT

principles for generating CCAI indicatorsand developing a CCAI M&E framework

Possibly hard to implement maybe too

broad and time consuming in practice

At times more conceptual than practical

Indicators need to be ADAPT, but no

actual ADAPT candidate indicators are

presented


Could apply parts of the framework to broad, well established development multi/bi-lateral programs in various sectors that need to consider some candidate indicators for

CCAI (similar to the UNDP framework)

5


19/34



IIED Climate Change working paper 1: Tracking adaptation

and measuring development(IIED, 2011)6
http://www.seachangecop.org/http://www.seachangecop.org/node/520http://www.seachangecop.org/node/520http://www.seachangecop.org/http://www.seachangecop.org/


20/34




and measuring development(IIED, 2011)6


21/34




and measuring development(IIED, 2011)


Very readable and clear document, easy

for the reader to get a quick grasp of the

essentials of the framework

Should be relatively easy to apply in

various contexts

Report split into clear background /framingpart and evaluation framework

section

Doesnt cover all aspects of CCAI, reader

would need a good understanding of this

before using the framework

Not much on M&E Learning

No indicators presented, and limited

guidance on the development ofindicators

No step-by-step approach for

implementation of the framework


Preferably programmes that either are at a local or sub-national level, but part of anational strategy. Or programmes that are focusing on a higher level (policy development),

and incorporate the concept of developing activities on lower levels as part of the policy

being developed

6


22/34



Making adaptation count: Concepts and options for monitoring

and evaluation of climate change adaptation (GIZ, WRI, 2011)7


23/34




and evaluation of climate change adaptation (GIZ, WRI, 2011)7

//
http://www.seachangecop.org/http://www.seachangecop.org/node/107http://www.seachangecop.org/node/107http://www.seachangecop.org/http://www.seachangecop.org/http://www.seachangecop.org/


24/34




and evaluation of climate change adaptation (GIZ, WRI, 2011)


Very good attempt at providing a

practical framework after some useful

discussion of M&E for CCAI

Clear, step by step methodology

including Theory of Change component

and good explanations for choosingindicators and establishing baselines

Focused on M&E learning

Mention of integration with existing

indicators and M&E system components

You need a good M&E understanding to

appreciate the document to the full

(not necessarily a disadvantage)

Maybe insufficient discussion on

consideration of M&E levels (i.e. global,

national, sub-national, local, community,household level)


Should be able to be applied to various contexts by experienced M&E practitioners

7

h // h
http://www.seachangecop.org/http://www.seachangecop.org/node/107http://www.seachangecop.org/node/107http://www.seachangecop.org/http://www.seachangecop.org/http://www.seachangecop.org/


25/34




AdaptME Toolkit for monitoring and evaluation of adaptation

activities (UKCIP, 2011)8

h // h
http://www.seachangecop.org/http://www.seachangecop.org/node/116http://www.seachangecop.org/node/519http://www.seachangecop.org/node/519http://www.seachangecop.org/node/116http://www.seachangecop.org/node/116http://www.seachangecop.org/http://www.seachangecop.org/http://www.seachangecop.org/


26/34






htt // h


27/34







10 clear steps following LFA, that should

be easy to trial

LFA example presented and discussed

Differences presented between building

adaptive capacity anddelivering

adaptation action Links to reference documents per topic

Good integration of M&E learning into

the framework

Little explanation or consideration of

CCAI background in the framework

Generic, and not really clear how long

would take to complete the 10 steps for

a CCA situation in reality


A useful step by step approach, following the LFA, to apply once the overall CCAI contexthas been analysed, documented and understood. Its probably the model that is a true

ready to go tool

htt // h


28/34


- A moment to reflect -

http //www seachangecop org
http://www.seachangecop.org/http://www.seachangecop.org/http://www.seachangecop.org/http://www.seachangecop.org/


29/34


Most of these CCAI M&E frameworks / approaches discussed are virtually untested inthe field / for particular sectors, and if they have been used results have often not been

shared

It is difficult to really compare them as they apply to different size CCA scenarios and

though most are multi-level, they apply best to certain program/policy levels They are generally complex and introduce several monitoring & evaluation concepts in

the same framework package

There is little mention of likely implementation timeframes when attempting to apply

them

There is generally not much how to in the frameworks (i.e. the particular steps to

readily apply the framework to the CCAI)

A few mention M&E learning, which is positive, but again this is untested and CCAI bring

particular M&E learning challenges.

Summarizing

http://www seachangecop org


30/34


Key issues involved in evaluating CCAI

1. Different definitions of adaptation

- What is adaptation?

- Decision-process vs. Actions vs. Outcome

- Process oriented adaptation vs. Result oriented adaptation

- Action linked to outcome vs. Action linked to social learning- And what is successful adaptation?

2. Adaptation and adaptive capacity

- Does adaptation lead to increased adaptive capacity?

- Does increased adaptive capacity lead to increased ability to adapt?

- Determinants are diverse and gathering information on these,

measuring change, can be extremely challenging.



31/34



3. Types of adaptation

- Changing natural resource management / agricultural practices

- Building institutions; creating new and strengthening existing

- Launch planning processes

- Raising awareness

- Promoting technology change

- Establishing monitoring / early warning systems

- Empowering people / income generating opportunities for adaptation

- Promoting policy change; establishing new and adjusting existing

- Improving infrastructure (roads, seawalls, water and irrigationsystems)

- Providing social protection

- Assisted climate migration

- Climate-smart disaster management



32/34



4. Successful adaptation

- Effectiveness: achieving objectives

- Flexibility: How far can we adapt?

- Equity: Inequality dimensions to adaptation

- Efficiency: Cost-effectiveness- Legitimacy: Acceptance by all stakeholders

- Sustainability: The wider implications of adaptation, beyond the project

5. Dealing with maladaptation

- What are the risks of vulnerability not reducing, but increasing?

- Processes are not linear and stagnation or reversal might be possible in

the rout to long-term success

- Do short-term successes guarantee long-term successes?



33/34



6. The nature of adaptation

- Long timescales associated with climate change

- The moving baseline presented by climate

- The mix of hazards and opportunities

- Multi-sectoral nature of adaptation, involvement of different partnersat different scales

7. Attribution Decoupling climate change risks from other drivers and

stressors is very difficult. And at times perhaps impossible.

8. Calibration Trying to assess the impact of an adaptation intervention

against the backdrop of changing hazard profiles.



34/34


SEA Change is a Community of Practice that develops a culture of

high quality and rigorous M&E frameworks, approaches and

methodologies for climate change interventions and practices in

Asia, and beyond.

Documents

SEA Change webinar "Monitoring and Evaluation of Climate Change Adaptation Interventions: What can we learn from existing CCA M&E Frameworks and Principles?" - presentation slides