OECD DAC Evaluation of Donor Activities in OECD DAC Evaluation of Donor Activities in Support of Conflict-Sensitive Development Support of Conflict-Sensitive Development and Conflict Prevention and Peacebuilding and Conflict Prevention and Peacebuilding

in Sri Lankain Sri LankaA Pilot Test of OECD DAC GuidanceA Pilot Test of OECD DAC Guidance

Presentation to DPSG Presentation to DPSG

October 20, 2009October 20, 2009

The Evaluation Study The Evaluation Study

The Purpose : The Purpose : to collect evidence on the applicability of the draft OECD to collect evidence on the applicability of the draft OECD

guidance that would enable its finalization, guidance that would enable its finalization, to provide targeted advice and support to DAC partners at to provide targeted advice and support to DAC partners at

headquarters and in the field to improve their effectiveness headquarters and in the field to improve their effectiveness and impactand impact

Three outputs :Three outputs :(i)(i) a report that presents the results of the pilot a report that presents the results of the pilot

exercise in Sri Lanka in November 2008, exercise in Sri Lanka in November 2008, (ii)(ii) a lessons learned paper documenting the process a lessons learned paper documenting the process

of conducting the pilot evaluation, and of conducting the pilot evaluation, and (iii)(iii) edited comments on the OECD DAC Guidance.edited comments on the OECD DAC Guidance.

Areas of focusAreas of focus Initial TOR ambitious : so narrowed focus and based evaluation on large Initial TOR ambitious : so narrowed focus and based evaluation on large

evidence base of evidence base of published published strategies and evaluations strategies and evaluations 17 strategies from 10 donors 17 strategies from 10 donors 28 evaluations from 13 donors28 evaluations from 13 donors

Excluded track 1, political/ diplomacy, security, humanitarianExcluded track 1, political/ diplomacy, security, humanitarian No independent baseline or conflict analysis: used SCA1+2No independent baseline or conflict analysis: used SCA1+2 Looked at Relevance (Strategies), Results (Evaluations), Process Looked at Relevance (Strategies), Results (Evaluations), Process

(Coordination)(Coordination) Three phases covered: Three phases covered:

Pre-Cease Fire Agreement periodPre-Cease Fire Agreement period 2002-2005 – CFA period2002-2005 – CFA period 2005 on – new govt, war situation2005 on – new govt, war situation

Target groups (national, conflict-affected, and special groups: journalists, Target groups (national, conflict-affected, and special groups: journalists, police etc)police etc)

TimeframeTimeframe Issues Paper Issues Paper January 2008January 2008 Original TOR Original TOR April 2008April 2008 Team recruited Team recruited September 2008September 2008 Inception Inception October 2008October 2008 Main mission Main mission November 2008November 2008 First Draft First Draft February 2009February 2009 Final Draft Final Draft June 2009June 2009

TeamTeamNick Chapman Team Leader, Development Evaluation Specialist

Debi Duncan, Conflict and Peacebuilding Specialist

David Timberman, Governance and Human Rights

Kanaka Abeygunawardana, Local Facilitation

Context Context PovertyPoverty: SL lower-middle income status but poverty reduction uneven. : SL lower-middle income status but poverty reduction uneven.

2004 tsunami worsened poverty levels in the affected areas, and the N & E 2004 tsunami worsened poverty levels in the affected areas, and the N & E much worse than the rest of the countrymuch worse than the rest of the country

ConflictConflict: rooted in : rooted in failure to institutionalise democratic politicsfailure to institutionalise democratic politics not in ethnic not in ethnic differences (SCA2) and also differences (SCA2) and also ‘political culture, the institutional framework of ‘political culture, the institutional framework of policy, uneven development patterns, and competing nationalisms’policy, uneven development patterns, and competing nationalisms’

Development assistance Development assistance ADB, World Bank and Japan account for 60% of aid (2002-07) but have no ADB, World Bank and Japan account for 60% of aid (2002-07) but have no

mandate to work on political / governance issues mandate to work on political / governance issues Bilaterals are either exiting or reducing their programmesBilaterals are either exiting or reducing their programmes Newer partners have emerged some with more pro-government stance ~ China, Newer partners have emerged some with more pro-government stance ~ China,

India, Iran and Pakistan. India, Iran and Pakistan.

Increased emphasis on global security and terrorism, but tackling sensitive Increased emphasis on global security and terrorism, but tackling sensitive issues is difficult with little financial leverage and a strong (now ‘victorious’) issues is difficult with little financial leverage and a strong (now ‘victorious’) governmentgovernment

Fragile state thinking relatively new (DAC principles 2005)Fragile state thinking relatively new (DAC principles 2005)

Strategies included Strategies included Development

PartnerStrategy

1. Ausaid Development Cooperation Regional Framework 2003-07

2. ADB Country Strategy and Program 2002-04Country Strategy and Program 2004-08Country Partnership Strategy 2009-11

3. EC Cooperation Strategy 2002-06 Multi Annual Indicative Programme 2007 –10

4. Japan Country Assistance Program 2004

5. The Netherlands Multi Annual Strategic Plan 2005-08Multi Annual Strategic Plan 2009-11

6. Switzerland Medium Term Plan for Human Security 2007-09

7. Sweden Country Strategy 2003-07Country Strategy 2008-10

8. UN / UNDP Development Assistance Framework 2002-06 UNDP Country Cooperation Framework 2002-06

9. USA Country Strategy Plan 2003-07

10. World Bank Country Assistance Strategy 2003-06Country Assistance Strategy 2009-12

Four other countries (UK, Germany, CIDA, Norway) were unable to share their strategies.

Strategies ~ FindingsStrategies ~ Findings Many strategies promoted “peace,” and some provided support for the peace

process. Only a few explicitly addressed the root causes of the conflict Few strategies were based on in-depth or recurring conflict analysis Liberal use of ‘peacebuilding’ and ‘peace dividend’. But no serious consideration of

whether a “peace dividend” could change the attitudes of hardliners Most focus on ‘costs’ not ‘causes’ of conflict. So less attention paid to power

sharing, the political system and problems of injustice and impunity Little recognition of political risks (such as delivering aid through a party to the

conflict or supporting the agenda of a government that represented only a portion of the political spectrum and was vulnerable to electoral defeat).

Over-emphasis of the extent to which civil society and citizens could bring about transformation and peacebuilding.

Increasing use of scenarios in strategies Whole of government approach an important strategic approach ~ but difficult to

evaluate A weak approach to conflict sensitivity in early strategies, but this aspect was more

explicit in later strategies

Theories of changeTheories of change Theories of change are not explicit in strategies, though several have

implicit causal logic linking proposed actions and the achievement of outcomes

The most common involve:The most common involve: Community reintegration and grassroots mobilisation building a culture of peaceCommunity reintegration and grassroots mobilisation building a culture of peace Meeting basic needs and improving economic conditions leads to poverty Meeting basic needs and improving economic conditions leads to poverty

reduction and a ‘peace dividend’ reduction and a ‘peace dividend’ Reintegration of displaced people to live in relative harmony with their Reintegration of displaced people to live in relative harmony with their

neighboursneighbours, will contribute to security and economic recovery, will contribute to security and economic recovery Peace is secured by establishing stable/reliable institutions that guarantee Peace is secured by establishing stable/reliable institutions that guarantee

democracy, equity, justice, and fair allocation of resourcesdemocracy, equity, justice, and fair allocation of resources Promote peace by Promote peace by mobilisingmobilising grassroots groups to either oppose war or to grassroots groups to either oppose war or to

changechange public attitudes and build greater tolerance in societypublic attitudes and build greater tolerance in society Economic action (trade sanctions) can alter political commitment to peaceEconomic action (trade sanctions) can alter political commitment to peace

Development and governance projects treat conflict as an external factor & Development and governance projects treat conflict as an external factor & in the post-CFA period, adopted a in the post-CFA period, adopted a post-conflict mind-set post-conflict mind-set that saw them that saw them engage in reconstruction work under the assumption that improved socio- engage in reconstruction work under the assumption that improved socio- economic outcomes would support the transition to peace. economic outcomes would support the transition to peace.

From 2005, socio-economic development projects increasingly accepted the From 2005, socio-economic development projects increasingly accepted the need for need for conflict sensitivityconflict sensitivity and “do no harm” principles, and dropped the and “do no harm” principles, and dropped the notion of a “peace dividend” notion of a “peace dividend”

For peacebuilding work, there has been growing concentration on For peacebuilding work, there has been growing concentration on local local initiatives initiatives through development approaches rather than more directly such through development approaches rather than more directly such as on human rights and at the “Track 1” level. Some saw development as on human rights and at the “Track 1” level. Some saw development projects as a way to explore peacebuilding work in a politically sensitive projects as a way to explore peacebuilding work in a politically sensitive environment. environment.

Several projects focus on conflict transformation through inter-ethnic Several projects focus on conflict transformation through inter-ethnic initiatives and initiatives and community peacebuildingcommunity peacebuilding, but little evidence of how they , but little evidence of how they explicitly addressed the driving factors of the conflict. Very few tried to explicitly addressed the driving factors of the conflict. Very few tried to address the “Sinhala south”. address the “Sinhala south”.

Project StrategiesProject Strategies

Results ~ Quality of EvidenceResults ~ Quality of Evidence Many evaluations are premature and impacts are not given time to Many evaluations are premature and impacts are not given time to

emerge. They are more concerned with lessons for future than about emerge. They are more concerned with lessons for future than about impact impact

Many evaluations: Many evaluations: focus on focus on resultsresults rather than outcomes rather than outcomes are based on are based on partialpartial evidence evidence are beset by a are beset by a shifting contextshifting context where project designs are changed as where project designs are changed as

circumstances altercircumstances alter miss miss baselinesbaselines and follow-up surveys and follow-up surveys are affected by both are affected by both natural and political eventsnatural and political events that have disrupted the orderly that have disrupted the orderly

tracking of progress. tracking of progress. contain contain sensitivesensitive findings that limit sharing of findings and subsequent lesson findings that limit sharing of findings and subsequent lesson

learning.learning.

Despite this, important findings emerge around the effective delivery of Despite this, important findings emerge around the effective delivery of benefits especially at the grassroots level; and on how conflict affects benefits especially at the grassroots level; and on how conflict affects project performance. But the centralised nature of politics means local project performance. But the centralised nature of politics means local initiatives rarely have any impact on peace processes.initiatives rarely have any impact on peace processes.

ResultsResults Some peacebuilding evaluations are too Some peacebuilding evaluations are too conceptualconceptual. Some focus more on . Some focus more on

organisational aspects than on the impact of the initiatives. organisational aspects than on the impact of the initiatives. Some peacebuilding programmes have Some peacebuilding programmes have shifted focusshifted focus from conflict from conflict

transformation / co-existence to more classical development work, since transformation / co-existence to more classical development work, since overt peacebuilding activities are not acceptable (and also post-tsunami overt peacebuilding activities are not acceptable (and also post-tsunami needs have stimulated this).needs have stimulated this).

The dilemma of most peacebuilding / conflict transformation work generally The dilemma of most peacebuilding / conflict transformation work generally is the is the relevancerelevance of a peace project when injustice and inequality are not of a peace project when injustice and inequality are not addressed.addressed.

Findings on Findings on gendergender show mixed performance. show mixed performance. Governance and human rightsGovernance and human rights projects generally have been more projects generally have been more

successful at addressing individual and/or highly localized needs than at successful at addressing individual and/or highly localized needs than at promoting broader group-based or systemic changes.promoting broader group-based or systemic changes.

Community-based programmesCommunity-based programmes aimed at building “capacities for peace” aimed at building “capacities for peace” were more successful at community level than in making linkages were more successful at community level than in making linkages nationally. Some evidence that programmes on inter-ethnic issues created nationally. Some evidence that programmes on inter-ethnic issues created space for communities, especially those working with youth.space for communities, especially those working with youth.

Results at grassroots levelResults at grassroots level

Rich evidence baseRich evidence base Many DPs targeted grassroots groups for Many DPs targeted grassroots groups for

either development or PB purposes, with a either development or PB purposes, with a range of results, butrange of results, but Weak linkage to national processesWeak linkage to national processes Weak capacity to do conflict transformationWeak capacity to do conflict transformation Muddled theories of changeMuddled theories of change Small efforts individuallySmall efforts individually

ButBut Positive resultsPositive results

Local capacity built, community relations improved, void filled for civic Local capacity built, community relations improved, void filled for civic participationparticipation

Economic and social assets builtEconomic and social assets built Inter-ethnic trust builtInter-ethnic trust built Maybe collectively donor effort had impact on CPPB, but not yet Maybe collectively donor effort had impact on CPPB, but not yet

evaluatedevaluated

Nevertheless, under conditions where parties to the conflict see the Nevertheless, under conditions where parties to the conflict see the continuation of war as preferable to a negotiated political continuation of war as preferable to a negotiated political settlement….settlement….

explicit peacebuilding measures are not necessarily more effective explicit peacebuilding measures are not necessarily more effective in mitigating conflict than long-term socio-economic investmentsin mitigating conflict than long-term socio-economic investments

Conducting EvaluationsConducting Evaluations Evaluation work in Sri Lanka has Evaluation work in Sri Lanka has limitationslimitations even without conflict issues. even without conflict issues.

Donors do little independent evaluation, mainly using supervision missions, Donors do little independent evaluation, mainly using supervision missions, completion reports or in-house reviewscompletion reports or in-house reviews

Most TORs for Most TORs for socio-economic development evaluationssocio-economic development evaluations don’t call for conflict don’t call for conflict prevention and peacebuilding aspects to be addressed. prevention and peacebuilding aspects to be addressed.

Few evaluations do their own Few evaluations do their own conflict analysis conflict analysis or were able to draw on a or were able to draw on a baselinebaseline against which to gauge impact. against which to gauge impact.

Most evaluations were largely donor-managed exercises with Most evaluations were largely donor-managed exercises with limited limited consultation with Govt. consultation with Govt.

Few examples of Few examples of joint donor evaluationsjoint donor evaluations, and opportunities have been , and opportunities have been overlooked, even where joint-funding occurred.overlooked, even where joint-funding occurred.

A A shortage of consultantsshortage of consultants with the mix of evaluation + conflict skills, and with the mix of evaluation + conflict skills, and shortage of institutional guidance on conflict sensitive evaluationsshortage of institutional guidance on conflict sensitive evaluations

Project Project M&E systemsM&E systems can be biased or affected by conflict setting can be biased or affected by conflict setting Only few examples where there is an explicit use of Only few examples where there is an explicit use of Theories of ChangeTheories of Change The climate of mistrust in Sri Lanka means that information sharing is reduced The climate of mistrust in Sri Lanka means that information sharing is reduced

and the willingness to discuss results and engage in and the willingness to discuss results and engage in lesson learninglesson learning is limited. is limited.

Donor coordinationDonor coordination Coordination hasCoordination has declineddeclined from the relatively strong period around the from the relatively strong period around the

ceasefire. The level of coordination between donors and the GoSL has ceasefire. The level of coordination between donors and the GoSL has become increasingly difficult - and for some pointless.become increasingly difficult - and for some pointless.

For peacebuilding, the Donor Working Group \ Peace Support Group For peacebuilding, the Donor Working Group \ Peace Support Group reduced its scope but set up useful reduced its scope but set up useful sub-committees sub-committees

Mixed reaction: some donors liked the opportunity to pursue themes in sub-Mixed reaction: some donors liked the opportunity to pursue themes in sub-groups, others regard structure as over-elaborate and irrelevant. groups, others regard structure as over-elaborate and irrelevant.

Useful Useful analysisanalysis commissioned that led to better understanding commissioned that led to better understanding Weak Weak policy coherence policy coherence amongst members ~ except in some sub-groupsamongst members ~ except in some sub-groups Limited consideration of Limited consideration of gendergender, of , of viewsviews beyond Colombo, or views of beyond Colombo, or views of other other

partiesparties in conflict beyond the two main ones in conflict beyond the two main ones

As donors have come under increasing criticism, there is a need for As donors have come under increasing criticism, there is a need for stronger coordination, yet DPSG has become weaker. The Trust Fund was stronger coordination, yet DPSG has become weaker. The Trust Fund was not used productively. not used productively.

In donor strategies, coordination has modest importance In donor strategies, coordination has modest importance

RecommendationsRecommendations on Strategies on Strategies1.1. More rigorous use of More rigorous use of conflict and political-economy analysisconflict and political-economy analysis (preferably (preferably

joint) will inform strategic choices joint) will inform strategic choices

2.2. For strategic and programmatic reasons, be clear exactly For strategic and programmatic reasons, be clear exactly which aspects of which aspects of

CPPBCPPB are to be addressed and are to be addressed and what theories underpinwhat theories underpin how interventions how interventions

will make a difference will make a difference

3.3. Look for strategic ways to address the Look for strategic ways to address the root causesroot causes of conflict of conflict

4.4. Careful consideration is needed of what can and cannot be achieved by Careful consideration is needed of what can and cannot be achieved by

offering a ‘offering a ‘peace dividendpeace dividend’. ’.

5.5. More use of More use of scenarios / flexibilityscenarios / flexibility helps strategies to be responsive and to helps strategies to be responsive and to

manage risk manage risk

6.6. Recognise and declare Recognise and declare institutional capacity and comparative advantage to institutional capacity and comparative advantage to

work on CPPBwork on CPPB

7.7. Improve Improve indicators indicators to measure strategic outcomes on conflict, specify how to measure strategic outcomes on conflict, specify how

they will be they will be measuredmeasured and what and what resourcesresources available to collect the data. available to collect the data.

Recommendations on ProjectsRecommendations on Projects1.1. Use Use short-term programmesshort-term programmes on CPPB, provided they have focused, on CPPB, provided they have focused,

specific objectives and a strategy for withdrawal. specific objectives and a strategy for withdrawal.

2.2. Be Be flexibleflexible in choice of partners, in types of peacebuilding support, and in in choice of partners, in types of peacebuilding support, and in funding channel when working on peacebuilding in a volatile conflict funding channel when working on peacebuilding in a volatile conflict settingsetting

3.3. RethinkRethink your programme strategy in response to major shifts in the your programme strategy in response to major shifts in the political environment, don’t carry on as normal or shift a littlepolitical environment, don’t carry on as normal or shift a little

4.4. Better address Better address horizontal inequalitieshorizontal inequalities (between ethnic groups and (between ethnic groups and

geographic regions). geographic regions).

5.5. Build strategic co-ordination across Build strategic co-ordination across different levelsdifferent levels for any future peace for any future peace work (i.e. across Tracks and linking national and local initiatives). work (i.e. across Tracks and linking national and local initiatives).

6.6. Don’t assume that Don’t assume that civil societycivil society can be a major force in support of conflict can be a major force in support of conflict transformationtransformation

7.7. But better to deliver through But better to deliver through CBOsCBOs rather than NGOs in grassroots rather than NGOs in grassroots empowerment and conflict mitigation empowerment and conflict mitigation

8.8. Address Address gendergender aspects better in CPPB work, especially at grassroots aspects better in CPPB work, especially at grassroots

Recommendations on M&ERecommendations on M&E

Require or do a Require or do a conflict analysisconflict analysis

Develop more explicit Develop more explicit theories of changetheories of change

Find good Find good indicatorsindicators at outcome level at outcome level

Use more Use more jointjoint evaluations evaluations

Focus more on Focus more on impactimpact and be prepared to wait and be prepared to wait

Use consultant teams with Use consultant teams with mixed backgrounds mixed backgrounds

Plan in Plan in advanceadvance and be and be flexibleflexible in timing in timing

Allow Allow additional timeadditional time for preparation and expect delays for preparation and expect delays

Recommendations on Recommendations on CoordinationCoordination Address Address leadership gapleadership gap Use the Use the Trust FundTrust Fund more effectively ~ coordinated more effectively ~ coordinated

action and sharing of responsibilities helps donors action and sharing of responsibilities helps donors reach beyond their limitsreach beyond their limits

Do more Do more joint workjoint work for greater buy-in (for example on for greater buy-in (for example on how partners have provided support to NGOs). how partners have provided support to NGOs).

Newer and larger donorsNewer and larger donors must engage more fully so must engage more fully so that coordinated approaches have a real impact on that coordinated approaches have a real impact on the ground. This will require finding areas of mutual the ground. This will require finding areas of mutual interest around do no harm principles, and may interest around do no harm principles, and may preclude wider discussion on more sensitive issues.preclude wider discussion on more sensitive issues.