Evaluation Unit EuropeAid

Martyn Pennington

Head of Evaluation Unit- Devco B2

Workshop on Lessons Learned from International Joint Evaluations French Ministry of Economy, Finance and Industry

6th February, 2012.

2

Budget Support Evaluation

MUTUAL TRUST

3

Consensus Building Process

Creation of two groups

Core Group Steering Group Head of evaluation

services OECD/ DAC

Delegate Tasks

Present work for further discussion and approval

4

Activities Include:

Agreement on the approach

Standard ToR for the evaluation of Budget support

Criteria for the choice of test evaluations

Inventory of BS support interventions and proposal for the tests

Introductory Note to the Partner country to ease involvement

Choice of (a) lead donor(s) for each evaluation

Ongoing work on Institutional Arrangements (multiannual evaluation plan,

training issues, quality control, harmonization and capitalization)

Donor Country

5

Partner Country

 

Activities Include:

•First contact through the representatives of the donors in partner country: leader in

country is the lead donor at the head quarter level.

•Agreement from the partner countries to undertake such an evaluation

•Different level of experience in regards to getting Govts on board

6

What worked well?

• Good involvement of the partner country, participation of different stakeholders

• Participation of some donors in a Reference Group at country level   • Good discussion among the Management Group at HQ level  • Important work provided by a small and strong team of consultants

• Rapid agreement on the conclusions and recommendations of the report

• Clear separation between lead donor and evaluation team (consultants)

The Case of Mali

7

Important Challenges

Management

Necessity to have a strong leadership at the HQ level of the lead donor

Not enough involvement of the partner country in reading and/or commenting the reports.

Final decision taken by the lead donor, well documented

In-depth work only with the donors available

and how have they been addressed

8

Desk analysis, field mission, synthesis phase and the 3 compulsory reports writing (inception, desk, final)

Too many versions of the report before reaching final agreement

Complexity of the 3 step approach

and how have they been addressed

Compromises on the less important issues

Acceptable for pilot evaluation but not for regular evaluations.

Continuous refinement for simplification and time efficiency purposes. But, budget support is a complex instrument and the evaluation of its impacts is even more complex.

Important Challenges

Methodology

9

•Finalisation of the ToR (inclusion of the information from every donors time consuming)

•Lack of information and too short foreseen field mission

•Significant difficulties to consolidate all the comments in an understandable and useful presentation for the consultants

and how have they been addressed themImportant Challenges

Basis of work, no compromise

Tailored according to the need

Work necessary

Timing:

10

Next Steps

• Simplification of the approach

• Current development of framework for Policy Dialogue for improved understanding (internal and

external) and better documentation (back to office reports, e-mail exchanges, etc)

• Raising awareness among Government officials, donors, NSAs

on the approach and how to improve the efficiency and effectiveness

of these evaluations.

First intervention : the workshop to be hosted by the Belgium government

( 21/22 March 2012)

• Increase pool of experts that can carry out such evaluation: development

and implementation of training workshops