Eustace Uzor,

Consultant, IDEV/AfDB

June 22, 2018

AfDB, Abidjan

IDEV Webinar Series

Promoting Development Effectiveness in Multilateral

Development Banks (MDBs): Striking the right balance

between Evaluation Independence and Operational

Embeddedness

Outline

1. Setting the stage

2. The objective of development evaluation

3. Independence of the evaluation function at MDBs

4. Striking the right balance between independence and

embeddedness.

5. Discussion

The Independent Development Evaluation of the African Development Bank

Subtopic

Setting the Stage

The Independent Development Evaluation of the African Development Bank

Subtopic

Development Evaluation = Enhanced Development Effectiveness

The Independent Development Evaluation of the African Development Bank

Subtopic

What development evaluation is not POLICING!

The Independent Development Evaluation of the African Development Bank

Subtopic

Independence in the Development Evaluation Function

What is evaluation independence?

The evaluation function is independent when it is structurally independent

from operational management and decision-making functions within an

organization so that it is free from undue influence, more objective, and has

full authority to submit reports directly to the appropriate levels of

decision-making (UNDP 2016).

Formal independence – can either enable/hinder evaluations.

Substantial independence – ensures integrity, objectivity, impartiality.

The independence of evaluation departments/offices should not

translate to isolation from operations

The Independent Development Evaluation of the African Development Bank

Subtopic

S

Striking the right balance between independence and embeddedness

Problem Statement:

Although independence is necessary to ensure high-quality and objective (White

2014), it is not a guarantee for a quality evaluation product (ADB, 2008).

Institutional independence does not necessarily safeguard against biases

toward positive evaluation (poor quality evaluation designs, contract

renewal bias, and ‘friends’ bias etc) World Bank (2003).

These biases are best overcome by having a good quality assurance system

What degree of embeddedness with operations is required to improve

evaluation quality and make evaluation recommendations more

actionable?

The Independent Development Evaluation of the African Development Bank

Subtopic

S

Striking the right balance between independence and embeddedness

Rationale for engaging with Operations/Embeddedness:

The Independent Development Evaluation of the African Development Bank

Subtopic

The BUSD-heuristic (betterment, undermining, support, distortion) Model of Influence

Explicitness of Influence

Explicit Implicit, Indirect

Direction of

Influence

Destructive

Distortion

Negative Pressure

Undermining

Negative Persuasion/Counsel

Constructive

Betterment

Positive Pressure

Support

Positive Persuasion/Counsel

Source: Pleger and Sager (2016)

S

Striking the right balance between independence and embeddedness

Engagement with Operations - Current Practice:

The Evaluator General is regularly invited to senior management meetings

such as Senior Management Coordination Committee (SMCC), and the

Operations Committee (OPSCOM), to ensure inclusion of the independent

evaluation function, and support feedback of evaluation knowledge (AfDB,

2016).

Capitalization Workshops, with 1-2 pages of Management Response to

Committee on Operations and Development Effectiveness (CODE).

Involvement from ToR, Reference Group, to Learning Events on

Evaluations.

The Independent Development Evaluation of the African Development Bank

Subtopic

S

Striking the right balance between independence and embeddedness

However, IDEV staff recuse themselves from evaluating:

Any project, program, or activity that they worked on or had line

responsibility for the work on, including preparation, appraisal, supervision,

and completion reporting, or that they had a personal influence over, in a

previous capacity either in the Bank or prior to joining the Bank (AfDB 2016,

Annex 1).

In contrast, the Independent Development Evaluation of the Asian

Development Bank (ADB) comments on and reviews important projects or

programs and TA before approval (ADB 2008).

What degree of embeddedness is sufficient to maintain

independence?

The Independent Development Evaluation of the African Development Bank

Subtopic

Striking the right balance between independence and embeddedness

My view:

Review and Validate the Results Frameworks and Theory of Change of operations/projects, ex ante.

The Office of Evaluation and Oversight at the Inter-American Development Bank reviews the proposed monitoring and evaluation (M&E) systems of projects at the design stage, rather than waiting around to criticize them once the project is over (White 2014).

Rotate Young Professionals and Low-Mid-level Evaluation Staff to Operational Departments for a few months, as part of onboarding.

ADB encourages evaluation staff rotation – in principle.

Is Independence preferred over influence?

The Independent Development Evaluation of the African Development Bank

Subtopic

Thank you for your time!

The Independent Development Evaluation of the African Development Bank

Subtopic

References

1. ADB. 2008. “Review of the Independence and Effectiveness of the Operations EvaluationDepartment.” Mandaluyong, Philippines:. Asian Development Bank (ADB).

2. AfDB. 2016. “Independent Evaluation Policy: The African Development Bank Group.”Abidjan, Côte D’Ivoire:. Independent Development Evaluation (IDEV), AfricanDevelopment Bank Group.

3. Pleger, Lyn, and Fritz Sager. 2016. “Betterment, Undermining, Support and Distortion:A Heuristic Model for the Analysis of Pressure on Evaluators.” Evaluation and ProgramPlanning 69: 166–72.

4. UNDP. 2016. “Evaluation and Independence: Existing Evaluation Policies and NewApproaches.” New York:United Nations Development Programme IndependentEvaluation Office.

5. White, Howard. 2014. “Is Independence Always a Good Thing?” International Initiativefor Impact Evaluation (3ie). http://blogs.3ieimpact.org/is-independence-always-a-good-thing/.

6. World Bank. 2003. “Independent Evaluation: Principles, Guidelines And Good Practice.”The World Bank Development Grant Facility (DGF) Technical Note. Washington D.C.:World Bank.

The Independent Development Evaluation of the African Development Bank

Subtopic