Upload
felicity-norris
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Evaluation Office
EV
ALU
ATI
ON
THE INDEPENDENT EVALUATION OFFICE OF THE UNITED NATIONS DEVELOPMENT PROGRAMME REFLECTIONS ON CONTRIBUTION TO TRANSPARENCY, LEARNING AND ACCOUNTABILITY
June 2013Indran A. NaidooDirector
2
Demand from Members states and management. The Evaluation Policy of the UNDP clarifies roles and responsibilities for
evaluation The Evaluation Office (EO) of the UNDP, the largest in the UN system, produces
independent evaluations at the corporate, programme and country level. It extends its influence by managing the UNEG Secretariat, producing guidance
and standards, engaging with networks and supporting evaluation capacity building across the globe
Setting the stage
Evaluation Office3
1. Mandate and functions
Reports directly to the Executive Board of the UNDP
Supports the Administrator in her substantive
accountability function
Operates within the UNEG Norms and Standards and
ethical guidelines, and engages stakeholders in the
conduct of evaluations to ensure transparency, learning
and accountability
1999EO was
established
2004GA 59/250 Resolutions
for UN System
2005UNEG
Norms & Standards
2006 UNDP
Evaluation Policy and
Independence
2011 UNDP
Evaluation Policy (revised)
4
Evaluations are planned and answer questions within the Strategic Plan - has been RELEVANCE, EFFECTIVENESS, EFFICIENCY and SUSTAINABILITY
DO UNDP INTERVENTIONS MAKE A CHANGE?
EO staff lead all evaluations, drawing on advisory panels and using engagement processes with evaluands and stakeholders as a means to enhance credibility.
As an independent office, the Director signs off on all evaluations.
In practice this means…
Evaluation Office5
2. Contributions to other organisational tier support and professionalasation)
• Setting standards; Guides; Assessment of evaluation quality Setting standards; Guides; Assessment of evaluation quality
UNDP: Support to decentralized evaluation function and products
• Norms and standards; Guidelines; Methodology; Peer Reviews; UNEG Secretariat. (www.unevaluation.org)
United Nations Evaluation Group: Coherence in UN system-
wide evaluation
• Regional: AfrEA, RELAC, Malaysia Eval Society, IPEN
• International: NONIE; ECG; IDEAS; IOCE; Eval Partners
Professional Networks: Advancing Development Evaluation
6
Commitment to evaluator professionalisation
Ability to engage in a supportive yet independent manner
High levels of methodological skill and content expertise
Strong strategic and communication skills
For evaluators it requires …
7
Has produced over 100 evaluations since 2000, and 80 country level evaluations. All have a management response.
Management uses evaluations to review policies, programmes and approaches – shown in 92%
uptake on recommendations. Independence respected, and
evaluation on the agenda at key top management meetings.
Board allocates significant time to engage with evaluation findings, management responds fully.
The Evaluation Office
8
The EO engages fully as evaluation processes are as important as the “big report”
Learning can come through accountability processes, these are not opposed to each
Stakeholder workshops held at country level, led by EO, with government as key stakeholder.
Transparency and stakeholders
9
Globally, across a broad mandate In complex situations where
interventions are difficult to embed and hard to measure
Implications for evaluation: difficult to work out additionality in terms of attribution or contribution
Context plays an important role, not easy to superimpose systems to measure results, but not impossible. Involves high levels of engagement to agree on what constitutes success
Where UNDP operates
Evaluation Office10
UNDP Programmes and Operations
177 countries in 5 Regions
National
Goals and
Priorities
United Nations
Development
Framework
UNDP Focus Areas
Poverty and MDG Democratic
governance Crisis prevention and
recovery Environment and
sustainability development
Programs
•Global•Regional•Country•Others
Projects
Non-Project operations (advisory, advocacy, standard setting/
normative, coordinatio
n, mobilizatio
n)
11
Evaluations need to be context specific, yet meet evaluation norms and standards.
At the country level they inform the next UNDP programme, and draw on lessons from the last programme period
Aggregation for synthesis needs to consider the question of scale, variability and the challenges of validity
Stakeholder workshops demonstrate to government UNDP commitment to transparency and accountability
What does this mean?
12
Credibility goes beyond the tools and methods. It rests on the leadership that directs evaluation processes. This comes about through transparent and logical evaluation plans and processes which ensure engagement opportunities throughout and across the spectrum to reduce bias
Results must be engaged publically Independence is central for the
reasons above, to ensure credibility and authenticity of reports
Independence, a credibility question
Evaluation Office13
Transparency
Defined in UNEG norms and standards Consultations with Stakeholders
TORS > Inception Report on Scope, Design and plan for data collection and analysis > Stakeholder Meetings > Draft Reports
The Audit Trail <> significance for mutual understanding and final decisions by EO
Public Access All UNDP plans and evaluations in the Evaluation
Resource Center (l) EO evaluations in the EO website Management response and tracking system (ERC) Ratings on quality of decentralized evaluations
Evaluation Office14
Independence - Structural
Functions, and staff - organizationally independent from operations and policy units and decision making. Executive Board
Director reports to the Executive Board (2 terms and no re-entry into UNDP)
Board approval of programme of work and budget (independent of programme budget)
Reporting Evaluation reports are the
responsibility of the Director Transmitted directly to the Board
following review and comment by management
Senior managers safeguard the independence: EO has access to all records and information
Executive Board
UNDP Administra
tor
Evaluation Office
Director
Evaluation Office15
Examples from …2013
As of June 2013 the Evaluation Office of UNDP presented eight independent evaluations to the UNDP executive board, which will contribute towards the development of the new
UNDP Strategic Plan.Five Regional Programme Evaluations
Evaluation of UNDP Strategic
Plan
Evaluation of South-south and Triangular
Cooperation
Evaluation of Global Programme of UNDP
Evaluation Office16
Country Programme Evaluation (Assessment of Development Results – ADRs)
Evaluation Office has conducted 80 ADRs since 2002
17
Engaging with decision-makers – engagement for quality and credibility (management responses and update)
Statistics on uptake – 92%
Acting on evaluations
18
Support of professional networks, associations and events, to profile evaluation:Supportive of IDEAS, Barbados 2013 (key notes, panels courses)Supportive of IPDET – course presentationSupportive of EvalPartners – management boardSupport to continental and regional networks (AFREA, APEN, …Hosting of the National Evaluation Capacity workshops, next San Paulo, September 2013www.NEC2013.org
Global influence
19
Evaluation is not an event, but a difficult journey requiring constant push, reappraisal, strategising to ensure relevance. It does not occur naturally and requires drivers
An evaluation function must be independent, which can be supported when it is both inward and outward focused. The outward – through events like these brings in critical insights necessary for ongoing revision. An outward orientation is the only way that one get professionalisation that is necessary for evaluation.
Conclusions