26 June 2014Geneva

Evaluation Office

Annual Evaluation Report 2014

A reminder: the purpose of evaluation at UNFPA is three-fold

It is means to demonstrate accountability to

stakeholders on performance achieved

It contributes important lessons learned to the knowledge base of the

Organization

It supports evidence-based decision-making

Corporate Evaluations

Programme-level

Evaluations

THEMATIC EVALUATIONSConducted by the Evaluation Office

Corporate Evaluations (2012-2013)

4

Country Case Studies: Burkina Faso, Cambodia, DRC, Ethiopia, Ghana, Kenya, Lao PDR, Madagascar, Sudan, Zambia

JOINT EVALUATIONSConducted by the Evaluation Office

Joint Evaluations (2012-2013)

6

Country Case Studies: Burkina Faso, Kenya, Senegal, Sudan (FGM/C)

Country Case Studies: Albania, Kenya, Liberia, Nicaragua, State of Palestine (JGE)

INDEPENDENT COUNTRY PROGRAMME EVALUATIONSConducted by the Evaluation Office

Independent CPEs covering the UNFPA Regions

LACRO - 2011 WCARO - 2011 ESARO – 2012

ASRO: Lebanon (2013 – under finalization in 2014)

PROGRAMME-LEVEL EVALUATIONSConducted by regional and country offices

Country Programme Evaluations 2012 and 2013

Programme 2012 2013 Total

Arab States 3 0 3

Latin America and the Caribbean 1 3 4

Asia and the Pacific 1 1 2

West and Central Africa 6 1 7

East and Southern Africa

6 (incl 2 independent) 1 7

Eastern Europe and Central Asia 0 1 1

Total CPEs 17 7 24Regional Programme 1 5 6

Grand Total 18 12 30

Coun

trie

s Re

gion

10

Quality: Programme-Level Evaluations Evaluation Office quality assessment: 27 final reports of

programme-level evaluations conducted by country and regional offices 5 (19%) reports rated as good 22 (81 %) reports rated poor or unsatisfactory

Improvement in quality compared to 2010-2011 (91% rated poor or unsatisfactory)

Some specific areas of improvement: 67% of reports meeting needs stated in terms of reference 44% of reports had a good/very good methodology 81% of reports had poor/unsatisfactory findings, analysis,

conclusions

Availability of skilled and experienced evaluators

Affordability of skilled and experienced evaluators

Evaluators

Adequate results frameworksResults oriented monitoring

system in placeAccess to documentationAccess to key informantsParticipation in reference

groups

Management

Clear evaluation governance mechanisms

Clear evaluation standards Skilled and experienced

evaluation managersAdequate planningAdequate budget

Adequate procurement systemQuality assurance system in

place

Evaluation function

Good quality

evaluation reports

Conditions for good quality and credible evaluation

?

?

EVALUATION CAPACITY, USE AND LEARNING

Human Resources

UNFPA Country Offices 38 % dedicated monitoring and evaluation officers 62 % focal point for monitoring and evaluation

Significant variation between regions WCARO 61%/ ESARO 64% country offices = dedicated M&E

staff EECARO 0%/LACRO 24% country offices = dedicated M&E staff Largely reflective of structure and size of the country

programmes Regional Office 6 Regional M&E Advisers

Evaluation Office 7 approved posts

Building evaluation capacity in UNFPA

Skilled monitoring and evaluation staff is a key element to ensure the quality, credibility and use of evaluation

To note the key role of Regional Office M&E Advisors Key Activities 2012-14:

Seven webinars focused on the preparation, use and quality control of programme-level evaluations

Evaluation quality assurance mechanisms established - terms of reference; pre-qualification of evaluators; assessment of the quality of final evaluation reports

Handbook on How to Design and Conduct a Country Programme Evaluation at UNFPA

Four evaluation training workshops for 134 M&E staff

PART 1Designing the

Evaluation

PART 2Conducting the

Evaluation

PART 3Tools &

Resources

Toolkit

Templates

Outlines

Glossary

Elements of Theory

www.unfpa.org/public/home/about/ Evaluation/Methodology

Key Activities: Handbook on How to Design and Conduct a Country Programme Evaluation at UNFPA

Key Activities: EO Methodology Training Workshops

YEAR PARTICIPANTS LEVEL OF SATISFACTION (%)

Total to date= 134 extremely useful

useful slightly useful

AFRICA (ARO)

2012 52 70 30

APRO 2013 35 62 29 9

EECARO 2013 19 38 62

LACRO March 2014 28 80 20

ARAB STATES

TBD in 2014

AFRICA (WACRO)

TBD IN 2015

17

UNFPA evaluation capacity

Orga

nis

ational Stru

ctur

e

Staff @ CO/RO/HQRoles and responsibilitiesAlignment with organisational structure

M&E Staff Capacity

ProfessionalisationCapacity development

UNFPA Managers

Results based managementEvidence based decision making

Evaluation on the UNFPA Website

19

The Evaluation Database

Evaluation follow up and use

Corporate/programme

eval

uations

Management response Follow up

Corporate Accountability

Executive/ managerial supervisory system

Strategic useof evidence

UNFPA EVALUATION PARTNERSHIPS

UNFPA evaluation partnerships

Working with national partners Emphasis on improving the capacity of partners to use evaluation

results Key role in evaluation governance (participation in evaluation

reference groups)United Nations Evaluation Group Contribution to the task forces of the group and to organisation of

the 2013 Evaluation Practice Exchange seminarUnited Nations system-wide action plan for gender equality Participation in piloting the system-wide action plan on gender

equality and the empowerment of women in UNFPA in 2013

EVALUATION FUNCTION IN TRANSITION

UNFPA 2013 Evaluation Policy

Creation of an independent Evaluation Office reporting directly to the Executive Board

Evaluation Office is independent from: operational, management and decision-making functions = impartial = objective

UNFPA Evaluation Plan and Budget 2014-15

Key support role of the Regional Office M&E Advisers

Programme Division: Follow-up of recommendations and methodological guidance on results-oriented monitoring (ROM) =

THE ROLE OF THE EVALUATION OFFICE

Where is the EO within UNFPA?

27Country Offices

Executive Director

Division for Oversight Services

(DOS)

Office of the Executive Director

(OED)

Arab States Regional

Office(ASRO)

Asia/Pacific

Regional Office (APRO)

Eastern Europe/

Central Asia Reg. Office(EECARO)

Deputy Executive Director (Programme)

Deputy Executive Director (Management)

Latin America/

Caribbean Reg. Office

(LACRO)

Ethics Office(EO)

Division for Human

Resources(DHR)

Division for Management

Services (DMS)

ProgrammeDivision

(PD)

TechnicalDivision

(TD)

Information and External

RelationsDivision (IERD)

Southern/Western AsiaSub-Regional

Office

PacificSub-Regional

Office

Central AsiaSub-Regional

Office

CaribbeanSub-Regional

Office

Country Offices

Country Offices

Country Offices

Procurement Services

(PSB)

Management Information

Systems(MIS)

UNFPA Executive Board

Western/Central Africa

Regional Office

(WESARO)

Evaluation Office(EO)

Eastern/Southern

Africa Sub-Regional

Office(ESARO)

CountryOffices

Country Offices

Office of the Security

Coordinator (OSC)

Role of the Evaluation OfficeThe custodian of the evaluation function at UNFPA.

Key functions:

Contribute to accountability for results and, through high-quality evaluations, to generate learning to help UNFPA improve its strategies and programmes, including delivery of the new Strategic Plan

Conduct corporate evaluations (thematic evaluations and independent country programme evaluations)

Ensure the quality and utilization focus of UNFPA evaluations

Role of the Evaluation Office (cont.)

Actively contribute to knowledge sharing on evaluation results and lessons learned

Lead capacity development and professionalization of the evaluation function at UNFPA

Prepare the UNFPA budgeted evaluation plan and report to the Executive Board

Contribute to the evaluation practice within the UN, including capacity development of partners

29

LOOKING FORWARDS: CHALLENGES AND PROSPECTS 2014-2015

Looking forwards: programme of work

Evaluation Office: UNFPA support to adolescents and youth, 2008-2014

Evaluation of UNFPA support to population and housing census data to inform decision-making and policy formulation, 2005-2014

UNFPA support to the availability of quality family planning services, 2008-2013

UNFPA country programme in Turkey, 2011-2015 UNFPA country programme in Bangladesh, 2012-2016

Country Programme Evaluations 2014 and 2015Geographical Coverage

4 March 2014 32

Looking forwards: areas for attention

1. Continued efforts to improve quality and credibility of all UNFPA evaluations

2. Increased coordination between the Evaluation Office and Regional Office M&E Advisers to better support Country Offices

3. Capacity development and professionalization of the evaluation function

4. Enhancing evaluation follow-up and use