Upload
buituong
View
237
Download
0
Embed Size (px)
Citation preview
The National Evaluation Policy Framework:Engendering an Evaluation Culture in the Philippines
3rd M&E Network Forum
06-08 November 2013
Asian Development Bank, Mandaluyong City
Deputy Director-General Rolando G. Tungpalan
National Economic and Development Authority (NEDA)
1
Presentation Outline
• Preliminaries
– Public Sector Management
– Policy Milestones
• Country Context
– Results Framework
– Evaluation Culture
– Institutional Mandates
• National Evaluation Policy
– Initial Activities
– Draft Policy2
Public Sector Management (PSM)
Source: Asia Pacific Community of Practice on Managing for Development Results, Framework for Results-Based Public Sector Management. September 2011.
Features of results-oriented
PSM:
a. presence of core result
attributes
b. a focus on common
results
c. interdependency among
the components
d. effective vertical and
horizontal linkages
4
NATIONAL
DEVELOPMENT
GOALS
PLAN FOR RESULTS• Indicators with targets are
specified for each level of
results
BUDGET FOR RESULTS• Budget supports planned
national priorities
IMPLEMENT FOR
RESULTS• Policies, people,
processes oriented to
deliver intended results
MONITOR RESULTS• Defined indicators are used
to track achievement of
agreed results
EVALUATE RESULTS
• Evaluation
methodology enables
assessment of results
and informed decision-
making
1987
EO 230
Reorganizing
NEDA mandates
the agency to
monitor and
evaluate the
implementation
of the PDP
1989 1992 1993 1996 1999 2001
EO 376 established
the Regional
Project Monitoring
and Evaluation
System (RPMES)
for M&E at the sub-
national level
EO 93 further
refined and
streamlined the
roles and
responsibilities
and operating
procedures under
the RPMES
NB Resolution
No. 30 instructed
the ICC to review
all ongoing ODA-
funded projects
with the aim of
improving
absorptive
capacity
RA 8182 (as
amended by RA
8555) mandated
NEDA to conduct
an annual review of
status of all ODA
projects
NB Resolution No.
3 provides for
reporting of project
outcomes and
impacts by ICC and
Implementing
Agencies
Sector Effectiveness
and Efficiency Review
(joint exercise by DBM
and NEDA, through
the 2001 National
Budget Call) assessed
the responsiveness of
programs and
projects to sector
outcome objectives
2007
DBM adopted the
Organizational
Performance Indicator
Framework (OPIF) which
seeks to align goods
and services supported
by the budget with the
government’s desired
outcome objectives
2011
AO 25 provided the
mechanism for the
establishment of a
unified and
integrated Results-
Based Performance
Management System
within the Executive
Branch of the
government.
5
Policy Context
Evaluation
• Definition
• Uses of Evaluation
a. for learning – to improve future projects and programs through
feedback of lessons learned
b. for accountability – disclosing information to stakeholders
Institution Definition
OECD
DAC
An assessment, as systematic and objective as possible, of an on-
going or completed project, program or policy, its design,
implementation and results.
UN
Evaluation Group
(UNEG)
An assessment, as systematic and impartial as possible, of an activity,
project, programme, strategy, policy, topic, theme, sector, operational
area, institutional performance, etc.
6
GPH Results Framework
Sectoral Goals/Outcomes
Good Governance and Anti-Corruption
Human Development & Poverty Reduction
Economic Development
Security, Justice,and Peace
Climate change
Societal Goals/Outcomes
Organizational Outcomes
Major Final Outputs (Citizen-focused Service and Product Result )
Enab
lers
/ D
rive
rs
Financial Stewardship Internal Process Leadership, Learning, and Growth
OP
IF
Strategic Performance Management System (Individual)
EO 4
3/P
DP
(RM
)P
DP
(RM
)
Res
ult
s/O
utc
om
esO
rgan
izat
ion
al
Man
agem
en
t R
esu
lts
Co
mp
reh
ensi
ve P
erfo
rman
ce In
dic
ato
rs
8
Country Context: Evaluation Culture
Criteria Assessment
Evaluation takes place in many domains. Emerging
Supply of domestic evaluators in different fields. Present
National discourse concerning evaluation. Emerging
Presence of a profession with own societies. Emerging
Institutional arrangements in the government. Present
Institutional arrangements in legislative bodies. Emerging
Pluralism exists (institutions, evaluators) Present
Evaluation takes place within the audit institution. Emerging
Outcome evaluations (not output and process) Emerging
Note: Criteria adapted from Furubu, Rist, Sandahl, 2002, International Atlas of Evaluation.
Assessment of country-context done by the presentor. 9
Country Context: Evaluations
• Evaluations more pronounced at the program/project
level. Usually conducted by development partners
through external evaluators.
• Policy evaluations
• Sectoral evaluations
• Evaluation of Country Assistance Strategies
• Country Level Evaluation (e.g., PD, MDG)
• Development Plans (at the national and regional levels)
10
Country Context: Institutional Mandates
• M, E, M&E conducted at the Implementing Agencies
(IAs), Oversight Agency (OAs), and Inter-Agency
committees (IACs).
IAs OAs IACs
• Internal M&E
system
• Project
Implementation
Officers (PIO)
System
• NEDA (planning)
• DBM (budgeting)
• COA (auditing)
• Office of the
President
• Office of the
Cabinet Secretary
• Investment
Coordination
Committee
• Project Monitoring
Committees under
the RPMES*
11*Regional Project Monitoring and Evaluation System
National Evaluation Policy
Preliminary Activities by NEDA
• Initial review of existing international evaluation policies
Evaluation Policy
What is it? A policy framework to guide evaluation activities.
Why do we
need it?
Evaluation is a crucial function that informs
decision-making. A set of policy statements on it
will promote and strengthen the practice and use
of evaluations.
13
Source: R.G. Tungpalan, “National Evaluation Policy Framework,” presented at the 2nd M&E Network Forum, 07 November 2012.
National Evaluation Policy
Elements of an Evaluation Policy
14
•evaluation
concepts
•role of evaluation
•guiding principles
•evaluation
standards
•conduct of evaluation (programming,
planning, design, management)
•defined institutional responsibilities
•disclosure and dissemination
protocols
•periodic review of policy
Source: R.G. Tungpalan, “National Evaluation Policy Framework,” presented at the 2nd M&E Network Forum, 07 November 2012.
National Evaluation Policy
• Consultants – Donald Hall and Ruperto Alonzo
• Technical assistance from UNICEF
• Methodology:– Review of other country experiences
• Literature review
• Interviews
– Interview of stakeholders• Oversight agencies – NEDA, DBM, DOF, OP-PMS, COA
• Implementing agencies – DPWH, DILG, DAR, DA, DOH
• Other institutions – House of Representatives, Senate, academe
• Development partners – UNICEF, ADB, USAID, AusAID
15
Consultations on the Policy
(timeline)
16
(based on original 30 Aug 2013 draft)
• Project Implementation Officers (9 Sep 2013)
• National Project Monitoring Committee / RPMES* Forum
(24 Sep 2013)
• Investment Coordinating Committee of the NEDA Board
(04 Oct 2013)
• Public Consultation Forum (18 Oct 2013)
*Regional Project Monitoring and Evaluation System
Consultations on the Policy
(comments)
17
• Broad-based support from all stakeholders.
• Concise policy document (i.e., details spelled out in
separate guideline/s).
• Options to effect the policy (i.e., legislative act, Executive
Order, NEDA Board Resolution, Memorandum Circular).
• Conscious effort on data generation (baselines, etc.)
18
• Greater elaboration on the roles and responsibilities of
institutions and organizations (e.g. participation of
oversight agencies, role of sub-national oversight and
line agencies, and internal arrangements for evaluation
units of IAs)
• Provision of resources in support of the evaluation
function (for the conduct of evaluations and for capacity
building).
Consultations on the Policy
(comments)
National Evaluation Policy Framework*
19
Section Content
1 Intent Emphasizes government’s aim to achieve PDP
societal goals of inclusive growth and poverty
reduction.
Project/program planning and implementation
characterized by evidence-based decisions,
accountability and learning which, in turn, are
supported by systematic, rigorous and impartial
evaluation.
Evaluation policy intended to govern the practice
of evaluation in the country towards more effective
development.
*based on revised draft as of 31 October 2013
20
Section / Content Highlights
2 Objectives
States the objectives of
the policy in support of
evidence-based
project/program
planning and
implementation
through systematic,
rigorous and impartial
evaluations.
Spells out the objectives of the policy:
a) support for evidence-based decisions;
b) promotion of program improvement;
and,
c) promotion of accountability.
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
21
Section / Content Highlights
3 Coverage
States the
scope of the
Policy
Proposed policy covers all programs/projects
managed by departments and agencies.
Evaluations address, at the minimum the following
questions covering three issues:
a) effectiveness (on achievement of objectives
and on unintended results);
b) operational efficiency (on efficient delivery of
outputs and on operational alternatives);
c) relevance (on alignment with national
priorities, on uniqueness/ coordination, and
on programmatic alternatives).
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
22
Section / Content Highlights
4 Institutional
Responsibilities
Enumerates the
responsibilities of
implementing and
oversight agencies,
and of the Evaluation
Secretariat.
Implementing agency heads responsible for
the establishment within their agency of a
capable, neutral evaluation unit. The head of the
evaluation unit reports directly to the
implementing agency head.
All project/program proposal put forward by
implementing agencies for approval in the
course of the annual budgeting process or at
other times include an evaluation plan. Proposal
lacking an adequate evaluation plan are not
eligible for approval. Disputes with respect to
evaluation plan adequacy are referred to the
Evaluation Secretariat.
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
23
Section / Content Highlights
4 Institutional
Responsibilities
Enumerates the
responsibilities of
implementing and
oversight agencies,
and of the Evaluation
Secretariat.
Implementing agencies to ensure that
evaluations are undertaken with due regard for
impartiality and in line with evaluation best
practices.
Implementing agencies to maintain a rolling 5-
year evaluation agenda listing projects/programs
to be evaluated during the current year and the
subsequent 4 years.
Protocols on Reporting, Dissemination and
Use of Evaluations prescribed.
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
24
Section / Content Highlights
4 Institutional
Responsibilities
Enumerates the
responsibilities of
implementing and
oversight agencies,
and of the Evaluation
Secretariat.
Establishment of a Government of the
Philippines Evaluation Secretariat at NEDA.
Quality Assurance
- Review and assess quality of IAs
evaluation reports and provide feedback to
agencies concerned
- Submit annually to the NEDA Board a
summary of GPH evaluation activities
including an assessment of the overall
quality and comprehensiveness of IA
evaluation
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
25
Section / Content Highlights
4 Institutional
Responsibilities
Enumerates the
responsibilities of
implementing and
oversight agencies,
and of the Evaluation
Secretariat.
Establishment of a Government of the
Philippines Evaluation Secretariat at NEDA.
Technical Support
- Provide advice and support to IAs in: (a)
preparing evaluation plans; (b) preparing
rolling 5-year evaluation plans; and, (c)
guidance in the conduct/management of
evaluation.
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
26
Section / Content Highlights
5 Implementation
Tackles the capacity
and financial
considerations in
implementing the
Policy. Also contains
a process for the
review of the Policy.
Department and agencies to allocate in their
annual budgets adequate resources (e.g.,
including for capacity building, external
professional service fees, etc.) to implement
policy provisions. Evaluation Secretariat
adequately supported by resources.
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
27
Section / Content Highlights
5 Implementation
Tackles the capacity
and financial
considerations in
implementing the
Policy. Also contains
a process for the
review of the Policy.
Evaluation and Amendment
-A formative evaluation of the Policy shall be
done within the second year of
implementation; the findings shall inform
amendments to the Policy as warranted.
-A summative evaluation of the Policy shall
be completed within five years following the
effective date of the policy
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
28
Section / Content Highlights
6 Ethics
Contains the
standard of ethics to
be observed in
undertaking
evaluation
All those engaged in designing, conducting
and managing evaluation activities shall abide by
the Code of Conduct and Ethical Standards for
Public Officials and Employees (RA 6713).
Evaluators shall respect the right of
implementing agencies and individuals to
provide information in confidence, and ensure
that sensitive data cannot be traced to its source.
Evaluators must ensure that those involved in
evaluations have the opportunity to review and
approve the statements attributed to them.
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
29
Section / Content Highlights
6 Ethics
Contains the
standard of ethics to
be observed in
undertaking
evaluation
Evaluators shall be sensitive to the cultural,
social and economic environment of all
stakeholders, and conduct themselves in a
manner that is fair and appropriate to this
environment.
Evaluators shall be accountable for their
performance and their products.
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
30
Section Content/Highlights
Annex A:
Historical Context
Describes the government’s reform initiatives
over time related to performance evaluation,
results orientation and monitoring and evaluation.
Annex B:
Impartiality
Ensure impartiality in order to maximize
objectivity and minimize potential for bias.
Annex C:
Best Practices in
Evaluation
Lays down best practices in evaluation in terms
of (a) evaluation scale, and (b) evaluation design
and execution.
Annex D:
Evaluation
Competencies
Those engaged in designing, conducting and
managing evaluation should demonstrate
competencies on: (a) technical foundations; (b)
leading, managing and delivering evaluations; (c)
communicating and sharing evaluation findings;
and (d) integrity.
National Evaluation Policy Framework
*based on revised draft as of 31 October 2013
Our Take on the Policy
31
• There is a need to step up on capacity
development
• Responsiveness of the statistical system should
be enhanced to support the results framework
• Policy will support organization learning
• Policy enhances public sector management
cycle integration