Upload
others
View
16
Download
0
Embed Size (px)
Citation preview
Managing for Results Results Based Management in practice
Trine Eriksen and Alex Stimpson Financial Mechanism Office Vilnius, 11 December 2013
1
Johannes Hahn (Oct. 2013) “billions of euros of EU taxpayers' money has in the past been spent with no clear strategy or benchmarks” "The progress has to be measured and, crucially, communicated," said the commissioner.
RESULTS BASED MANAGEMENT
What is Results Based Management (RBM) RBM is a management strategy by which all actors, contributing directly or indirectly to achieving a set of results, ensure that their processes, products and services contribute to the achievement of desired results (outputs, outcomes and impact)
In the context of the Grants: A result is the output and the outcome of a development intervention A risk is an event that may occur and impede the objective
3
What is a result and a risk?
KEY CONCEPTS
Results achievement Beneficiary oriented Managing for results and reduced risks
4
In practice RBM is about • defining realistic expected results based on appropriate
analysis; • clearly identifying program beneficiaries and designing
programmes to meet their needs; • monitoring progress toward results and resources consumed
with the use of appropriate indicators; • identifying and managing risk; • increasing knowledge by learning lessons and integrating them
into decisions; and • reporting on the results achieved and resources involved.
RBM is supposed to help us:
– clarify what we mean by results and lets us deal with differences of understanding before a programme/project begins
– build capacity, because it clarifies for us what we need to concentrate on – produce more realistic schedules, forcing us to think through the preconditions
and sequence for actions – prepare programmes/projects for effective evaluations. – better identify unplanned results, as they occur, and assess if these are desirable,
or problematic, requiring support or coping strategies – making the links between inputs, funded activities and the results they should be
leading to, which reduces the potential for corruption
However…
• Agreeing on “clear” goals and objectives is often difficult • Using results-information to continually improve one’s own work and operations
requires time to reflect and acceptance from management • Ensuring that staff members can employ their creativity and resourcefulness to
engage on the “quest” for results requires a mind shift within the organisation In other words: red tape often remains and hampers the efforts of public agencies to increase their results-orientation
The challenge is to:
• Keep it simple • Getting stakeholder “buy-in” • Fostering organisational use of information • Making outcome accountability “operational”
and realistic • Measure • Ensure attribution • Linking financial and performance information
• The role of the PO is to keep the focus on the big picture during implementation – To create a simple and clear roadmap that guides
the project promoters
• What the donors (and we) want to be able to do, is to explain our results clearly to other people
Part I Quality at entry - Basis for good management
10
The Programming Cycle and Continuous Feedback
Planning
Programming
Budgeting
Implementation Monitoring
Evaluation
Reporting
PROGRAMMING CYCLE
11
Identify main programme elements of EEA/N Grants Objective (impact) Widespread improvements in societ or a sector
32 predefined objectives
Outcome(s) Intermediate effects on target groups and systems
160 predefined outcomes In DoRIS data base
Outputs Products and services delivered
Self-defined (at least two outputs per outcome)
In DoRIS data base
Activities Tasks transforming inputs to outputs
Self-defined Not in DoRIS
Inputs Financial, human and material resources
Budget + Budget in DoRIS
12
Programme logic
Inputs
Activities
Outputs
Expected outcomes
Objective
Overall objectives
if the inputs are available, then the activities will take place
which will contribute to the two overall objectives
if the activities take place, then the outputs will be produced
which will contribute to the fulfilment of the objective if the outputs are produced, then the expected outcome is likely to be achieved
Should be able to guarantee that this will happen
Likely to happen
Contribute to
Contribute to
13
ASSUMPTIONS
• The intervention logic is an aim-means chain and a cause-effect chain.
• It is assumed that “certain events” need to occur at one level in order to achieve the desired effect at the next level.
• It is not certain that these events happen; they represent uncertainties.
14
There are Assumptions between each level: Identify Assess Monitor Manage and
Mitigate
Impact/Objective
Outcome
Outputs (deliverables)
Activities
Inputs
15
Identify important assumptions
• Assumptions are worded as positive conditions (see objectives) • Assumptions are linked to the different levels of the
Programme Matrix. Each level must contain the necessary and sufficient conditions for the next level above.
• Start from bottom of matrix and work upwards • Are weighted according to importance and probability Example: Scholarship recipients return to assigned jobs
16
Programmes and projects
Programme proposal
Strategic approach
Projects Pre-defined projects
Pre-defined projects
17
EXAMPLE Green Industry Innovation
29
Increased competitiveness of green enterprises, including greening of existing industries, green innovation and green entrepreneurship
Realisation of the business opportunities of greening of the European economy
Environmental technologies in industry developed, improved or implemented by SMEs
Green products or material in industry developed or improved by SMEs
• Environmental technology development
• Implementation or replacement of the existing technology
• Processing sequence and/or synthesis of these innovations in order to run the processes at a higher efficiency and minimize waste, energy and emission generation
• New green product and material development or modification of the existing product and material to minimize environmental impact during or after its use.
Objective (pre-defined)
Expected outcome(s) (pre-defined)
Outputs
Types of projects
Call for proposals
Be clear about which outcomes and outputs you want the projects to contribute to. Which projects are necessary and sufficient? Project template in DoRIS: http://www.eeagrants.org/id/3192
19
Logical relationship
20
PART IV
21
Basis for results measurement
Basis for measuring results
Result based management
22
"The starting point for a results-oriented approach is ex ante setting of clear and measurable targets and outcome indicators. Indicators must be clearly interpretable, statistically validated, truly responsive and directly linked to policy intervention, and promptly collected and publicised." From the conclusions of the 5th Cohesion Report, adopted by the European Commission in November 2010
QUALITY AT ENTRY!
30
INDICATOR
A means to measure achievement, or to help assess a development
24
HOW SHOULD INDICATORS BE FORMULATED? • Once indicators have been identified, they should be
formulated to reflect measurable change. Indicators frequently utilize the following metrics to depict measurable amounts:
• Number; • Ratio; • Percentage; or • Rate; • Time Indicators shall whenever relevant, be broken down per gender
In DoRIS, always specify unit of measurement!
25
LESS IS MORE
EXAMPLES OF INDICATORS Example 1: Outcome: Cultural heritage restored, renovated and protected Indicator: Number of buildings of cultural heritage value restored or rehabilitated Example 2: Outcome: Awareness of cultural diversity raised and intercultural dialogue strengthened Indicator: Number of local cultural associations involved in the implementation of projects
26
INDICATORS
To describe how results will be measured, output indicators must be specific, observable and come from reliable data. • Does the indicator provide the most direct evidence
of the condition or result to be measured? • Is the indicator important, understood, and accepted
by important stakeholders?
27
Think about…
• Are data for the indicator collected in the same way over time?
• Will data be available for the indicator? • Will data be collected frequently enough? • Are data currently being collected? • If data are not currently collected, can cost effective
instruments for data collection be developed quickly?
28
Indicators related to Bilateral Relations
• All programmes should contribute to strengthened Bilateral Relations
• Use the Bilateral guideline. Four types of bilateral results expected: Extent of cooperation Shared results Wider effects Knowledge and mutual understanding 29
Indicators related to «Horizontal issues»
• A human-rights based approach and concerns for horizontal issues like fighting discriminiation, racism, hate speech etc. should guide defining the outputs and the indicators.
Example: % of members of X Parliament are minorities Example: Number of annual incidences of hate speech in Y school
30
Programme matrix - indicators
Sources of Verification
• When defining the indicators, consider the data sources. Where can you easily get data that are reliable and timely? • Specify your sources of information
• Assess where you can get easy access to data Example Ministry of Health’s statistics on cancer (health status of population) and various statistics on cancer detection and treatment services
Programme matrix – Sources of Verification
WHAT ARE BASELINES? • Baselines provide context for the setting of
targets and capture the situation before a development intervention begins, or at the beginning of a time period that will be monitored and assessed.
• Baselines describe the conditions prior to programming efforts. The present situation!
• Essential to make credible and meaningful assessments on progress towards outcomes. 34
Programme matrix – Baseline data
WHAT ARE TARGETS? • Targets set the level for the magnitude of change
expected by the end of a predetermined time period. They reflect what organizations commit themselves to achieve by the end of some duration of time.
• Targets provide tangible and meaningful points of
discussion with beneficiaries, stakeholders, and partners, and allow us to add further specificity to the outcomes from the results logic.
36
TARGETS
• When you have defined the indicators and know the baseline situation, you set the targets.
• What do you want the situation for the beneficiaries to be in 3 or 5 years?
• Examples: • 1. 60% of W 40+ called in for cancer test in Southern Region by year
2016. We want to increase the % of W 40+ called in from 30% in 2010 to 60% in 2016. • 2. Time lag between detection of cancer to operation is not more
than 3 months in 2016.
37
BASELINES AND TARGETS SUMMARY • A baseline is the value of the indicator before the
implementation of the project starts • Targets orients stakeholders to the tasks to be
accomplished • Targets help establish clear expectations • Targets serve as the guideposts for monitoring
whether progress is being made • Targets promote transparency
38
Part IV Quality at implementation • Monitoring • Reporting on results and risks • Evaluation
39
The Programming Cycle and Continuous Feedback
Planning
Programming
Budgeting
Implementation Monitoring
Evaluation
Reporting
PROGRAMMING CYCLE
40
PERFORMANCE MONITORING • Define processes and systems for performance
measurement; • Decide the intervals at which measurement will occur; • Assign roles and responsibilities; • Set standard requirements for project promoters • Build in adequate time and resources for analysis of
performance measurement data so that progress is reviewed and assessed.
• Use web site for communication with project promoters and other stakeholders
41
Performance monitoring
• Status regarding the output and outcome indicators is measured and reported towards the baseline situation (“where we were”), and also towards the future, desired situation (“the targets”).
Programme performance framework
Group work – Results based management • Questions for discussion: With the principles of results based management in mind, in your view: • Is there a clear intervention logic (link between
outputs/outcomes/objective)? • Are the Outputs sufficient to achieve the Outcome? • Are the indicators good measures of achievement? • Is it clear what the indicators refer to/mean?
44