Results-Based Management (RBM) training

Results-Based Management (RBM) training

9-11 April 2014

Anders Pettersson

[email protected]

Training rules

Everyone can participate

Value each other’s ideas

No mobile devices

Respect each other by being on time

Questions any time (answers may come later)

Other?

Presentation

• Your name

• One expectation

• One fear

• A secret about yourself

Purpose and content • To familiarise participants with the concepts and

ideas behind the RBM-methodology

– Results chain

– Problem analysis

– Setting goals (RAF)

– Indicators and baseline

– Risk analysis

– Budgeting

– Monitoring and reporting results

What is RBM? Background: Aid Effectiveness Agenda

RBM is a shift in focus from activity-based to results-based.

- What do we want to change?

- How to we plan to do it?

- Are we making progress?

RBM is a concept just as much as a number of tools!

Meier (2003): “Results-Based Management (RBM) is a management strategy aimed at achieving important changes in the way organisations operate, with improving performance in terms of results as the central orientation.

RBM provides the management framework with tools for strategic planning, risk management, performance monitoring and evaluation. It’s primary purpose is to improve efficiency and effectiveness through organisational learning, and secondly to fulfil accountability obligations through performance reporting.“

Input -

Activities

Output

Immediate outcome

Intermediate outcome

Ultimate outcome - Impact

Construction of a well using money,

people and material

A well with hand pump

Increased access to clean water in the village

Increased usage of clean water in the village

Improved health in the village

Tiger prawns are produced through cutting down mangroves and overfishing, which damages environment. Your job is to design a project /campaign within Ukraine to address this problem.

Results chain – one result will lead to another!

Problem analysis

• To achieve results, you must know the problem!

• Problem analysis is a crucial tool to design a good project/programme and part of the RBM approach!

– A well designed project/programme is more likely to achive results!

Simplified model

Effects

Core problem

Causes No wells in the

village

No clean water available

People get sick

Task

• Work in groups and design a ”Problem

tree” based on the information in the

summary of the Ukrainian energy policy

problems.

Input

• Controlled by you

•Internal risks

•Few assumptions

Output


• Internal risks

•Few assumptions

Immidiate outcome

• Likely to happen if outputs are produced

• External risks

•Basic assumptions


• You are making a valuable contribution

• External risks

•Assumptions required

Ultimate outcome/Impact

• You are hopefully making a contribution

•External risks

•Other factors involved

Results chain

Achieving institutional change

Changes in behavior of people

Changes in behavior of institutions

Changes in behavior of system

RAF/LFA/RBM-matrix

• ”A dear child has many names”

• Tool for planning and follow-up of a project.

• Important to understand principles rather than details!

Link between problem tree and LFA/RBM

Problem

Effects

Causes

Overall objective or

ultimate outcome and intermediate

outcome

Project purpose or

Immediate outcome

Expected outputs or

Outputs

Activities or Inputs

Link example

No wells in the village

No clean water available

People get sick

Problem

Effects

Causes



outcome

Project purpose or

Immediate outcome

Expected outputs or

Outputs

A well with a hand pump in

the village

Water contain 50 % less bacteria

Reduced cases of diarrhoea by

80 %

Group 1

Gov. does not have strategy in energy sector

Bad state regulation in

energy sphere

Overspending energy of

government and business

Problem

Effects

Causes



outcome

Project purpose or

Immediate outcome

Expected outputs or

Outputs

1. A new strategy developed.

2. New legislation developed

3. Int. standards are advocated to join int. networks

1. Int. standards ratified by gov.

2. Ukr. Join at least 2 networks

3. Adoption new legislation

Effective system in energy sector

Group 2

Bad legislation Lack of public

control

Corruption in

energy sector

Energy dependency

Problem

Effects

Causes



outcome

Project purpose or

Immediate outcome

Expected outputs or

Outputs

1. Developed PP “Reducing depency in energy sector”

2. Developed PP “How to improve investment climate and anti-corrputon”

3. Conducted lobbying campaign

1. New energy policy adopted which includes recommendations.

2. Implementation of new policy

1. Reduced energy dependency 2. Reduced corruption in energy sector

Group 3

Lack of knowledge Lack of awareness Not much NGO action

Nobody cares about energy

use/consumption

High energy intensity causes environmental

damage No use of nature

friendly technolody

Problem

Effects

Causes



outcome

Project purpose or

Immediate outcome

Expected outputs or

Outputs

1. Research and information materials produced on level of energy.

2. Built advocacy coalition of NGOs

3. Awareness and advocacy campaign conducted on….

4. Draft of new legialstion 5. Annual medals awarded to

business (best and worst)

1. Public pressure on business and politicians to cut energy consumption

Contribute to reducing pollution Promote use of green tech. by business Decrease of energy consumption

Poor environment

Common mistakes

• No established results chain

• Results not specific and not possible to

measure

• Attribution problem

• To ambitious and lack of outcomes

• Only established outputs and activities

Are your goals SMART?

S – Specific

M – Measurable

A – Achievable

R – Relevant

T – Time-bound

Task

• Work in the same groups and scrutinize the results chain.

What is good and what is not good?

Link between problem tree and LFA/RBM

Problem

Effects

Causes



outcome

Project purpose or

Immediate outcome

Expected outputs or

Outputs


Task • Work in the same groups and develop

– Overall objective (Impact)

– Project purpose (Outcomes)

– Expected outputs (Outputs)

You should base it on your problem tree. (Adjust if needed!)

If in doubt, be SMART!

1) Ukrainian think tank

2) Ukrainian environmental rights NGO

3) Bi-lateral donor

Round up day 1

• Each participant has maximum 15 seconds:

– “Today, the most valuable learning for me was….”

– “Anders, tomorrow you must really improve by…”

– “I’m a bit confused, I hope XXX is clarified better tomorrow”

– “I’m expecting more….”

– “Before I go to bed, I’ll be thinking about….”

Key points day 1

• Shift in focus from activities to results

• Results chain – outcome results

• Define your problem

• Link results to your problem – be specific

Agenda day 2

• Indicators

• Risks

• Budget

(day 3 Reporting and Monitoring)

Example of framework Intervention

logic Target group

Indicators Baseline values

Risks Risk mitigation

Overall objective (Impact)

Project purpose (Outcome)

Expected outputs (Outputs)

Target groups

• Often omitted/forgotten not specific

• Should be as specific as possible

• Primary target group

• Secondary target group

• (Ultimate target group)

Task

• Identify target groups for your established goals. Do you need to re-phrase them?

Setting indicators

• Few and carefully selected

• Not contain targets

• Objectively verifiable (preferably)

• Not necessary require a numerical

• (Linked to a source of verification – method)

• Would normally require a starting point/baseline to measure against

Types of indicators

Input -

Activities

Output

Immediate outcome


Ultimate outcome - Impact

Measure changes in knowledge and

behaviour

Measure changes in status Measures

numbers (trainings,

participants, activities etc.)

Common mistakes when setting indicators

• To many

• Not relevant to the goals

• Contain targets

• Not possible to measure

• Not possible to establish baseline

• Not objective

Task

• Work in the same groups and scrutinize the indicators.

What is good and what is not good?

Task

• Work in the same groups and develop indicators with sources of verification to your Intervention logic in your LFA.

Test questions:

1) Does it measure your goal/objective?

2) Is it possible to measure?

3) Can a baseline be established?

What is Risk Management? • Important component in RBM

• To identify and manage potential problems that could undermine your organisation or your projects

• To integrate the risks into planning, implementation and learning processes.

Why is Risk Management

important? • All projects will encouter problems and have risks! –

Be open with them!

• Make decisions on how to move forward

• Avoid failiures

• Improve results

• Handle changes in your environment

Input


•Internal risks

•Few assumptions

Output


• Internal risks

•Few assumptions

Immidiate outcome

• Likely to happen if outputs are produced

• External risks

•Basic assumptions


• You are making a valuable contribution

• External risks

•Assumptions required

Ultimate outcome/Impact

• You are hopefully making a contribution

•External risks

•Other factors involved

Results chain

Risk mitigation strategies

• Two general strategies

a) Reduce probability

b) Reduce impact

– Preventive

– Corrective

– Directive

– Detective

• Be concrete! – What can you do!?

Risk mitigation strategies - example Risk: Corruption within the organization (internal strategic risk)

• Preventive control Develop Anti-corruption policy that all employees will have to sign. Payments above 10 000 UAH must be signed by Executive Director AND Board Chairman. • Corrective control Immediately contact the police. Suspend person from handling money. • Directive controls Have guidelines in place that describes which procedures that should be taken by whom in case of suspicion of corruption. • Detective control Carry out an external financial audit every year.

Task

• Identify risks in your project – internal or external risks?

• Identify mitigation strategies – do they reduce probability or impact?

“There are known knowns; there are things we know we know.

We also know there are known unknowns; that is to say we know there are some things we do not know.

But there are also unknown unknowns – there are things we do not know we don't know.”

Donald Rumsfeld

Rumsfeld Matrix

Pettersson Matrix

Side effects Positive Negative

Known Include in project

planning/results chain

Include in project planning or risk

mitigation actions

Unknown Document success

stories Document and/or

adapt project

Budgeting

• RBM is a shift in focus from activity-based to results-based....

• ...budgeting should then follow the same logic!

Demands on a budget

• Support achievement of results

• Allow for follow-up on how funds have been spent

• Disallow misuse of funds (level of detail?)

Task

• Study the two selected budgets – Are they supporting:

– Support achievement of results

– Allow for follow-up on how funds have been spent

– Disallow misuse of funds

Highlight good and bad things!

Task

• Design a budget structure for the project on Water and Sanitation – Make sure it:

– Support achievement of results

– Allow for follow-up on how funds have been spent

– Disallow misuse of funds

Round up day 2

• Each participant has maximum 15 seconds:

– “Today, the most valuable learning for me was….”

– “Anders, tomorrow you must really improve by…”

– “I’m a bit confused, I hope XXX is clarified better tomorrow”

– “I’m expecting more….”

– “Before I go to bed, I’ll be thinking about….”

Key points day 2

• Linking indicators of success with results

– Neutral (no targets)

– Be possible to measure

• Linking internal and external risks to results

• Risk mitigating actions to reduce impact or probability

• Budget that should support achievement of results

Baseline values

• Always required for an indicator – Where do you start?

• Can often be 0 or “does not exist”.

• Should be established in planning process or beginning of a project.

Creating a monitoring framework

• Monitoring of results is a key component of RBM

• Follow-ups on the developments of the indicators

• Sets frequency, methods and responsibilities

• Meta-plan – monitor and act on the plan…

Task

• Create a monitoring framework for a selected project.

• Framework will be given to another group for improvement.

• Framework will be sent to a thrid group to ask questions.

Reporting • RBM is a shift in focus from activity-based to results-

based....

• ...reporting should then follow the same logic!

– Information on progress on the goals/objectivs/results achieved using indicators

– Analyse intended and unintedend results

– Monitoring spending of resources

– Projection if goals/objectivs/results will be met or not

– Follow-up on risks and assumptions

– Learning lessons and integrating them into decisions

Task

• Discuss in your group for possible headings for a Progress Report to donor for a project.

Link between projects and program



outcome

Project purpose or

Immediate outcome

Expected outputs or

Outputs




outcome

Program purpose or

Immediate outcome

Expected outputs or

Outputs

Program Project

Link example

Support to organizational development

of NGOs

Increased legitimacy and

effectiveness of NGOs in the

country

NGOs influence government

decisions

Development of strategic

plan

NGO identifies area of

influence and implements

activities

Examples of change



outcome

Program purpose or

Immediate outcome

Expected outputs or

Outputs

Program



outcome

Project purpose or

Immediate outcome

Expected outputs or

Outputs

Project

Discuss in groups

• In what way is this model and theory realistic?

• How can IRF benefit from this model for its programs?

• What needs to be done in that case?

• Reflections? – Was expectations met?

• Fill in evaluation!

• ... RBM guide will come…

• THANK YOU!

Documents

Results-Based Management (RBM) training