Understanding Outcome Measures David McPhee Communities Analytical Services

Understanding Outcome Measures

David McPhee

Communities Analytical Services

Outline

• Background to outcomes approach• Can you identify an outcome?• Examples of different measures• Linking outcomes and indicators• Considering your data• Menu of local indicators• Support• Questions

Background

• The new administration came to power in 2007 and stated that it would focus on a set of high level outcomes

• The National Performance Framework outlines the 15 National Outcomes which would sit below the Purpose of the Scottish Government

Background

Background

Background

• Concordat between the Scottish Government and Local Authorities

• Removal of ring fencing and the creation of Single Outcome Agreement

• First 2 rounds of SOA’s already carried out with many lessons learned between round 1 and round 2

Background

• The SOA’s will drive the local housing strategy with the key housing objectives forming a core set of outcomes

• The LHS’s will therefore be a key driver in delivering SOA’s and also delivering the SG outcomes (particularly 7 and 10)

• Template already created for recording outcomes, indicators, targets and actions.

• Examples will follow in later sessions

Outcomes and Indicators

• Not synonymous!

• Outcomes are the positive improvements to life we are all working to achieve

• Indicators are numbers that should indicate whether an outcome is being achieved

Can you identify an outcome?

• “% of housing advice cases resulting in the prevention of homelessness”

• Outcome?

• Indicator?

• Target?

• Action?

Can you identify an outcome?

• “% of housing advice cases resulting in the prevention of homelessness”

• Indicator

Can you identify an outcome?

• “Identify land and unsold private properties for the development of affordable housing”

• Outcome?

• Indicator?

• Target?

• Action?

Can you identify an outcome?

• “Identify land and unsold private properties for the development of affordable housing”

• Action

Can you identify an outcome?

• “Improved housing choices in the size and types that people want in the areas they want to live and that they can afford”

• Outcome?• Indicator?• Target?• Action?

Can you identify an outcome?

• “Improved housing choices in the size and types that people want in the areas they want to live and that they can afford”

• Outcome

Can you identify an outcome?

• “Increase the number of social rented properties meeting the SHQS standard by 10% each year”

• Outcome?• Indicator?• Target?• Action?

Can you identify an outcome?

• “Increase the number of social rented properties meeting the SHQS standard by 10% each year”

• Target

Can you identify an outcome?

• “Potentially homeless households are able to sustain settled accommodation”

• Outcome?• Indicator?• Target?• Action?

Can you identify an outcome?

• “Potentially homeless households are able to sustain settled accommodation”

• Outcome

Examples of Different Measures

• Outcome:– The overall impact which is trying to be made– Not necessarily easily measured

“People will have better access to appropriate and affordable housing”

Examples of Different Measures

• Indicator:– A measure which provides an indication of

whether the outcome is being achieved– Measurable, but does not necessarily

measure the impact of the full outcome

“Number of affordable houses provided in a given year”

Examples of Different Measures

• Action:– The practical steps which will be taken to

deliver the outcomes (and indicators)

“Review lettings procedures and develop new protocols with RSL’s”

Examples of Different Measures

• Target:– More specific version of an indicator with a specific

aim– Targets should where possible be Specific,

Measurable, Achievable, Relevant and Time Specific (SMART)

“Increase the % of housing advice cases resulting in the prevention of homelessness by 16% by 2011/12”

Examples of Different Measures

• Inputs and Outputs:– Measures of what are being put into and delivered by

“a system”– Outputs and Inputs can look like outcomes but they

do not tell you about the impact of actions only the actions themselves

– One persons output could possibly be another persons outcome

“The number of people being dealt with by homelessness teams in LAs”

Linking Outcomes and Indicators

• Outcomes not always possible to measure in an easy way

• Indicators should provide true indication of whether outcome is being achieved (not just a small part)

• Good indicators will show success of a variety of programmes/sectors/policies

• Depending on outcome it may need a number of indicators to determine success

Considering Your Data

• Key Considerations for your data:– Is it survey or administrative data?– What is the coverage?– Is it consistent over time?– Is it quality assured?– Can you compare across different areas?– Is it precise enough?

Menu of local indicators

• Produced by the Improvement Service – current versions based on commonly used indicators in SOAs not all of which are high quality

• BUT….• SOLACE are leading an Improving Local

Indicators Project • First output will be Version 4 of the menu – due

to be published in October

Improving Local Indicators Project• Main aim of project is to improve data available

for measuring outcomes at a local level• The project will continue for a couple of years • Project involves SOLACE, Scottish Government,

Improvement Service, LARIA, Audit Scotland, and many many more…

• Everyone can get involved• Details available here:http://www.scotland.gov.uk/Topics/Statistics/scotstat/analystsnetwork

Support• Scottish Government stats website:

– www.scotland.gov.uk/stats• CHMA Website and Contacts:

– www.scotland.gov.uk/chma– chma@scotland.gsi.gov.uk

• SCOTSTAT Analysts Network– Analysts.Network@scotland.gsi.gov.uk– www.scotland.gov.uk/scotstat

• Guidance on measuring change and monitoring targets– http://www.scotland.gov.uk/Topics/Statistics/scotstat/a

nalystsnetwork/MCMTguidance

Support

• Outcome Template– Worked example:

http://www.scotland.gov.uk/Resource/Doc/1035/0062216.pdf

– Blank Template: http://www.scotland.gov.uk/Resource/Doc/1035/0062275.doc

• Future Events – More information to follow

Questions?