Improving Care: More Method, Less Uncertainty,

Impact summit

30th October 2013

Professor Moira LivingstonClinical Director of Improvement Capability NHS Improving Quality

Housekeeping

Starting the journey

The journey so far…EVENT Improving Care: More Method, Less Uncertainty

Friday 6th September

The first in a series of measurement master-classes for senior clinicians

Dr Bob Lloyd, Institute for Healthcare Improvement US, Professor Moira Livingston, NHS Improving Quality, Professor Sir Bruce Keogh, NHS England, Julian Hartley, NHS Improving Quality, Dr Maxine Power, Salford Royal NHS Foundation Trust

WEBINAR Thursday 10th Oct

Different national approaches - how to use national data to drive improvement at all levels

Dr Veena Raleigh, Kings Fund, Göran Henriks, Jönköping County Council, Sweden, Prof Jonathon Gray, Dr Mataroria Lyndon, Counties Manukau Health, New Zealand

WEBINAR Thursday 17th Oct

19 Delegates

Different national approaches – mortality, exploring how to use complex indicators to drive improvement

Dr Bob Lloyd, Institute for Healthcare Improvement US, Dr Anna Trinks, Jönköping County Council, Sweden

WEBINAR

Wednesday 23rd Oct

Different national approaches - improvement and transparency

Dr Carol Peden, Royal United Hospital Bath, Alide Chase, Diane Waite, Kaiser Permanente, US

Time Topic Lead

0930-0945 Welcome, introductions and overview of the day

Professor Moira LivingstonClinical Director of Improvement Capability NHS Improving Quality

0945-1000 View from the top Professor Sir Bruce Keogh

National Medical Director, NHS England

1000-1100

The strategic measurement for improvement journey• Choosing the right measures

Mike Davidge with Dr Bob Lloyd (15 min video) Dr Maxine Power Dr Veena Raleigh

1115-1130 Break

1130-1230

The strategic measurement for improvement journey• Collecting good data• Making sense of data

Mike Davidge with Dr Maxine Power Dr Veena Raleigh

1230-1310 Lunch

1310-1430 Knowledge Exchange: Making it happen• Details on your desks Mark Outhwaite

1430-1550 Steering the measurement journey: what next? Mark Outhwaite

1550-1600 Summary and Closing Professor Sir Bruce Keogh

Shape of the day

Purpose of the impact summitThe key aims:

• Reflect and review learning and implications from the master-class so far

• Build depth of knowledge

• Discuss and identify how to make improvements in our measurement systems– based on better / more informed decision making

• Promote understanding of the difference between measurement for improvement and for other purposes

• Share and embed practical techniques for choosing measures, applying measures and interpreting measures

We will do this by:

• Case studies of real world examples, with opportunity to discuss and question

• Providing interactive sessions to work through some personal measurement challenges, to identify some actions and next steps

• Create the opportunity to identify further support needed to take for forward a measurement for improvement system, culture and practices

Note: this course will be eligible for CPD points, information to be circulated after the event

Speakers for this morning

Professor Sir Bruce KeoghNational Medical Director, NHS England

Mike DavidgeDirector (Measurement), NHS Elect

Veena S Raleigh PhDSenior Fellow, The King’s Fund

Maxine Power PhD, MPHDirector of Innovation and Improvement Science, Salford Royal NHS Foundation Trust and Managing Director of Haelo

Knowledge Exchange Speakers• Mel Varvel, Improvement Manager, NHS Improving Quality

• Preventing People from Dying Prematurely: GRASPing the Measurement Nettle

• Dr Frances Healey, RGN, RMN, PhD, Senior Head of Patient Safety Intelligence, NHS England & Matthew Foggarty, Patient Safety, NHS England

• The genie is out of the bottle: when Measurement for Improvement is used for other purposes

• Clare Howard, MRPharmS, Deputy Chief Pharmaceutical Officer NHS England• Developing metrics for safer medication practice

• Dr Carol Peden, Quality Improvement Fellow-Health Foundation and Consultant in Anaesthesia and Critical Care Medicine, Royal United Hospital Bath

• Mortality Reviews

• Martin McShane, Director (Domain 2) Improving the quality of life for people with Long Term Conditions, NHS England & Professor Alistair Burns, National Clinical Director for Dementia, NHS England and The University of Manchester

• Dementia

Professor Sir Bruce KeoghNational Medical DirectorNHS England

Mike DavidgeDirector (Measurement)

NHS Elect

Using Poll EverywhereLive feedback and polling

Either

Text: mfimp to 07624806527 to link your phone to the session

Then all you do is send poll responses to that number as a normal SMS/textWill not work if you withhold your number

OrPoint your smartphone/tablet browser at

www.pollev.com/mfimpTo participate in the polls

Wifi: MMCNHSIQ – no passwordNo premium costs – just contained within your normal contract rates

Question 1

Question 2

A word from our teacher

• Bob Lloyd reminds us briefly what he covered on 6th September

• We will be revisiting some of these points this morning with practical exercises

CHOOSING THE RIGHT MEASURESBe clear why you are measuring and the messiness of life

Choosing indicators

Veena Raleigh

Senior Fellow

30 October 2013

Precursors of measurement: clarity about...

Who (audience): providers, commissioners, patients etc

Why (aim):- quality improvement, judgement, research

What (content):- dimension of quality, efficiency- population, service/sector, pathway- unit of measurement

How (process):- definition, data sources- statistical methods- interpretation

Audience for measurement (1)

parliament / government

the NHS:- commissioners- managers - professional staff

patients, families, carers

the public

regulators, auditors

researchers

the media

The appropriate content and presentation formats of indicators for these audiences differ

Audience for measurement (2)

For example:

clinicians need disaggregated, risk-adjusted information at small unit level, benchmarked against peers, and showing trends over time

commissioners want information on outcomes, and quality linked to cost-effectiveness

patients, public want information that is simply constructed, clearly presented, and easy to interpret ie good vs bad

• Judgement:- performance assessment/management - incentivising quality improvement (P4P eg QOF, CQUIN, quality premiums)- supporting patient choice- public accountability

assumes unambiguous evidence of performance, designed for EXTERNAL accountability

Aim of measurement

• Quality improvement:- internal use- benchmarking against peers for feedback and learning

assumes indicators are 'tin openers' for INTERNAL use, designed to prompt further investigation and appropriate action

or

Indicators for judgement Indicators for improvement

unambiguous interpretation variable interpretation possible

unambiguous attribution ambiguity tolerable

definitive marker of quality screening tool

good data quality ‘good enough’ data quality

good risk-adjustment partial risk-adjustment tolerable

statistical reliability preferred but not essential

cross-sectional time trends (SPCs, run charts etc)

punishment/reward learning, change in practice

external control internal control

data for public use data for internal use

stand-alone allowance for context

risk of unintended consequences low risk

Content of measurement (1)

dimension of quality:effectiveness, patient experience, safety ………..timely, access, equity, VfM, care coordination and integration

population group, condition, service

structure, process and outcome indicators: S + P = O

unit of measurement eg commissioner and/or provider

Content of measurement (2)

Indicators for commissioners (CCGs, LAs):- population based

Indicators for providers:- Primary care- Community care- Out-of-hours care- Hospital care (emergency and planned)- Tertiary and specialist care- Mental health care- Palliative care- Social care (residential & home care)

Indicators by population group,

condition

Example: cancer

NHSOF / COIS domain 1 indicators:cancer mortality < 75cancer survival

reducing cancer mortality depends on:reducing cancer incidence ANDimproving cancer survival

these outcomes require improvement in the underlying drivers eg:cancer incidence: preventive measures eg smoking cessation services (process measure)cancer survival: screening, timely referral, treatment rates (process measures), staff capacity/skills and surgical volumes (structure measures)

Cancer (example indicators)

Risk factors and prevention Diagnosis, treatment, end-of-life care

Rates of: - incidence O- smoking prevalence, diet etc IO- population awareness P

- no of smoking cessation clinics S- smoking quitters O

Rates of:- screening P - referrals, diagnostic tests, time to results P- detection rates O- stage at diagnosis O- access, waiting times P- cancers detected at emergency presentation P- surgical volumes S- treatment (surgery, radiotherapy) rates P- information for patients P - length of stay, readmission, mortality rates O- one-year survival: proxy for late diagnosis O- management by a multidisciplinary team P

- staff skills, training S- adherence to guidelines P- access to end-of-life care P- patient experience and wellbeing O- cancer deaths by place of death O- participation in national clinical audits S

PRIMARY OUTCOME MEASURESCancer mortality O

Cancer incidence O Cancer survival O

KeyS=structure measureP=process measuresIO=intermediate outcome measureO=outcome measures

Inequalities

30

If you were in a lift with the rest of your table group could you clearly and briefly describe your aim in a sentence – i.e. the time it takes to travel from one floor to the next?

Write your aim statement down

Share with your table

Aims exercise

Driver Diagrams

Aim

Measurement

Drivers (changes)

What is a Driver Diagram?

• Reinforces the aim statement as the goal • Clarifies the big picture • Identifies primary system components• Identifies projects which will influence • Aids in development of measurement

Most importantly: Helps to articulate the overall aim and avoid missing important system components

33

What are driver diagrams used for?

• Personal improvement projects• Clarification in complex tasks• Project / Programme Management• Strategy, design and execution

Primary Drivers

• Push conceptual thinking • Avoid focus on one area alone • Usually categorical• Abstract• Removal reduces likelihood of success• Projects wrap into them

Secondary Drivers

• Projects• Tasks• Actions • Focus Areas

• Aid allocation of workload• Ensure clarity and focus for testing

Lose

2 stone

by March

2014

Healthy Eating

Measurement & feedback

Exercise

Prevent avoidable complications

(Lifestyle)

• Regular shopping• More fresh fruit• 3 meals per day• No food after 6pm• 2 litres of water per day

• Weekly weight• Measure Inches• Pictures on the fridge• Regular support• Weight record chart updated showing trend

• Daily exercise for a minimum of 20 mins• Measure progress• Identify barriers• Build distractions to help• Add something nice – sauna / jacuzzi• Search for an exercise that suits

• Plan for eating out / weekends• Beer & wine – develop a plan• Know your weaknesses• Habits and patterns• Avoid bad influencers• Encourage contact with supportive

people

My driver diagram for weight loss

Develop & test a

measurement instrument for harm free care from pressure ulcers, falls,

catheters and VTE by

September 2011

Agree Operational Definitions

Develop Technical Capability

Determine how the instrument is used

Determine the level of user satisfaction

• Evidence review• Expert debate / input• Grey areas agreed• Practical use

• Design characteristics• Local, regional, national• Universal platform• Guidelines for use

• Who collects & when?• From where?• What happens after?• How are data used?

• Local users - feedback• Data leads - feedback• Leadership• Senior stakeholders

Outcome : Rate of patient’s harmed by falls

1

Process : training in falls2

Cancer (example indicators)

Risk factors and prevention Diagnosis, treatment, end-of-life care

Rates of: - incidence O- smoking prevalence, diet etc IO- population awareness P

- no of smoking cessation clinics S- smoking quitters O

Rates of:- screening P - referrals, diagnostic tests, time to results P- detection rates O- stage at diagnosis O- access, waiting times P- cancers detected at emergency presentation P- surgical volumes S- treatment (surgery, radiotherapy) rates P- information for patients P - length of stay, readmission, mortality rates O- one-year survival: proxy for late diagnosis O- management by a multidisciplinary team P

- staff skills, training S- adherence to guidelines P- access to end-of-life care P- patient experience and wellbeing O- cancer deaths by place of death O- participation in national clinical audits S

PRIMARY OUTCOME MEASURESCancer mortality O

Cancer incidence O Cancer survival O

KeyS=structure measureP=process measuresIO=intermediate outcome measureO=outcome measures

Inequalities

REDUCE MORTALITY

FROM CANCER IN ENGLAND

BY XX% BY MARCH 2016

PRIMARY PREVENTION

SECONDARY PREVENTION

SERVICE OPTIMISATION

END OF LIFE AND SOCIAL CARE

• Lifestyle • Genetics • Campaigns• Social determinants

• Screening • Primary care • Access to L2/3 service • Lifestyle change• Medicines optimisation

• Value driven• Quality greater than cost• Equity in access• Excellent experience

• Cross sector working• Hospice & faith • Seven day HSC service• Equipment• Pain management

Cascading drivers

1

2

3

1

2

3

Limitations of driver diagrams

• Not a perfect science

• Two dimensional & simplistic

• Working schematic – requires amendment

• Interplay between drivers

• Contribution of each driver is not equal

Question 3

Question 4

COFFEEPlease take only 15 minutes

COLLECTING GOOD DATAThe measurement journey

Introducing the Measures checklist

Define measures

An operational definition is a description, in quantifiable terms, of what to measure and the steps to follow to measure it consistently

Example definitionMeasure name:

DNA rate for clinic AWhy is it important? (Provides justification and any links to organisation strategy)

We need to ensure that the clinic is not disrupted by having unexpected gaps in the clinic schedule. The policy for this clinic is to offer another appointment which means that other patients may be disadvantaged if we have too many patients being rescheduled.Who owns this measure? (Person responsible for making it happen)

The outpatient clinic manager

Measure definitio

n

What is the definition? (Spell it out very clearly in words)

The percentage of patients booked to attend clinic A who did not attend for their appointment and no warning was received at the clinic before it started.What data items do you need?

The number of patients booked to attend clinic (B) and the number of patients who failed to attend without warning (F)What is the calculation?

100 x DNA patients (F) / Booked patients (B)Which patient groups are to be covered? Do you need to stratify? (For example, are there differences by shift, time of day, day of week, severity etc)

All patients booked into clinic

Collecting data

• What – All patients, a portion or a sample?

• Who – collects the data? • When – is it collected

– real time or retrospective?

• Where – is it collected?• How – is it obtained

– Computer system or audit?

You need a plan which you test using PDSA cycles

Checklist exercise

• Complete page one and collect on page two of the measures checklist provided - for a measure that you are using or are planning to use

• Share with your colleagues

You have 15 minutes

Question 5

Question 6

MAKING SENSE OF DATAVariation

Variation exercise

• Using the materials provided make the best paper aeroplane you can

• Put your initials on it

You have 15 minutes

When instructed - throw your planes!

Aeroplanes fly different

distances

Problem

Equipment People Procedures

Materials

Causes

Types of paper e.g.

card, tracing paper,

No clear instructions provided

Some tables had scissors, rulers to help

Skills / ideas

Throwing styles

Fishbone diagram

Environment

Air /Wind

CommonCause

Special Cause

Classifying variation

Stable in time and therefore relatively

predictable

Irregular in time and therefore unpredictable

The paper used

Persons technique

Design of the plane

Water spill

Mike’s plane

“There are different improvement strategies depending of which type of variation is present (common cause or special cause), so it is important for a team to know the difference.”

Michael GeorgeChairman and CEO of George Group

Consulting

Why classify variation?

Question 7

Question 8

Instructions for the afternoon session

The Knowledge Exchange Carousel

• After lunch you will be directed to move direct to a Knowledge Exchange Carousel ‘Pod’ with the same number as your table number

• You will rotate through 3 ‘Pods’ at 25 minute intervals• In each Pod you will discuss a case study presented by a speaker• After the third Knowledge Exchange session you will remain in

the Pod for the next task

Knowledge Exchange Speakers• Mel Varvel, Improvement Manager, NHS Improving Quality

• Preventing People from Dying Prematurely: GRASPing the Measurement Nettle

• Dr Frances Healey, RGN, RMN, PhD, Senior Head of Patient Safety Intelligence, NHS England & Matthew Foggarty, Patient Safety, NHS England

• The genie is out of the bottle: when Measurement for Improvement is used for other purposes

• Clare Howard, MRPharmS, Deputy Chief Pharmaceutical Officer NHS England• Developing metrics for safer medication practice

• Dr Carol Peden, Quality Improvement Fellow-Health Foundation and Consultant in Anaesthesia and Critical Care Medicine, Royal United Hospital Bath

• Mortality Reviews

• Martin McShane, Director (Domain 2) Improving the quality of life for people with Long Term Conditions, NHS England & Professor Alistair Burns, National Clinical Director for Dementia, NHS England and The University of Manchester

• Dementia

Sharing your learning

• At the end of the Knowledge Exchange you will remain in your last Pod

• Using the A0 poster template rapidly brainstorm the Barriers and Drivers in the current environment for each step in the measurement process

• Identify your top 2 Barriers and top 2 Drivers (dot vote if necessary)

• Transfer them to your Action Planner Driver Diagram

Action Planning

• Identify the actions you could take collectively as a senior leadership cadre to address the barrier or driver

Or• The support you need as a

senior leadership cadre to address the barrier or driver

Feedback

• One barrier or driver and the associated actions• One headline – if a journalist had been in the Pod with you what

would be the headline they would have written

Personal Action Planner

Afternoon thoughts and reflections

Lunch1230 - 1310

Knowledge Exchange

Feedback

Professor Sir Bruce KeoghNational Medical DirectorNHS England

The Improving Care: More Method, Less Uncertainty,

Impact summitFurther details about the webinar series :

www.nhsiq.nhs.uk