Improvement Forum

A webinar series for QI Managers, Nurse Leaders and others supporting healthcare improvement

in Wisconsin’s hospitals

June 2012

How do you know you have improved?

Our Topic for June 2012

Travis Dollak, Quality CoordinatorTom Kaster, Quality Coordinator

Today’s Agenda

3

• Introduction• Content Sharing

– Measurement is for Learning– What to measure – Measuring what matters– The problem with “drift”– Resources

• Discussion Questions

Focus on Measurement

Why measure?The main reason for conducting an improvement

project is to achieve results, no matter the issue or topic.

And how do we know we have achieve a desired result that can be proven to others?

We must demonstrate change from a baseline, or initial measurement, and assess the degree of change after an intervention.

4

70 Million Americans Benefit from Quality Measurement

• 96% of heart attack victims were prescribed beta-blocker treatment in 2005, up from 62% in 1996*

• 77.7% of children enrolled in private health plans received all recommended immunizations, up 5% from 72.5% in 2004*

• Evidence-based guidelines from the American College of Cardiology and the American Heart Association have reduced mortality among patients who have had a heart attack

* National Committee for Quality Assurance

Areas of Measurement

Relies on the actual execution of the PDSA cycle

Disclaimer information here… 6

Aims

Measurement

Change ideas

Testing ideas before implementing changes

Process Measures

Diet Driver Diagram

7

Reducing Falls Driver Diagram

Disclaimer information here… 8

Poll Question 1: Process Measures

• How often does your facility measure processes for your improvement projects?– Always– Almost Always – Sometimes – Never

Disclaimer information here… 9

How to develop process measures

• Ask: – How does the work get done?– How would I know?– What is important to know?– What is the easiest way to know?– What is already collected? Is it good enough?

Real Word Example – Losing Weight

11

• Outcome Measure: I want to loose 10 lbs by July 4, 2012– Stepping on the Scale can lead to moderate

improvement but will plateau

• Process Measures: To lose 10 lbs by July 4th, I will measure:– Calorie intake – Analyze what I eat

– Time spent exercising – Analyze how often and what type of exercise

Clinical Example – Falls Prevention

• Outcome Measure: Reduce all falls by 50% by 12/31/2013

• Process Measure: To reduce all falls by 50% we will measure:– The prevalence of a daily fall risk assessment being

completed– How often the care plan identified in the risk

assessment is in place and adhered to

12

Poll Question 1 Results: Process Measures

• How often does your facility measure processes in your improvement projects?– Always– Almost Always – Sometimes – Never

Disclaimer information here… 13

Aspect Improvement Regulatory Research

Aim Improve care Compare, reassure, spur change

New knowledge

Methods

Test Observable

Yes N/A. Evaluate current performance

Test blind or controlled

Bias Accept stable bias Adjust data to reduce bias

Design to eliminate

Sample Size Just enough data, small sequential samples

N/A. Report 100% Just in case data

Hypothesis Flexible

No. Revised as learn and test

No hypothesis Fixed hypothesis

How to determine improvement

Run or control charts No focus on change Hypothesis, Statistical tests: F-test, t-test, chi square, p value

Testing Strategy Small sequential tests No tests 1 large test

Data confidential Data used only by those involved in improvement

No subjects. Data is for public

Subjects protected

Measuring Effectively

• Seek usefulness, not perfection• Use sampling• Plot data over time• Don’t wait for the information system

15

Usefulness, Not Perfection

• Usefulness means measuring just enough to tell you what direction you are headed

• Perfection can lead to paralysis by analysis • State/Federal Criteria can cause us to focus

efforts on perfect data and less on improvement

16

Keeping measurement simple

• Use Simple Visuals• Use Tic and Tally Sheets• Make your measures easyto track on a daily or weeklybasis

Why Sample?

Benefits:• Lower cost• Saves time (receive information faster)• With smaller data set, its easier to improve the

accuracy/quality of the data

Example:Sample 20 pts/month using IHI trigger tool to identify ADEs yields

the same results as sampling entire population

http://www.1000livesplus.wales.nhs.uk/sitesplus/documents/1011/T4I%20%284%29%20How%20to%20use%20Trigger%20Tools%20%28Feb%202011%29%20Web.pdf

18

Displaying Data Over Time

Why use graphs & charts?

Graphing and charting are useful tools when there is a lot of data to display, or a simple comparison of data in a table is not adequate to explain changes in the data.

Some methods to display data are more appropriate than others.

19

Why be visual?

20

# of ADEs per 1,000 Doses # of ADEs per 1,000 Doses

21

Remember to “tell the story” about how you achieved these results….

Poll Question #2: Annotated Run Charts

• How comfortable are you with developing and using annotated run charts to measure your improvement projects?– Very comfortable – Somewhat comfortable but would like more help– Not comfortable and need more help– What is an annotated run chart?

Disclaimer information here… 22

When Reaching Your Goal

• Measurement does not stop• Staying at ‘zero’

– Continuous monitoring• Monitoring early warnings

– New orientation*– Revisit training*

23

Summary

• Measure to learn – use process measures• Seek usefulness, not perfection• Display your data in a meaningful way• Connect your driver diagram to your process

measures• Avoid drift – continuously monitor

• 24

Questions and Answers

What can we learn from each other?

Stephanie Sobczak, MS, MBAManager QI, Wisconsin Hospital Association

Next Month’s Topic: Accelerating Change through small tests