Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
2C – Using Data to Improve
Advanced Measurement for
Improvement Seminar
March 20-21, 2017
The Data CycleMeasures identified
and defined
Data collection process defined,
tested
A P
DS
Data acquisition
Data Entry
Storage,
aggregation,
analytics
Reporting
Interpretation
and application
Data Acquisition
Operational IT systems gather granular data on
standard processes
Clinical: Nursing, EHR, Labs, Pharmacy, etc.
Administrative: Billing, scheduling, etc.
Supplemented by systems to gather clinical
process data
Institutional
Ad-hoc
PDSA data is real-time, front-line, manual.
Interpretation and Application
Who needs to know what?
What level of information
How often? How soon?
Will the audience interpret the measures
appropriately?
How will you train them?
How will you keep them consistent?
Will process owners know how to respond?
How will you coach them?
Scoville, R., Little, K. et al. (2014). Sustaining Improvement. IHI White Paper. Cambridge, Massachusetts: Institute for Healthcare Improvement; 2016. IHI Whitepaper. Cambridge, MA, Institute for Healthcare Improvement. Avaliable at IHI.ORG.
A Management System Architecture
Dat
a
Dat
a
Gu
idan
ce
Align
men
t
Lean Management System
Ideal management system to support value-based
production:
Leader standard work
Visual controls
Daily accountability and planning
Respect for people who do the work
Unity of purpose
Strategic intent, operational goals, and system
views must be vertically aligned!
Mann, D. (2010). Creating a Lean Culture: Tools to sustain lean conversions. Boca Raton, FL, CRC Press.
Systems Hierarchy
Macro-systems
e.g. trust, facility, region
Meso-systems
e.g. division, clinical dept,
pathology, IT
Microsystems
e.g. unit, clinic, surgical team
‘Catchball’ P8
Source: Virginia Mason Health System
“Catchball” process aligns levels
Reporting Improvement
Senior Leaders, Boards, Executive
Sponsors (Macro-system)
Percent of target sites engaged in key
improvement initiatives
Percent of target population exposed to interventions
Phase of intervention by site or project: Plan? Pilot?
Implementation? Spread?
Time-series family of key ‘current care’ and ‘population’
measures by site, with goals
Comparison to ‘best practice,’ national/regional datasets,
comparative benchmarks
Comparison to control sites
Source: Keith Mandel MD
Reporting Improvement
Improvement Initiative Leaders, Department
Heads, etc. (Meso-system)
Time-series dashboard of all
‘current care’
and ‘population’ measures by site, with goals.
Key current care measures segmented by unit, patient
sub-population, risk groups. Measures matched to
domain of improvement work.
Current QI capability of site leaders and teams, other
‘foundational’ requirements (e.g. registry, EMR)
Degree of involvement/effort of QI teams
Data quality
Source: Keith Mandel MD
Reporting Improvement
Front-Line (Micro-system) Teams
Time-series dashboard of all
‘current care’
and ‘population’ measures by site, with goals.
Key current care measures segmented by unit, patient
sub-population, risk groups. Measures matched to
domain of improvement work.
PDSA measures for current process change testing.
Data quality
Source: Keith Mandel MD
Exercise
For Your Own Project:
Identify the key data ‘customers’ and their
relationship to (or role in) the project?
What is their degree of involvement in the
project and familiarity with QI methods?
How can you leverage measurement to
maximize their engagement in the work?
What information are they receiving now? Is it
timely and accurate?
What are your ideas for improving data
feedback?
Individuals or Groups Role in Project
Degree of
Involvement
(1=never – 5=daily)
Comprehension of
Methods and Goals Ideas for Engagement
SENIOR LEADERS, BOARDS, SPONSORS
IMPROVEMENT INITIATIVE LEADERS
FRONT LINE IMPROVEMENT TEAMS
Key Data
Customers
Currently Receiving
Information?
Time Lag,
Data Quality Ideas for Improvement
Percent of target sites engaged in key
improvement initiatives
Percent of target population exposed to
interventions
Phase of intervention by site or project:
Plan? Pilot? Implementation? Spread?
Time-series family of key ‘current care’ and
‘population’ measures by site, with goals
Comparison to ‘best practice,’
national/regional datasets, comparative
benchmarksComparison to control sites
Time-series dashboard of all ‘current care’
and ‘population’ measures by site, with
goals.
Key current care measures segmented by
unit, patient sub-population, risk groups.
Measures matched to domain of
improvement work.
Current QI capability of site leaders and
teams, other ‘foundational’ requirements
(e.g. registry, EMR)
Degree of involvement/effort of QI teams
Data quality
Time-series dashboard of all ‘current care’
and ‘population’ measures by site, with
goals.
Key current care measures segmented by
unit, patient sub-population, risk groups.
Measures matched to domain of
improvement work.
PDSA measures for current process change
testing.
Data quality
SENIOR LEADERS, BOARDS, SPONSORS
IMPROVEMENT INITIATIVE LEADERS
FRONT LINE IMPROVEMENT TEAMS
Are They
Being
Served?
Aligning Measurement for
Quality Control and
Improvement
Alignment
The measure ‘cascade’
Strategic measure
deployment
Dynamic & Static Views of a Process
0
10
20
30
40
50
60
70
80
90
100
3/1/
2008
3/8/
2008
3/15
/200
8
3/22
/200
8
3/29
/200
8
4/5/
2008
4/12
/200
8
4/19
/200
8
4/26
/200
8
5/3/
2008
5/10
/200
8
5/17
/200
8
5/24
/200
8
5/31
/200
8
6/7/
2008
Control charts show
change over time
Histogram, radar charts,
etc. show cross-
sectional ‘snapshots’ at
a point in time 0
2
4
6
8
10
12
14
16
18
20
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
Caldwell, C. (1995). Mentoring Strategic Change in Health Care: An Action Guide. Milwaukee, ASQC Quality Press.
Strategic Intent and Strategic Measures
Short Term – This year’s goals
Cash flow & cost reduction
Productivity, net revenue, receivable days
Meet current clinical targets
CHF readmits
Mid Term – Next year’s goals
Increase market share
Customer satisfaction, complaints
Longer Term – 3 year goals
Increase organization agility
# Improvement projects, improvement project cycle time
Caldwell, C. (1998). Results-driven management: Strategic quality deployment. The handbook for managing change in health care. C. Caldwell. Milwaukee, ASQ Quality Press: 37-87.
Strategic Intent and Strategic Measures
Dimensions of system performance
Rate of innovation and improvement
Reduce non-value-added costs
Improve cash flow
Increase customer satisfaction
Progressively integrate the organization as a
system (additional business units, standard
practice, IT)
Vertical
Horizontal
Source: Caldwell, C. (1998)
West Paces Ferry Quality Dimensions c.1992
Productivity Sales Development Customer Loyalty
Source: Caldwell, C. (1998)
West Paces Ferry Level 1 Measures c.1992
Productivity Sales Development Customer Loyalty
Cost per member per month1
Target doctor recruits Net revenue from new products
Days to resolve a complaint
Cash flow percent prior year (growth)
Corporate contracts QI projects completed Health status – quality of life
Cost of poor quality Public awareness of brand
Employee satisfaction – open communication
Patient brag
Income percent prior year
Market share QI project percent complete
Operating expense percent prior year
Readmit percent2
1WPF was an integrated delivery system 2Quality target for corporate strategy
Source: Caldwell, C. (1998)
Kano – Customer Judgment as a Basis for
Performance Appraisal
Kano, N. (1984). "Attractive Quality and Must-Be Quality." Journal of the Japanese Society for Quality Control 14(2): 39-48.
I
II
III
III. Delightful. Unexpected and
exciting
II. Normal. A satisfactory
experience
I. Expected. Below this level repels
customers
Radar Chart: Quality Dimensions
I
II
III
Productivity
Productivity
Development
Sales
Patient BragCost of poor
quality
Source: Caldwell, C. (1998)
Suboptimized Systems
Source: Caldwell, C. (1998)
Level 1 Radar Chart in Action
Source: Caldwell, C. (1998)
The Information Cascade
Macro-systems
e.g. system, trust, facility, region
Meso-systems
e.g. service line, division,
clinical dept, pathology, IT
Microsystems
e.g. unit, clinic, surgical team
Levels of Measurement
1 - Strategic measures
• Derived from strategic
dimensions (e.g. Balanced
Scorecard)
• Target current, mid, long term
goals
• Align with strategic plan
2 - Division measures
• Structural units comprising key
organizational functions
• Most L3 are operational
‘management indicators’
3 - Business process indicators
• Measures of high-level process
effectiveness and efficiency
• Components may have different
owners
4 - Core mainstay and support process
indicators
• Single process owner
• This is where QI work is focused
(1)
(2)
(3)
(4)
(Levels)
Data
flo
w
Man
agemen
t View
Macro
Micro
Meso
Micro
Admin errorsper 100 scripts
Wrong patientper 100 scripts
% errors intercepted
Non-path orders% cases
Allergy alertsper 100 scripts
Medicationerrors % dsch
Prescribing errorsper 100 scripts
Moving up:• Cause-effect theory (e.g. driver diagram, clinical evidence)• Observed correlation (e.g. regression models)• Aggregation
Data flow to more macro levels
Management ‘line of sight’
‘Line of Sight’ Measures
Source: Caldwell, C. (1998)
# Calls to rapid response team
Environment
Hand hygiene compliance
‘Line of Sight’ Measures
Percent inpatient mortality
Compliance with “bundles”
% Surgical bundle
% Pressure ulcer bundle
% CL bundle
% VAP bundle
Hospital Acquired Infection
rates
% Sepsis bundle
L1 L2 L3 L4
AggregationDriver Model
Observed
correlation,
clinical
evidence
Aggregation Methods
• Individual Patient Data to Population
Average, median, distribution of patients: Cost, Time, Scores,
etc.
Percent conforming: Protocol-driven care
Count of events: Falls, Mortality, ADEs, etc.
• Micro to Meso to Macro
Numerators and denominators summed across units
Overall averages, medians
Average unit performance
• Aggregating Across Different Measures
Staging systems
Build composite measures or indices
Staging System
Griffin, F. A. and D. C. Classen (2008). "Detection of adverse events in surgical patients using the Trigger Tool approach" Qual Saf Health Care 17(4): 253-258.
Discussion
Consider how the aim of your project fits into
your organization’s strategic goals:
Do the key measures that track the success of
your project fit into a measure cascade within
the organization? What would that look like?
Do you have recommendations for your client
regarding a strategy for operational
measurement?
Dashboards
Examples
Why not ‘Red-Yellow-
Green’?
An ideal alternative
Who Uses Hospital Dashboards?
“Shorter, more focused dashboards that are reviewed on a
frequent basis are associated with higher performance.
According to the results of this dashboard analysis,
hospitals that use dashboards with fewer measures are
more likely to be in the high-performance group, suggesting
that higher-performing hospitals have developed
dashboards that focus on areas they see as critical for
quality. Furthermore, performance data are more
actionable when such data are consistently reviewed by the
board on a relatively frequent basis.”
Kroch et al. (2006)
A Common Type of Dashboard
Source: Provost, Murray & Britto (IHI Forum 2010)
This ‘goal-driven’ view does not provide an actionable view of system dynamics
How is Time to 3rd Next Available Doing?
Source: Provost, Murray & Britto (IHI Forum 2010)
How is Perfect Care Doing?
Source: Provost, Murray & Britto (IHI Forum 2010)
How Is Error Rate Doing?
Source: Provost, Murray & Britto (IHI Forum 2010)
Alternative
A view where
Each measure is displayed on an appropriate control chart
All control charts are on same page to see the whole system
Advantages
More accurately assess meaning of system changes
Become aware of system interrelationships
Appreciate dynamic complexity
Base decisions for action on improvement signals
HOWEVER…
Requires the viewer to understand variation!
Source: Provost, Murray & Britto (IHI Forum 2010)
12. Physician Satisfaction
Control Chart Dashboard
Source: Provost, Murray & Britto (IHI Forum 2010)
SPN Dashboard Report Fall 2010
Small Multiples
One measure,
all sites
Source: Dentaquest Institute
Small Multiples: One site, all measures
Source: Dentaquest Institute
Richard Gareth
Follow-Up Call
April 11, 2017 – 12:00 – 1:00 PM ET