Upload
colin-snyder
View
224
Download
4
Tags:
Embed Size (px)
Citation preview
Quality Quality ImprovementImprovementPitfalls and Pitfalls and
OpportunitiesOpportunitiesJames Hallenbeck, MDJames Hallenbeck, MD
ACOS/Extended CareACOS/Extended Care
VA Palo Alto HCSVA Palo Alto HCS
ObjectivesObjectives
Compare top-down and grassroots Compare top-down and grassroots strategies for quality improvement in strategies for quality improvement in terms of advantages and disadvantagesterms of advantages and disadvantages
Discuss different uses of quality Discuss different uses of quality monitorsmonitors Comparisons across groupsComparisons across groups Quality improvementQuality improvement
Better respond to queries regarding Better respond to queries regarding monitors, especially performance monitors, especially performance measures, especially MDSmeasures, especially MDS
Be able to…….
How this might be useful to How this might be useful to you…you…
Self defense - protecting you and Self defense - protecting you and your program when data does not your program when data does not accurately reflect the quality of care accurately reflect the quality of care you deliveryou deliver
Promote better use of monitors Promote better use of monitors (MDS and others) for quality (MDS and others) for quality improvementimprovement
Purposes of Quality Purposes of Quality Improvement Improvement
MonitorsMonitors ComparisonsComparisons
VISNs, facilities, wards, cliniciansVISNs, facilities, wards, clinicians Stimulus for quality improvement Stimulus for quality improvement
effortsefforts Benchmarking and ongoing Benchmarking and ongoing
evaluation of quality improvement evaluation of quality improvement effortsefforts
What Makes a Good What Makes a Good Monitor?Monitor?
(Top-down or grassroots)(Top-down or grassroots) Desired outcome strongly linked to Desired outcome strongly linked to
what is measured (i.e. minimally what is measured (i.e. minimally influenced by factors outside the health influenced by factors outside the health system’s control)system’s control)
System change, informed by monitor, System change, informed by monitor, would likely result in better outcomeswould likely result in better outcomes
Accuracy Accuracy Minimal false negatives & false positivesMinimal false negatives & false positives
QI INDICATORSQI INDICATORSTOP TO BOTTOMTOP TO BOTTOM
Enables comparisons Enables comparisons across large systemsacross large systems
Establishes Establishes organizational organizational prioritiespriorities
Buy-in from top Buy-in from top managementmanagement
If successful, results If successful, results in broad, measurable in broad, measurable improvementimprovement
Blind to local issuesBlind to local issues By attending to By attending to
certain priorities, certain priorities, others may be others may be ignoredignored
Lack of buy-in from Lack of buy-in from front-line stafffront-line staff
Need large numbersNeed large numbers Disempowerment at Disempowerment at
local levelslocal levels
Advantages:
Disadvantages:
What Makes a Good Top-Down What Makes a Good Top-Down Monitor?Monitor?
Simple, discrete dataSimple, discrete data Counts of thingsCounts of things
Minimal variation in local systems and Minimal variation in local systems and populationspopulations Comparing apples to applesComparing apples to apples
Large data numbers – in numerator and Large data numbers – in numerator and denominatordenominator
Data gathering, measurement and comparison Data gathering, measurement and comparison automatic automatic
Encourages GOOD clinical practice (not just Encourages GOOD clinical practice (not just designed to identify bad practice)designed to identify bad practice)
Examples Good Top-Down Examples Good Top-Down MonitorsMonitors
Flu vaccine administrationFlu vaccine administration
Appointment waiting timeAppointment waiting time
QIQIGrassrootsGrassroots
Tailored to local Tailored to local circumstancescircumstances
Better buy-in from Better buy-in from front-line stafffront-line staff
Can improve Can improve morale through morale through empowermentempowerment
Comparing Comparing outcomes across outcomes across systems difficultsystems difficult
Potential lack of Potential lack of buy-in from top buy-in from top managementmanagement
Harder to Harder to disseminate best disseminate best practicespractices
Politically, invisiblePolitically, invisible
Advantages Disadvantages
Examples Grassroots QIExamples Grassroots QI
Use of out-of-hospital DNR formsUse of out-of-hospital DNR forms
Addressing options of autopsy/organ Addressing options of autopsy/organ donationdonation
Apples to Apples Apples to Apples Problem…Problem…
Case-mix issuesCase-mix issues Severity of illness (Problematic: Severity of illness (Problematic:
pressure ulcers, weight loss, pressure ulcers, weight loss, dehydration in Hospice)dehydration in Hospice)
Gender (Problematic: UTI rates)Gender (Problematic: UTI rates) Depression/atypical neuroleptics Depression/atypical neuroleptics
(Problematic: geropsych units, hospice)(Problematic: geropsych units, hospice)
Vertical Hierarchy Vertical Hierarchy ProblemProblem
What is “true” at one level of a What is “true” at one level of a vertical hierarchy is not necessarily vertical hierarchy is not necessarily “true” at another (higher or lower) “true” at another (higher or lower) Example: Newtonian physics versus Example: Newtonian physics versus
quantum mechanicsquantum mechanics 30,000 feet view: reveals certain 30,000 feet view: reveals certain
patterns, otherwise invisible, but patterns, otherwise invisible, but obscures otherobscures other
Vertical HierarchyVertical Hierarchy
Nation
VISN 1 VISN 3VISN 2
Facility 1
Facility 2 Facility 3 Facility 4
Division/Ward
Division/Ward
Clinician + PatientClinician + Patient Clinician + Patient
Connection Between What Connection Between What is is MeasuredMeasured and Desired and Desired
Outcome ProblemOutcome Problem Many outcomes (good and bad) Many outcomes (good and bad)
minimally dependent on what we as minimally dependent on what we as clinicians doclinicians do
Examples:Examples: UTI rates UTI rates Fall ratesFall ratesImportant to tease out those aspects of the outcome that are
dependent on the healthcare system. Ideally, these aspects should be measured and worked on.
Environment
Luck
Healthcare System
Clinician Patient
Patient
Variables
Example: FallsExample: Falls Falls related to transfers/ambulation---18Falls related to transfers/ambulation---18 Falls related to toileting—13Falls related to toileting—13 Falls related to Falls related to
reaching/picking/bending—7reaching/picking/bending—7 Claimed did not fall/eased self to floor--5Claimed did not fall/eased self to floor--5 Slipped/tripped-2Slipped/tripped-2 Falls related to medical condition---2Falls related to medical condition---2 Falls related to inappropriate footwear-2Falls related to inappropriate footwear-2
1 month data: 47 falls
Can you tell from this data how these falls might be related to the healthcare system?
Numerator/Denominator Numerator/Denominator ProblemProblem
Percentages Percentages meaninglessmeaningless in isolation in isolation (numerator/denominator not given)(numerator/denominator not given)
Small numbers – 1 – 10 always Small numbers – 1 – 10 always suspicioussuspicious
Numerator problem: rare eventsNumerator problem: rare events Denominator problem:Denominator problem:
Restricted population based on:Restricted population based on: Geography (single ward)Geography (single ward) Time interval (monthly tracking)Time interval (monthly tracking)
977
27
50
6
1510
13
8
19
12
1312108
1712
11
20
1715
1011
7410
11
103
42
33
2328
36
26
172520
16
60
5359
56
71
63
5557
40
0
10
20
30
40
50
60
70
80
Oct
ober
Novem
ber
Decem
ber
Januar
y
Februar
y
Mar
chApril
May
June
July
Line 1
Line 2
Line 3
Line 4
TOTAL
Oct ‘04 - Jun ‘05 TotalFALLS REPORT
Coin-Toss QICoin-Toss QI
10 Facilities toss a coin 10 times10 Facilities toss a coin 10 times Heads is Good, Tails is BadHeads is Good, Tails is Bad True chance (within infinite tosses) True chance (within infinite tosses)
= 50%= 50%
Let’s compare facilities to “national” Let’s compare facilities to “national” datadata
Coin TossCoin Toss
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
1 2 3 4 5 6 7 8 9 10
PercentageHeads
Of 100 tosses 56/100 Heads = “National” average
“Problem”
Facilities“Exceptional” Facilities
ImplicationsImplications
Unadjusted percentages create a Unadjusted percentages create a bias in which small bias in which small
facilities/programs with small facilities/programs with small numbers to report for a given numbers to report for a given indicator are more likely to be indicator are more likely to be
classified at extremes (failing or classified at extremes (failing or exceptional)exceptional)
Data Validity ProblemData Validity Problem
Ideal:Ideal: AutomatedAutomated Unambiguous resultsUnambiguous results
Problematic:Problematic: Multiple people/services responsible for Multiple people/services responsible for
data entrydata entry Results dependent on subjective Results dependent on subjective
judgmentsjudgments Questionable inclusion/exclusion criteriaQuestionable inclusion/exclusion criteria
MDS ExamplesMDS Examples
Dehydration monitor:Dehydration monitor:
Little or no activity monitorLittle or no activity monitor
Bedfast Status monitorBedfast Status monitor
FY 05 Bed Fast Status
0.0%10.0%20.0%30.0%40.0%50.0%
Oct
ob
er
De
cem
b
er
Fe
bru
ary
Ap
ril
Jun
e
Blue = sub acute and hospice wards, Red = national
Prevalence of Little or No Activity
0.0%2.0%4.0%6.0%8.0%
10.0%12.0%14.0%
Palo Alto
Menlo Park
Livermore
VISN 21
National Average
DehydrationDehydrationSo, What’s the Problem So, What’s the Problem
Here?Here?VA Sierra Pacific Network (VISN 21)
QI#15 Prevalence of dehydrationFiscal Year 2005
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep
CCHCS NCHCS SFVAMC PIHCS SNHCS PAHCS VISN 21 National Average Target
MDS Quality Indicators:MDS Quality Indicators:What does the literature What does the literature
show?show? MDS tends to under-report problemsMDS tends to under-report problems
Example: depression: Example: depression: Differences in rates often reflect either Differences in rates often reflect either
differences in reporting or case mix – differences in reporting or case mix – NOT qualityNOT quality Example: High/Low Pressure Ulcer NH did Example: High/Low Pressure Ulcer NH did
not differ significantly in care processes not differ significantly in care processes (Bates-Jensen)(Bates-Jensen)
Little evidence that MDS indicators per Little evidence that MDS indicators per se useful in improving processes of care se useful in improving processes of care (Rantz)(Rantz)
VISN 21 National Average Target
First Quarter 0.0% 7.9% 4.8% 7.4% 11.0%
Second Quarter 0.0% 12.0% 3.3% 8.8% 11.0%
Third Quarter 3.9% 9.1% 0.0% 7.3% 10.7%
Fourth Quarter 0.0% 9.2% 0.0% 7.2% 10.4%
Prevalence of Depression Without Antidepressant Treatment
Month VISN 21 National Average Target
First Quarter 0.0% 3.7% 4.0% 3.6% 4.7%
Second Quarter 0.0% 5.0% 1.3% 3.4% 4.5%
Third Quarter 0.0% 2.9% 0.0% 2.1% 4.4%
Fourth Quarter 0.0% 3.5% 0.0% 2.2% 4.3%
Comparison to Published Comparison to Published RatesRates
Percentage of NH depressed on Percentage of NH depressed on MDS: ~ 10-36%MDS: ~ 10-36%
Gold standard comparisons: ~ 46-Gold standard comparisons: ~ 46-55% (Schnelle, Simmons) 55% (Schnelle, Simmons) “The MDS “The MDS depression quality indicator underestimates the prevalence depression quality indicator underestimates the prevalence of depressive symptoms in all homes, but in particular, of depressive symptoms in all homes, but in particular, among those reporting low or nonexistent rates”among those reporting low or nonexistent rates”
Percentage of depressed patients Percentage of depressed patients without antidepressant treatment: without antidepressant treatment: 45% (Brown)45% (Brown) Note: not
validated against pharmacy database
Suggestions for Using MDS Suggestions for Using MDS (and similar) Monitors for (and similar) Monitors for
QIQI Consider the good intentConsider the good intent of the monitor of the monitor
It would be bad to have patients starving or It would be bad to have patients starving or thirsty, unnecessarily incontinent, subject to thirsty, unnecessarily incontinent, subject to polypharmcy, inappropriately treated with polypharmcy, inappropriately treated with antipsychotics, who never get out of bed and antipsychotics, who never get out of bed and are bored to tearsare bored to tears
Step BackStep Back from the individual and the from the individual and the percentages and ask, percentages and ask, “what part of this “what part of this outcome might we have some influence on?outcome might we have some influence on?
Brainstorm with staffBrainstorm with staff
Suggestions for Using MDS Suggestions for Using MDS (and similar) Monitors for (and similar) Monitors for
QIQI Consider feasibility of possible Consider feasibility of possible
interventionsinterventions
Benchmark interventionsBenchmark interventions
Note changes following interventionNote changes following intervention
Revise interventionRevise intervention
Example: FallsExample: Falls Transfers/ambulation:Transfers/ambulation:
Wheelchair breaks working and routinely inspected on Wheelchair breaks working and routinely inspected on patients’ wheelchairs?patients’ wheelchairs?
ToiletingToileting Nightlights available, sleepers, patient instructions to Nightlights available, sleepers, patient instructions to
call for assistancecall for assistance Slipped/trippedSlipped/tripped
Inspect rugs, walkways especially during rainy/snowy Inspect rugs, walkways especially during rainy/snowy seasonsseasons
Falls related to medical conditionFalls related to medical condition Review drug therapy for high-risk patientsReview drug therapy for high-risk patients
Inappropriate footwearInappropriate footwear Inspect patients’ footwear, none-slip socks and slippersInspect patients’ footwear, none-slip socks and slippers
SummarySummary
Top-down monitors by themselves Top-down monitors by themselves WILL NOT result in improved care WILL NOT result in improved care deliverydelivery
Top-down monitors work best when Top-down monitors work best when combined with grass-roots efforts to combined with grass-roots efforts to improve careimprove care
Where monitors work poorly for a given Where monitors work poorly for a given purpose, it is our obligation to say so – purpose, it is our obligation to say so – and then work to make things betterand then work to make things better