Upload
charity-herrera
View
46
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Using Multiple Data Sources to Understand Variable Interventions. Bruce E. Landon, M.D., M.B.A. Harvard Medical School AcademyHealth Annual Research Meeting June 10, 2008. The Problem. - PowerPoint PPT Presentation
Citation preview
Using Multiple Data Sources to Understand Variable
Interventions Bruce E. Landon, M.D., M.B.A.
Harvard Medical School
AcademyHealth Annual Research Meeting
June 10, 2008
The Problem
• Quality improvement interventions often teach a method for improvement, rather than a specific intervention
• Variability in implementation across sites– Site specific needs– Resources, leadership, etc.– Specific interventions
• Variable evidence of success at the level of individual sites
• What can be learned from this variability?
Outline
• The HRSA Health Disparities Collaboratives– What do organizations do in a QIC?
• The EQHIV Study in Ryan White Funded HIV Clinics – What accounts for negative results?– How reliable are organizational
assessments?
• Conclusions
IHI and the Breakthrough Series
• Collaborative method for improving the quality and value of health care
• Short term (6-18 months) programs that bring together learning teams from multiple organizations
• Developed by IHI in 1995 – Over 50 collaboratives (just by IHI)– Over 2000 improvement teams
Setting Aims
Measuring Progress
Selecting/Implementing Changes
Testing Changes
The IHI Learning Model
Source: The Institute for Healthcare Improvement
Health Disparities Collaboratives
Prevention and Screening
Pre Post
Collaborative 39 51*** Collaborative – Int. Control = 6.5% ***
Collaborative – Ext. Control = 5.5% ***
Int. Control 41 45 ***
Ext. Control 39 45 ***
Disease Monitoring and Treatment
Collaborative 47 57 *** Collaborative – Int. Control = 5.9% ***
Collaborative – Ext. Control = 6.5% ***
Int. Control 46 50 ***
Ext. Control 50 54 ***
Source: Landon BE, et al. NEJM 2007 ***p<.001
Looking Inside the Collaboratives
• Interventions systematically recorded by each site in “monthly reports”
• Coding instrument developed to categorize interventions based on the CCM and stage of implementation
• Coded by two trained abstractors independently
The Chronic Care Model
Source: Wagner et. Al, The McColl Institute
Delivery System Design (Changes in the organization of human resources)
Sub-Categories• Care management roles• Practice team • Care delivery• Proactive follow-up• Planned visit• Visit system changes
Examples• Nurses take on more
patient follow-up tasks• Multidisciplinary team
meetings• Standardize specialist
referral process• Call patients who are
overdue for visit• Targeted reminders in
charts prior to visits• Group visits
Decision Support (guidance for provider behavior or decision-making)
Sub-categories• Institutionalization of
guidelines, protocols and prompts
• Provider education• Expert consultation
support
Examples• Structured forms to
replace progress notes
• Podiatrist teaches nurses to do foot exams
• Case conferences with specialists
Quality Improvement Activitiesby CCM Categories
CCM Category Total Number of Activities
Mean number of Activities per Center (Range)
Delivery system design 342 8.8 (2-15)
Self-management support 279 7.2 (1-19)
Decision support 231 6.1 (1-14)
Information support 352 8.8 (3-16)
Community linkages 343 8.8 (1-24)
Health system organization 207 5.2 (1-12)
Total 1,754 43.9 (8-84)
Intensity of Quality Improvement Activities, by CCM Categories
CCM Category
Number (%) of Activities
Implemented (“Actions”)
% of Actions Institutionalized
and/or
Refined
% of Actions Evaluated
% of Actions of “high” or “very high”
Probable Impact
Delivery system design
295 (86%) 57% 28% 2%
Self-management support
251 (90%) 62% 43% 8%
Decision support 200 (87%) 51% 32% 9%
Information support 313 (89%) 50% 35% 1%
Community linkages 284 (83%) 42% 10% 0%
Health system organization
189 (91%) 60% 21% 2%
Total 1,532 (87%) 53% 28% 3%
Relationship between QI Activities and Quality Change
• No significant relationships between:– Total number of activities– Mean impact score of activities– % implemented– % institutionalized– Number graded as “high” or “very high”
And changes in observed quality of care
Limitations to This Type of Analysis
• Dramatic loss of power when changing from a controlled design to an observational design with n=~40
• One size might not fit all
• Specific interventions are designed to meet local needs and might not be transferrable
• How to account for local context?
Landon, B. E. et. al. Ann Intern Med 2004;140:887-896
The EQHIV Study
DifferenceIntervention Clinic
Difference Control Clinic
Percentage points P Value
Antiretroviral therapy
Receipt of HAART on last visit for appropriate patients
-3.0 2.9 >0.2
Viral load controlled 11.0 5.4 0.18
Screening and prophylaxis
TB screening 0.1 -2.1 >0.2
Influenza shot 7.3 6.8 >0.2
Hepatitis C status 5.5 6.2 >0.2
Pap smear 4.6 -4.2 0.06
Prophylaxis against Pneumocystis carinii pneumonia
0.4 3.5 >0.2
Access to care
Visits in 3 or 4 quarters 5.4 2.7 >0.2
Possible Explanations
• The intervention did not work• Clinics were not “prepared” for the
intervention (or ready)• Change might be required across the
spectrum of the CCM to achieve results• Structure and culture at individual clinics
might impede change• Improving chronic care requires broad
across the board changes
Assessment of Organizational Change
• Pre/post surveys of:– Clinic directors– Clinicians
• Results show modest pre/post changes in 3 domains of the CCM, with little change in the other domains
• System changes were likely not sufficient to lead to broad improvements
How to Increase Reliability of Organizational Surveys?
Scale (# items) ρr
at nj=N/J
Needed for =0.7 ρr(i) Cronbach's α
Openness to QI (7) 0.365 0.674 4 0.803 0.79
HIV knowledge (6) 0.255 0.522 7 0.566 0.57
Research emphasis (3)
0.693 0.891 1 0.693 0.70
Autonomy (3) 0.271 0.570 6 0.566 0.57
Patient help (3) 0.172 0.410 11 0.602 0.60
Guidelines emphasis (2)
0.101 0.272 21 0.530 0.51
Barriers to QI (5) 0.115 0.323 18 0.651 0.65
Patient load (3) 0.221 0.486 8 0.502 0.50
Source: Marsden, Landon, et. al. HSR, 2006.
ρr
Other Potential Methods
• Pre/post surveys of clinicians/support staff/leadership
• Site visits/qualitative data
• Pre surveys to assess “readiness for change”
• ….and so on
Conclusions
• Additional data sources are needed to understand what happens at the individual clinic level
• These data can be useful for understanding more about the implementation of the interventions
• Data are limited by small sample sizes, lack of a control group, and low reliability
• Many questions remain unanswered (e.g., What other components of care are important (leadership, resources, composition, etc.)?)
Final Thoughts
• Quality Improvement is difficult
• Studying Quality Improvement is particularly challenging
• Quality Improvement Implementation Research is a rich area that brings together many disciplines