Upload
others
View
1
Download
0
Embed Size (px)
Citation preview
The Promise of Big Data and what it means to PSOsBrad Winters, Ph.D., M.D.The Armstrong Institute for Patient Safety and Quality
Objectives
• Know what Big Data is.• Be able to give examples of how Big Data can
serve patient safety and quality endeavors.• Know how Big Data approaches and analytics
can improve PSO collaboration, scope and efficiency.
• Be able to describe the basics of the AHRQ Quality and Safety Review System.
• Understand how EHRs and other developing healthcare IT systems can further PSO goals.
What is Big Data?
• Data sets that are too large orcomplex for traditional softwareprocessing applications
• But what it really refers to, in termsof its usefulness and where thepromises lies is– The patterns and information
that emerge from the data– Medicine has always relied on
observation and recognition ofpatterns in information
– Because of this, “Big Data” carries huge potential for transforming how we practice medicine and how we make it safer and improve quality
Some Potential Pitfalls of Big Data
• Clinical Information is complex– Large data sets can result in differences that are determined to be
statistically different (i.e., ”significant”) but not clinically different or relevant
– Computers don’t yet have the ability to make this comparison• Thus there will always (?) need to be human assessment
– Humans need to frame the question– Humans need to determine the results’ relevancy
• Data Security and appropriate use• During collection• Storage • Analysis• Reporting
Sepsis
• 12th leading cause of mortality in the U.S.• Often recognized late in its course• Despite this, we have driven mortality down from about 40% to 25%
across the sepsis spectrum– Based largely on better treatment but limited impact of earlier
recognition.• Early recognition and intervention is considered essential to driving
down morbidity and mortality further• The problem is that sepsis is a heterogeneous syndrome• It defies a single definition making it difficult to create a reliable
criteria for diagnosis in the traditional way• But can we, using “big data analytics”, find the underlying patterns
and thread that will reliably predict who is at risk or already starting to develop the condition?
• “Sepsis Sniffers”=mining data
“Sniffing” for Sepsis
– ICUs and Emergency Depts. generate monstrous amounts of data (patient under a microscope) through continuous monitoring and testing
– Physiological monitors (Heart rate, respiratory rate, temperature, blood pressure, oxygenation, etc. etc.)
– Laboratory data (general labs, multiple potential biomarkers)
– Physical exam data• Can we leverage this big data for detecting
patterns and generating clinically useful information?
Yes, we can!
• The Problem: Clinicians identify sepsis throughstatic observation of continuous data withoutaccounting for the dynamic interactions betweendata signals.
• The Approach: Network-based data analysis to separate septicfrom non-septic patients
• Account for interactions in continuous and intermittent data• Look for features that distinguish between sepsis and non-sepsis• While raw data could not distinguish (this is how we do it now!)• Data analyzed using a computer algorithm (median values of
something called an eigenvalue) was able to distinguish the sepsisvs. the non-septic state
• These values also may reflect disease progression (who is gettingworse) Santaniello et al. Conf Proc IEEE Eng Med
Biol Soc. 2014;2014:3825-6.
“Sniffing” for Sepsis on the General Ward: the challenge of data sensitivity and specificity
• Not everyone is in the ICU, most patients are on general wards
• Can we transfer what we learn in the ICU to the general ward?– Staff to patient ratios are low– Data is is more scarce, not continuous– Fidelity of this data and its communication is known to be
poor• Brandt et al. (Am J Med Qual 2015, 30: 559-65) created an electronic algorithm
for sepsis “sniffing” on general wards– 100% sensitive (excellent at ruling out sepsis) compared to
clinicians– 62% specific (not so good at ruling in sepsis) compared to
clinicians– Still relies on intermittent, low fidelity data
Big Data is not Big Data unless the Data is Big!
• The General Hospital ward– While it is a trove of data– It is a desert landscape
compared to the ICU• Can we enrich this data
stream and marry it to these kinds of algorithms to improve specificity for all kinds of threats to the patient?
Failure to Rescue (FTR)
http://www.newsnet5.com/longform/dead-in-bed-a-deadly-hospital-secret
Our main response to FTR?
• The Rapid Response Systems • Effective
– 40% reduction in arrests– Mortality reduction 12-18%
• We have made little progress in addressing the failure of what is known as the Afferent Limb (the process of signal definition and detection and notification)
• Can big data and big data analytics move this forward?• I say it can but;• We need to change little “big data” into big “Big Data”
without disrupting workflow or generating alarm/alert fatigue
Surveillance Monitoring
Taenzar et al.Anesthesiology 2011
Surveillance Monitoring
• Can we monitor all patients easily, with high fidelity?
• Can it go beyond “physiological deterioration”?– Falls, Pressure ulcers– Other harms
• Why this would fail– $$$$$$$$$ this is a red herring
– this is the real problem
Results of a Hopkins Surveillance Pilot Project
422 pts, 1266 pt/days, 4118 total alarms, No statistical differences in
gender or age
True w/intervention=1590
(38.6%)
Total True=2668(64.8%)
False=1249(30.3%)
Unclear=202(4.9%)
𝑃𝑃𝑃𝑃𝑃𝑃 =𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜 𝑡𝑡𝑛𝑛𝑛𝑛𝑛𝑛 𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛𝑛𝑛𝑎𝑎
𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜 𝑡𝑡𝑛𝑛𝑛𝑛𝑛𝑛 + 𝑜𝑜𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛 𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛𝑛𝑛𝑎𝑎
Total PPV was 68%
Complications statistically reducedNo FTR deaths during study period1 death in historic control period
Results of a Hopkins Surveillance Pilot Project
422 pts, 1266 pt/days, 4118 total alarms, No statistical differences in
gender or age
True w/intervention=1590
(38.6%)
Total True =2668(64.8)%
False=1249(30.3%)
Unclear=202(4.9%)
𝑃𝑃𝑃𝑃𝑃𝑃 =𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜 𝑡𝑡𝑛𝑛𝑛𝑛𝑛𝑛 𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛𝑛𝑛𝑎𝑎
𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛𝑛 𝑜𝑜𝑜𝑜 𝑡𝑡𝑛𝑛𝑛𝑛𝑛𝑛 + 𝑜𝑜𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛 𝑎𝑎𝑎𝑎𝑎𝑎𝑛𝑛𝑛𝑛𝑎𝑎
Total PPV was 68%
Complications statistically reducedNo FTR deaths during study period1 death in historic control period
Is This Good Enough?
Staff Satisfaction Survey: Selected Results
Surveillance Monitoring and Big Data
• Where are the Big Data analytics for harm and disease detection?– Currently there aren’t any– Big Data Analytics here are still focused on controlling the
alarm and alert fatigue issue and preventing nurse and physician workflow disruptions.
– However, while many argue that these need to be controlled first, Big Data Approaches can be applied to the process of controlling alarm/alert fatigue.
– Like sepsis, all other harms (including alert and alarm fatigue) are complex and discrete elements don’t change in isolation.
– Big Data approaches can make these connections and not only improve harm prediction, but also control the false alert/alarm problem.
All Patient Harms: The AHRQ Quality and Safety Review System (QSRS)
• Patient Safety Reporting systems• Numerous Patient Safety Surveillance systems
have developed over the last 2 decades.• Some are very specific
– MEDMARX (US Pharmacopeia)• Specifically tracks medications related errors and
harm• Some are very generic
– The Veterans Administration Patient Safety Reporting system
• Some require membership– University Health Consortium Patient Safety Net
• Some are local homegrown, some at the state level and others national or international
Big Data and All Harms
• Many such databases are used for Big Data analytics already (e.g., MedMarx) and have led to a body of literature correlating harms’ causes, predictors, etc.
• But there is so much un-tapped opportunity out there
• Some surveillance systems interface but many function in silos
• Integrated holistic databases would be ideal.Can we do this?
Why the QSRS?
• What if you could directly abstract medical records for harms?
• Generate population and individual event reports for harms analysis?
• Populate a national database for Big Data analytics looking for patterns and predictors of harm?– You could more specifically identify individual
patient harms and link them to events in the record.
– This could then augment the AHRQ PSIs and other tools to help guide hospitals in patient safety and quality improvements.
What is the QSRS?
• The QSRS is a logical query algorithm that guides medical coders through a medical record abstraction.
• Designed to– Identify adverse events – Identify certain quality indicators (e.g., fall
assessment at admission)– Collects demographics data for analysis
The QSRS
• Efficiency– Uses a set of Entry Questions that determine
whether harms are even possible for the admission– For example: asks whether the patient had a
central line placed at any time• If answer is “No” then a CLABSI could not occur
– Entire blocks of subsequent questions are suppressed so for this example all subsequent CLABSI related questions are not posed.
– Depending on the patient, nearly all questions could be suppressed or nearly all questions could be posed.
The QSRS
• POA modifiers built into the query for accurate determination of hospital event attribution
• Uses the Common Formats
• Reports can be generated at several levels:– Population level– Event level– Patient level (Case summary)
Population Reports
Case Summary Report
The QSRS
• QSRS is still under development, Ongoing refinement
• “QSRS 1”– First Pilot assessment outside of a Federally
maintained database and first wholly based on EHRs (previous assessments used EHR printouts)
• “QSRS 2”– 2 Health Systems
• Medstar-Washington, D.C./Baltimore system• Johns Hopkins (2 hospitals) partnered with Allegheny
Health in PA, and Queen’s Medical Center in Hawaii.
“QSRS 1”
• System is efficient• Average data abstraction time ≅30 minutes
• Inter-rater reliability is good
• Usability ratings were good (≥73% on most items in survey)
Record set Mean (SD) (minutes) Median (IQR) (minutes)
All sites 33.55 (26.19) 26.72 (25.79)
Facility Percent Agreement
BMC 78.21%
JHH 74.30%
Sibley 81.59%
Where can the QSRS go?
• Outcomes and Process measures• EHRs are a giant repository of “big data”• So far, they have been little more than hand-writing
tools (eliminating errors due to bad handwriting)• But they can be so much more:
– Automated/semi-automated data abstraction/extraction from EHRs to reduce measurement burden
– Natural Language Processing• will require more standardization of EHRs
Leveraging the EHR and QSRS: the Holy Grail
Big Data???(not so much)
VS
This is Big Data
The future of the QSRSand Big Data
• The QSRS can be the Big Data Analytic for Patient Safety and Quality
• Provide for Standardization (Common Formats, common language and reports)
• Support PSO expansion, sharing and integration– “stop re-inventing the wheel”
• Expand scope and understanding
The future of the QSRSand Big Data• PSO learning for rare events (individual
PSOs may never even see some of theseevents) since Big Data throws a wide net.
• National repository for comparators• Drive the standardization of Patient Safety
and Quality elements we want in the EHRs would be ideal (right now each EHR platform is highly proprietary and therefore different).– Where are the assessments?– Where are the harms documented?– Relevant labs, orders, medications etc., etc.– Improve efficiency and accuracy of data abstraction
and analytics
To improve we need to know wherewe are and where we need to go
Thank you