Resident Research ConferenceNovember 20, 2013

Alex Bryant

Physician handling of medication order alerts

Overview

What are electronic order alerts?Rationale for studyObjectives and HypothesisDesign and scopeFindingsDiscussionFuture directions and quality improvementQuestions and acknowledgements

A familiar frustration?

Why is the EMR alerting my orders?

Part of CMS “Meaningful Use” criteria for EMR All practice groups who meet MU criteria get incentive pay If no meaningful use of EMR by 2015, reimbursement is cut Requirements include CPOE, clinical decision support, allergy

and drug interaction checking at medication entry

Electronic order entry with medication checking reduced errors and adverse drug events (ADEs)1

Serious ADE rates dropped from 0.7% to 0.1% Caveat: multiple simultaneous interventions

Bates et al. JAMA 1998;280:1311-16.

Are order checks preventing patient harm?

Extremely difficult to answer Alerts now integral in most commercial EMRs Multiple confounders, difficult to detect ADEs

Providers strongly dislike intrusive alerting2

Is this intervention still beneficial, or just distracting?

Horn et al. Am J Health-Syst Pharm 2013;70:905-9.

Override rates: a proxy for relevance

Alert override rates serve to gauge clinical utility Alerts are meant to be sensitive, not specific Still, 50-90% of alerts are overridden No change in override rates despite 10+ years of QI

effort Partly attributed to “alert fatigue” and information

overload

80-90% of alerts overridden at our VA in 20063

Lin et al. JAMIA 2008; 15:620-6.

Surely the UW can do better?

Ongoing effort to improve alert relevance since 2008 Implemented many usability changes from the literature

Panel of MDs, RNs, pharmacists, IT staff meets monthly to review alert and override statistics Integrate expert opinions on interaction risk and feedback

from practicing physicians Low-risk or irrelevant interactions are removed from

alerting, or downgraded so that only Pharmacy sees them

Shouldn’t alerts be more relevant and accepted now?

Objectives and Hypotheses

Objective: Analyze critical medication alert override rates and associated factors at UWMC & HMC

HypothesesRates will be lower than historical norms,

including those at our VA, due to ongoing improvement efforts

Physicians who see more alerts will be more likely to override due to “alert fatigue”

Capturing physician behavior

Required reason

Alert type (only “critical” interactions alerted)

OVERRIDE

Override gets logged with time, drugs, patient and provider/team

MD orders

• MD override required for any interactions of “critical” severity

Pharmacy releases

• RPh sees all interactions (severe or not), may call MD or hold order

Nurse administe

rs

• Monitor for ADE

Medication order processing

Order and alert logged

Design and Scope

Retrospective observational study of all medication orders and “critical” alerts at HMC and UWMC

All providers ordering June 10 – June 13 2013 (96h)

Filtered to include only physician-entered orders 461 unique MDs saw alerts during this period

No observation of behavior outside of alerts

Alert data breakdown

Category Total% of

alertsOverridenumber

Override rate  

Medication orders

18354

       

Unique alerts 2455 100 % 2280 92.9 %  

Interaction type

         

Drug-drug 1153 47.0 % 1097 95.1 %} p < 0.0001Drug-

allergy1302 53.0 % 1183 90.9 %

Physician level

         

Attending 480 19.6 % 454 94.6 %} p = 0.11

Resident 1975 80.4 % 1830 92.5 %

Hospital          

HMC 1200 48.9 % 1111 92.3 %} p = 0.25

UWMC 1255 51.1 % 1175 93.5 %

High alerting and override rates

We’re alerting 13% of orders (2455/18354) Compare with 2.5% (VA 2006), up to 20% in other

studies

But higher volume is probably not the reason drug-drug alerts are overridden 95% of the time Average MD sees only 1 alert per day < 5 % of MDs see more than 4 alerts per day MDs with more alerts were not more likely to override

Short term “alert fatigue” not significant

0 2 4 6 8 10 12 14 1680%

85%

90%

95%

100%

f(x) = 0.00156232139818099 x + 0.927856117069122R² = 0.0257796627982382

Alerts per physician per day

Mea

n o

verr

ide

rate

P = 0.41

Why are override rates still so high?

Neither hospital site nor training level matterMost interaction pairs are overridden every

time Including antithrombotics, antipsychotics, sedatives,

analgesia Almost no drug triggers have < 80% override

Reglan + antipsychotics a notable exception Antibiotics usually have < 90% override, though some

higher

Allergy alerting data are only slightly better Due to known inaccuracies in allergy charting? Even exact allergy matches (9%) have 75% override Hard to believe such a large override rate is

appropriate

Provider specified override reason says little

Vast majority of us choose “Provider approved”

None of the options gives much information for QI

18% of drug-drug alerts had an “Allergy” reason! Inappropriate selection suggests many users are not

even reading the alert window

In summary

Override rates remain high despite best QI effortsIt’s not a problem of pure alert fatigue, but

relevanceSuspect that providers are ignoring the alert

windowLarge proportion of interactions are always ignored

All to suggest that medication order alerts are NOT preventing ADEs at point of entry ...but we can’t prove this without a much more difficult

study!

How can we improve from here?

Improve override reasons and allergy charting?

Restriction of alert triggers with 100% override? i.e. not alerting antipsychotic interactions to Psych MDs Limited by liability concerns Cutting down alert volumes alone won’t increase acceptance

Can we encourage conscious processing of alerts? Provide specific risk information in alert window Suggest alternate therapy

Questions?

Many thanks to: Tom Payne for his mentorship Joe Smith for data and technical assistance Grant Fletcher, John Horn, and Paul Sutton for their

insights

Publishing soon!