Making Guidelines WORK

Richard N. Shiffman, MD, MCISYale School of Medicine

NYAM Teach 2011

Disclosure

• Funding from AHRQ and NLM

• Will describe several applications developed in our laboratory available gratis for non-commercial use

Today

• Making Guidelines WORK– Improving the delivery of knowledge to the point

of care– Computer-based clinical decision support

3

• Making Guidelines THAT Work– Improving the product– Clarity, transparency, and implementability– GLIA and BRIDGE-Wiz

Guidelines have problems…• Cluzeau (Int J Qual Healthcare 1999), Shaneyfelt (JAMA 1999)

majority of guidelines failed quality criteria

• Grilli: 431 specialty society guidelines (Lancet 2000)– 82% did not apply explicit criteria to grade evidence

– 87% did not report whether a literature search was performed

– 67% did not describe type of professionals involved in development

• Shaneyfelt (JAMA 2009): persisting biases; lack of specificity, flexibility, regular updating

• Alonso-Coello: in 42 reviews of 626 guidelines over past 20 years, mean quality scores for rigor of development, stakeholder involvement, editorial independence, and applicability are “moderate” or “low” (GIN 2009)

CPGs are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options

Guidance on the use of glitazones for the treatment of type 2 diabetes

• For people with type 2 diabetes, the use of a glitazone as second-line therapy added to either metformin or a sulphonylurea--as an alternative to treatment with a combination of metformin and a sulphonylurea-- is not recommended except for those who are unable to take metformin and a sulphonylurea in combination because of intolerance or a contraindication to one of the drugs. In this instance, the glitazone should replace in the combination the drug that is poorly tolerated or contraindicated.

Authors Should Be Explicit About

• WHEN {under what circumstances} • WHO {in the Intended Audience}• Ought to {with what level of obligation}• DO WHAT• {To WHOM} {which members of the target population}

• HOW• WHY

IF

THEN

Denominator

Numerator

Guidance on the use of glitazones for the treatment of type 2 diabetes

• If a patient is unable to take the combination of metformin and sulfonylurea (because of intolerance or contraindication), the clinician should prescribe a glitazone to replace the drug that is not tolerated.

Guidance on the use of glitazones for the treatment of type 2 diabetes

• If a patient is unable to take the combination of metformin and sulfonylurea (because of intolerance or contraindication), the clinician should prescribe a glitazone to replace the drug that is not tolerated.

UNDER WHAT CIRCUMSTANCES?

WHO?

OUGHT? To do WHAT?

Statement of fact is NOT a recommendation

• Adjuvant hormone therapy for locally advanced breast cancer results in improved survival in the long term.

• Clinicians should prescribe adjuvant hormone therapy for locally advanced breast cancer (when/unless?)…

Lomotan E, et al. Qual & Safety in Health Care 2010

How “Should” We Write Guideline Recommendations:

Interpretation of Deontic Terminology

• Goal: To describe the level of obligation conveyed by deontic terms commonly used in practice guidelines

• Can level of obligation be standardized?

Measuring Obligation

0 50 100

Level of Obligation

Musts (19/1250 – 1.5%)• Narcotic use must be carefully titrated and supervised.• Clinicians working in juvenile justice settings must be vigilant for

personal safety and security issues and aware of actions that may compromise their safety and/or the safety and containment of the incarcerated youth

• Nurses working with individuals with asthma must have the appropriate knowledge and skills to identify the level of asthma control, provide basic asthma education, conduct appropriate referrals to physician and community resources

• Treatment of duodenal adenomas depends on adenoma size and the presence of severe dysplasia. Small tubular adenomas with mild dysplasia can be kept under surveillance, but adenomas with severe dysplasia must be removed

15

The Dreaded “Consider”

• The Expert Panel concludes that initiating daily long-term control therapy should be considered for reducing impairment in infants and young children who consistently require symptomatic treatment more than 2 days per week for a period of more than 4 weeks (Evidence D).

• Referral may be considered if a child 0–4 years of age requires step 2 care or a child 5–11 years of age requires step 3 care.

Measurement

• If you can’t measure it, you can’t manage it.

• If you don’t measure it, you can’t improve it.

Peter Drucker

MonitorTest

Gather Data Interpret Act

Dispose

Action-Types

Conclude Prescribe

Educate/counsel

Document

Procedure

Consult/refer

Advocate

PreparePrevent

Inquire Examine

Action-Type Pattern: Prescribe

• Drug information

• Safety alerts (allergy, drug-drug, drug-disease, drug-lab)

• Formulary check

• Dosage calculation

• Pharmacy transmission

• Patient education

• Corollary orders

A Transparent Process for Generating Recommendations

A transparent development process makes clear…

• How authors weighed• evidence • pathophysiologic reasoning (first principles)• expert experience• patients’ and society’s values

• Allows users to judge reasonableness of recommendations

Requires untangling and specifying 2 related (but distinct) concepts

• Quality of evidence

• Recommendation strength

{Elegant and erudite work of GRADE Collaboration}

<---developers’ focus

<-what implementers need to know to design systems that influence care

• level of expected adherence• level of enforcement / incentive

Evidence Quality

• An indication of the authors’ confidence in their appraisal of benefits and harms

• Based on an analysis of the validity, consistency, and directness of the evidence supporting a recommendation

Recommendation Strength

• Implementers need to understand experts’ assessment of strength of recommendation

• Communicates authors’ assessment of the importance of adherence

• Levels based on aggregate evidence quality and balance of anticipated benefits and harms– Strong recommendation (“MUST”)– Recommendation (“SHOULD”)– Option (“MAY”)

Grading Recommendation Strength

Evidence Quality

Preponderance of Benefit or

Harm

Balance of Benefit and

Harm

A. Well designed RCTs or diagnostic studies on relevant population

B. RCTs or diagnostic studies with minor limitations;overwhelmingly consistent evidence from observational studies

C. Observational studies (case-control and cohort design)

D. Expert opinion, case reports, reasoning from first principles

X. Exceptional situations where validating studies cannot be performed and there is a clear preponderance of benefit or harm

Strong

Strong

Rec

RecOption

Option No Rec

GuideLine Implementability

Appraisal

• Goals– To identify intrinsic obstacles to implementation, i.e., those

that are within the purview of guideline developers– To provide feedback to guideline authors to anticipate and

address these obstacles before a draft guideline is finalized– To assist implementers in guideline selection and to target

attention toward anticipated obstacles• GLIA (and eGLIA) available from http://gem.med.yale.edu/glia

BMC Medical Informatics and Decision Making 2005

GLIA v2.0 Dimensions

• Decidability - precisely under what conditions (e.g., age, gender, clinical findings, lab results) to do something

• Executability - exactly what to do under the circumstances defined)

• Validity - the degree to which a recommendation reflects the intent of the developer and the strength of evidence

• Flexibility - the degree to which a recommendation permits interpretation and allows for alternatives in its execution

• Effect on process of care - the degree to which a recommendation impacts upon the usual workflow in a typical care setting

GLIA v 2.0 Dimensions (cont’d)

• Measurability – the degree to which the guideline identifies markers or endpoints to track the effects of implementation of this recommendation

• Novelty/innovation - the degree to which a recommendation proposes behaviors considered unconventional by clinicians or patients

• Computability - the ease with which a recommendation can be operationalized in an electronic information system

Bridge the Gap

Between Authors and Implementers

With BRIDGE-Wiz

(Building Recommendations In a Developer’s Guideline Editor)

BRIDGE-Wiz• Displays a sequence of screens representing chunks of

information about a recommendation• The authors systematically and sequentially determine:

– action(s) to be recommended– condition(s) under which the action is to be performed– benefits, risks, harms, and costs of the proposed action– the quality of the evidence supporting the action.

• The program’s output is an IF…THEN rule and supporting recommendation profile

30

Bridge-Wiz Demo

31

BRIDGE-WizBuilding Recommendations in a Developer’s Guideline Editor

• Formalizes a process for writing implementable recommendations• Focuses discussion• Incorporates prompts based on COGS to improve guideline quality• Controlled natural language

– Offers verb choices based on action-type– Traps and disallows use of “consider”– Discourages “statement of fact” masquerading as recommendation– Limits boolean connectors to all ANDs or ORs in a statement

• Incorporates decidability and executability checks• Requires systematic appraisal of evidence quality and benefit-harms

– Suggests appropriate obligation term (deontic modal)• Output includes a high-level “rule” and an evidence profile

Making Guidelines Work

33

Interventions to Influence Practice

• Education (conferences, courses)

• Audit & feedback

• Financial incentives/disincentives

• Patient-mediated interventions

• Computer based decision support

34

Grol, Grimshaw Lancet 2003

Clinical Decision Support: Definition

• Use of the computer to bring relevant knowledge to bear on the health care and well-being of a patient (Greenes).

• Systems that link health observations with health knowledge to influence health choices by clinicians for improved health care (Hayward)

35

Computer-Based Decision SupportSystematic Reviews

Mary Johnston McMaster JAMA 1994

Derek Hunt McMaster JAMA 1998

Amit Garg Univ. Western Ontario

JAMA 2005

Ken Kawamoto Duke BMJ 2005

Basit Chaudhry UCLA Ann Intern Med 2006

•Computer-based decision support regularly—but not always—improves the process of care

•Outcomes—though infrequently measured—sometimes improve

Identifying Features Critical to Success

• Significant improvement in practice in 68% of 70 trials• Predictors of improved practice:

– Automatic provision of DS as part of workflow– Providing DS at time and site of decision making– Providing recommendations, not just assessments– Providing periodic performance feedback– Sharing recommendations with patients– Requesting reasons for not following recommendations

Kawamoto K. BMJ 2005

Allergy Alert

38

Palette of CDS Interventions

Alert

Infobutton

Algorithm

Calculator

ReminderDocumentationtemplate

Flowsheet

OrderFacilitator

Selected Guideline

• Asthma – EPR3 Diagnosis and Management of Asthma from

the NHLBI (2007)– Demonstrates challenges involved in implementation

of recommendations for chronic management of complex disease

40

Prompts for documentation

Real-time calculation and display

Information Access

Prompts forAssessments

Display of RelevantPast Information

Alert

Order Set

46

Customizable Handout

Medication Authorization

Summary

• Making Guidelines WORK– Computer-based clinical decision support– Improving the delivery of knowledge to the point

of care

47

• Making Guidelines THAT Work– Must address:

– Clarity, transparency, and implementability– GLIA and BRIDGE-Wiz

Thank You!

ycmi.med.yale.edu/GLIDES

richard.shiffman@yale.edu