EBM Principles Are Not In Crisis

Terry Shaneyfelt, MD, MPHDepartment of Medicine

University of Alabama at BirminghamEBMTeacher.com

@EBMTeacher

Disclosures• Associate editor for BMJ EBM ($)

• Member of American College of Physicians Performance Measurement Committee

• No speakers bureaus, industry grant money, industry stocks

Poorly practiced EBM is not a valid reason to

criticize EBM

High jacked EBM is not a valid reason to

criticize EBM

ACPJC March/April 2002

EBM: A Movement in Crisis?

EBM: A Movement in Crisis?

Problems with the evidence base• Industry sets the research agenda

• Manipulating studies

• Publication bias

• Out of date

• Conflicts of interest / Affect heuristic

Industry shouldn’t be sponsoring studies

Government can’t affordthe failures

How do we get “Real” EBM?

• Patients must demand better evidence, better presented, better explained, and applied in a more personalized way

• Clinical training must go beyond searching and critical appraisal to hone expert judgment and shared decision making skills

• Producers of evidence summaries, guidelines, and decision support tools must take into account who will be using them, for what, and under what circumstances

• Publishers must demand studies meet usability and methodological standards

How do we get “Real” EBM?

• Independent funders must increasingly shape the production, synthesis, and dissemination of high quality clinical and public health

• Research agenda must become broader and more interdisciplinary, embracing the experience of illness, the psychology of evidence interpretation, the sharing of evidence, and how to prevent harm from overdiagnosis

“Real” EBM = different evidence

developed by different people in a

different format

All systems give the results

they are designed to give

Rewards what we have• Goal of industry drugs/devices that get paid for

• What 3rd party payers pay for is what MEDICARE pays for

• What MEDICARE pays for is what the FDA approves

• FDA requires proof of efficacy and safety o Not cost-effectivenesso Not any specific degree of effectivenesso Doesn’t matter how many other similar drugs are out there

“In

div

idu

ali

zed

evi

den

ce”

“Usa

ble

evi

den

ce”

Real EBM Movement

How do we take back EBM?

• The FDA (& equivalent agencies) should:o Demand all studies be registeredo Demand properly conducted trials with hard clinical outcomes (currently

planned trials only reviewed to ensure safety to patients )o Demand publication of all studies o Consider costs and competing array of treatments

• Intensify EBM skills training

• Quality (guidelines) industry needs to be controlled

EBM principles are notin crisis

• Evidence-based moniker has been high jacked

• Algorithmic medicine is antithetical to EBM

• EBM is designed to deal with multimorbidity

• Need lots of changes in the world for “REAL EBM”o Time o Overhaul of drug/device approval/payment schemeo Need experts 1st for “expert judgment”- clinical training improvements

EBM principles are not in crisis

• Good EBM skills can detect problems in the evidence

• Unintended consequences of this movement

• Critical appraisal skills are still criticalo “If peer review was a drug it would never be allowed onto the market. Peer

review would not get onto the market because we have no convincing evidence of its benefits but a lot of evidence of its flaws.”

o “There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.”—Drummond Rennie