Evaluating the Medical Evidence

A TOOLKIT FOR THE INTERPRETING THE EFFECTIVENESS OF

INTERVENTIONS

Niteesh Choudhy, M.D., Ph.D.

Take home points•Evid

ence-based medicine has revolutionized the way health care is delivered

1

•There is “evidence” to support whatever you believe!

2

•Academic detailers are ambassadors of the evidence and need to know how to embrace its strengths and limitations

3

Evidence matters

Evidence-based medicine aims to apply the best evidence gained from the scientific method to clinical decision making

Gained prominence in the early 1990’s

De-emphasizes intuition, unsystematic clinical experience and pathophysiologic rationale

Application of evidence in patient care has resulted in substantial reductions in morbidity and mortality

SOURCES: Guyatt et al. JAMA 1992;268:2420-2425; Ford et al. NEJM 2007; 356:2388-98

Contributors to cardiovascular death rates

The volume of “evidence” is overwhelmingNOT ALL EVIDENCE IS OF EQUAL QUALTY

In 1992, internists needed to read an estimated 17 articles every day of the year in order to “keep up” with the literature

The volume of published articles since then has increased exponentially

Made more difficult because not all evidence is of equal quality (i.e. difficult to identify those studies that are particularly important)

Creates a virtually impossible problem for practicing physicians

SOURCES: Davidoff et al BMJ 1995; 310: 1085; http://www.nlm.nih.gov/bsd/medline_lang_distr.html

2005

-200

9

1995

-199

9

1985

-198

9

1975

-197

9

1965

-196

9

1955

-195

9

pre-

1950

0

500,000

1,000,000

1,500,000

2,000,000

2,500,000

3,000,000

3,500,000

Art

icle

s i

n M

ed

lin

e

A hypothetical example

A new cholesterol lowering pill, nolipid, has been synthesized and developed into tablet form for oral consumption

In a prospective study, nolipid: significantly reduced LDL

cholesterol levels by 50% (p<0.0001)

was well tolerated

had no adverse effectsWOULD YOU RECOMMEND THE USE OF NOLIPID FOR PATIENTS WITH ELEVATED

CHOLESTEROL?

The questions we should be asking:

Did they choose the

right comparator?

Did they choose the

right outcome?

Absolute or relative

changes?

Overall or subgroup results?

Choosing the right comparatorSPARCL

4731 patients who had stroke or TIA one to six months before study entry and NO CAD

Randomized to atorvastatin 80 mg daily or placebo

Significant reduction in primary end-point (fatal or non-fatal stroke) Placebo: 13.1%

Atorvastatin 11.2%

SOURCE: SPARCL investigators. NEJM 2006; 355: 549-59

Choosing the right comparatorSPARCL

BUT… Many patients in SPARCL

would already be on a statin according to current treatment guidelines

SPARCL should have compared high and lower intensity statin therapy

More generally, to get FDA approval, drugs generally only need to demonstrate superiority over placebo but in reality, clinicians and decision makers want information about comparative efficacy/safety

SOURCE: SPARCL investigators. NEJM 2006; 355: 549-59

Current NCEP/ATPIII cholesterol treatment guidelines

Risk Category LDL Goal

(mg/dL)

LDL Level at Which

to ConsiderDrug

Therapy (mg/dL)CHD or

CHD Risk Equivalent

s(10-year

risk >20%)

<100

130 (100–129:

drug optional)

Evaluating the right outcomeEZETIMIBE AND THE ENHANCE TRIAL

720 patients with familial hypercholesterolemia

Randomized to simvastatin + ezetimibe or simvastatin alone

Substantial reductions in LDL from combination therapy

A widely used “surrogate” outcome in cardiovascular trials

However, there was no change in atherosclerosis (carotid-artery intima-media thickness)

Thankfully, this was the trial’s “primary” outcome although many other trials that preceded it only evaluated LDL

SOURCE: Kastelin et al. NEJM 2008; 1431-43

Surrogate end-points

Use of surrogate end-points may lead to rapid and appropriate dissemination of new treatments (e.g. HIV)

However, may also lead to excess morbidity/mortality (e.g. inotropes may improve hemodynamics but some may cause excess mortality)

The majority of clinical trials focus on these outcomes

A surrogate end-point is “a laboratory measurement or

physical sign used as a substitute for a

clinically meaningful end-point that

measures directly how a patient feels,

functions, or survives”

Looking at the “right” resultsTHE SUB-GROUPS OF CHARISMA

Enrolled 15,603 patients with established cardiovascular disease or multiple risk factors

Randomized to clopidogrel 75 mg or placebo added to aspirin 75-162 mg daily (median follow-up duration 28 months)

The published conclusion:

HR: 0.93 (95% CI 0.83 to 1.05)

Overall v. subgroup resultsCHARISMA

Numerous pre-specified sub-group analysis some of which reached (borderline) statistical significance

When analyzing multiple subgroups, some will reach statistical significance by chance alone

While there are statistical methods to deal with this, we should ideally focus on the overall trial results (or the results of a limited set of prespecified subgroups)RR 0.88 (95%CI: 0.77-

0.998)

Absolute v. relative risksJUPITER TRIAL NEJM 2008; 359: 2195-207

Absolute v relative reductionsJUPITER TRIAL NEJM 2008; 359: 2195-207

Rosuvastatin Placebo0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Even

t ra

te p

er

10

0 p

ers

on

years

RELATIVE REDUCTION = 44%ABSOLUTE REDUCTION = 0.59 events per 100

person years

An EBM toolkit•Some

thing is often better than nothing

•We often care about how well a new treatment compares with the existing standard of practice (not placebo)

Right comparator

?

•Surrogate end-points are easier to measure and are often sufficient for a new drug to be approved

•If available, we should really focus on “hard” outcomes

Right end-point?

•Often possible to find subgroups that derive less or more benefit from a treatment

•Should focus on the overall trial results (entire cohort, primary outcome)

Overall v. subgroup results?

•Absolute and relative effects can lead to very different assessments of the benefit/safety of a treatment

•Should use both when weighing the significance of a therapy

Right effect measure?

Take home points•Evid

ence-based medicine has revolutionized the way health care is delivered

1

•There is “evidence” to support whatever you believe!

2

•Academic detailers are ambassadors of the evidence and need to know how to embrace its strengths and limitations

3