JAMA: Users’ guide to evidence-based medicine

Therapy

Diagnosis

Harm

Prognosis

7/9/13

8/6/13

11/26/13

12/24/13

How to use an article about therapy or

prevention

MKSAP 16 General Internal Medicine Question 92

A 75-year-old man is hospitalized with sepsis leading to multi-organ failure. A meeting with family members is convened to discuss goals of care for the patient. The treatment team, including infectious disease and critical care consultants, has indicated that the patient is deteriorating despite optimized therapy, and the prognosis is poor. The daughter brings an Internet printout of a trial of a new medication for sepsis. The abstract states “We gave drug ‘X’ to 100 consecutive patients with refractory sepsis in our five intensive care units located in the same geographic region. Eight percent were alive at 30 days.” Although drug “X” is marketed in the United States, it is not FDA-approved for treatment of sepsis. A quick literature search reveals no other studies of drug “X” in the treatment of sepsis.

Which of the following is the main reason that it is difficult to determine the effectiveness of drug “X” based on the published study?

A.No comparison group

B.Outcome assessment not blinded

C.Patients not randomly assigned to treatment

D.Small study size

The three questions

Are the results of the study valid?

What were the results?

Will the results help me in caring for my patients?

Are the results valid?

Primary guides:• Was the assignment of patients to

treatments randomized?• Were all patients who entered the

trial properly accounted for and attributed at its conclusion?– Was follow-up complete?– Were patients analyzed in the groups

to which they were randomized?

Are the results valid?

Secondary guides:• Were patients, health workers, and

study personnel blind to treatment?• Were the groups similar at the

start of the trial?• Aside from the experimental

intervention, were the groups treated equally?

Randomization

Clinical outcomes may result from– Underlying severity of illness– Presence of comorbid conditions– Known and unknown prognostic factors– Treatment effect

What if there are no randomized trials?cohort > case control > case series

Did they adjust for confounding variables?

Follow-up

In positive trials, if the number of patients “lost to follow-up” is large, assume that:

– All patients in the treatment arm did badly

– All patients in the control arm did well

If the conclusion would change, the strength of inference is weakened

Intent-to-treat

As in routine practice, patients in randomized trials sometimes forget to take their medicine or even refuse their treatment altogether.

Non-compliant patients tend to fare worse, regardless on prognostic factors.

If the study attributes all patients to the group to which they were randomized, it is an intent-to-treat analysis

Primary guides of validity

• Randomization• Complete follow-up• Intent-to-treat analysis

Blinding

If physicians and patients could not be blinded (e.g. surgery), then were those who assess clinical outcomes?

Group similarity

Randomization doesn’t always produce groups balanced for known prognostic factors

Magnitude is important, statistical significance of the difference is not

Look for adjustments, and reasons for adjusting

Equal treatment

“Cointerventions” interventions other than the treatment under study, differentially applied to the treatment and control groups

E.g. giving steroids for a COPD exacerbation in a study of a β2 agonist

Permissible cointerventions should be listed, and frequency of administration documented

Secondary guides of validity

• Blinding• Group similarity• Equal treatment of all groups

What were the results?

• How large was the treatment effect?

• How precise was the estimate of the treatment effect?

Magnitude of effect

In a study of 100 patients, 20% in the control group died, 15% of the treatment group died

Risk without therapy (Baseline risk):X 20/100=0.20 or 20%Risk with therapy: Y 15/100=0.15 or 15%Absolute Risk Reduction (Risk Difference): X – Y 0.20-0.15=0.05Relative Risk: Y/X 0.15/0.20 = 0.75Relative Risk Reduction (RRR): [1-0.75]x100=25% [1-Y/X] x 100 or [(X-Y) / X] x 100[0.05/0.20]x100=25%95% Confidence Interval for the RRR -38% to +59%

Precision of the estimate

Confidence interval helps interpret both positive and negative trials

The larger the sample,the narrower the interval

No CI for RRR?

Examine the p-value– p=0.05 → lower bound of the 95% CI = 0

Use the standard error (SE)– upper and lower bounds of the 95% CI

= X ± 2SE

Calculate the 95% CI yourself– http://www.graphpad.com/quickcalcs/NNT1.cfm

Will the results help me in caring for my patients?

• Can the results be applied to my patient care?

• Were all clinically important outcomes considered?

• Are the likely treatment benefits worth the potential harms and costs?

MKSAP 16 General Internal Medicine Question 127

A 50-year-old woman is evaluated for nonischemic cardiomyopathy. Her exercise tolerance is not limited. She takes an ACE inhibitor daily. She took a β-blocker briefly but discontinued because of fatigue. Results of the physical examination are normal.

The patient inquires whether she should receive drug “H”. Drug H was studied in 2000 patients ages 40 to 80 years (mean age 63 years) with New York Heart Association functional class III or IV heart failure. Patients were randomized to receive drug H or a placebo in addition to usual medications. Eighty percent of patients in the trial also took a β-blocker and 70% an ACE inhibitor. At the end of 3 years, patients taking drug H had a significantly reduced rate of a composite outcome of death or heart failure exacerbations. Approximately 5% of the patients taking drug H had serious adverse events, compared with 2% in the placebo group.

Which of the following is the main reason why this patient should not be treated with drug H?

A.her heart failure is too mild

B.she is too young

C.she should be treated with a β-blocker first

D.the drug’s adverse event rate is too high

Does it apply to my patients?

Any compelling reasons why it shouldn’t?

Beware of the subgroup analyses– Often not planned ahead of time– “Data mining” in negative studies– Can use them if:

The difference in effects is very large;Analysis was specified before the study began;There was a very small number of subgroups;Results were replicated in other studies.

Were outcomes clinically important?

Statins improve lipid profilesMetformin lowers HgA1cIsosorbide mononitrate improves COFlecainide supresses ventricular depolarizations

Even when one outcome is clinically important, watch out for effects on others (e.g. mortality versus QoL)

MKSAP 16 General Internal Medicine Question 58

A physician is asked to advise the Pharmacy and Therapeutics Committee of the hospital regarding a new drug to prevent deep venous thrombosis (DVT), drug “Z.” The physician reviews a recent randomized controlled trial of 5000 patients that compared drug Z with drug C, which is commonly used and is on the hospital’s formulary. The following data are abstracted from the trial:

Drug DVT Cases

Drug Z (n = 2500) 25

Drug C (n = 2500) 50

Based on these data, how many patients need to be treated (number needed to treat, NNT) with drug Z, compared with drug C, to prevent one extra case of DVT?

A. 1

B. 2

C. 25

D. 100

E. 167

Number needed to treatRisk with drug C (baseline risk): X 50/2500=0.02 or 2%Risk with drug Z (therapy): Y 25/2500=0.01 or 1%Absolute Risk Reduction (Risk Difference): X – Y 0.02-0.01=0.01 or 1%Relative Risk: Y/X 0.01/0.02 = 0.5 or

50%Relative Risk Reduction (RRR): [1-Y/X] x 100 or [(X-Y) / X] x 100 [1-0.5]x100=50%95% Confidence Interval for the RRR +19% to +68%Number needed to treat: 1/ARR 1/0.01=100

Always compare to NNH

The NNT: http://www.thennt.com/

Remember the questions

Are the results of the study valid?Primary: randomization, follow-up, intentSecondary: blinding, group similarity and

equality

What were the results?Effect magnitude (A/RRR) and accuracy (CI)

Will they help me in caring for my patients?Applicable, clinically important outcomes, NNT