Session 3: Assessing a Document on Treatment

Session 3: Assessing a Document on Treatment

Peter Tarczy-Hornoch MDHead and Professor, Division of BHIProfessor, Division of Neonatology

Adjunct Professor, Computer Science and Engineering

faculty.washington.edu/pth

October 27, 2008

Assessing a Document on Treatment

Context for Assessing a Treatment Document Treatment Statistics Statistics 102 Applying to a Scenario

Diagnostic vs. Therapeutic Studies

Patient Data & Information

General Information & Knowledge

Case specificdecision making

Diagnostic Testing(What is it?)(Session 4)

Therapy/Treatment(What do I do for it?)(Session 3)

Steps to Finding & Assessing Information

1. Translate your clinical situation into a formal framework for a searchable question (Session 1)

2. Choose source(s) to search (Session 2)

3. Search your source(s) (Session 2)

4. Assess the resulting articles (documents) Therapy documents (today) Diagnosis documents (Session 4) Systematic reviews/comparing documents (Session 5)

5. Decide if you have enough information to make a decision, repeat 1-4 as needed (ICM, clinical rotations, internship, residency) (Session 6)

from Evidence-Based Clinical Practice, 2000, JP Geyman, RA Deyo, SD Ramsey

Assessing a DocumentSearch Result Comparison

Problem at hand Problem studied Are they really the same?

Patient characteristics Population characteristics Is patient similar enough to population studied?

Intervention most relevant to patient/provider

Intervention studied (primary one)

Are they the same?

Comparison – other alternatives considered

Comparison – alternatives studied

Are alternatives studied those of interest to you?

Outcomes – those important to pat/prov

Outcomes – those looked at by study

Are outcomes studied those of interest to you?

Number of subjects Does study have enough subjects to trust results?

Study design hoped for Statistics – study design and statistical results

Is study design good? What do results mean?

Sponsor – who paid for study

Is there potential bias?



Relative Risk, Relative Risk Reduction

Risk = outcome event rate

= number having event

number receiving the intervention

Relative Risk = risk in intervention group

(RR) risk in control group

Note: RR can also be treatment1 vs treatment2

Relative Risk Reduction (RRR) = 1 – RR

Note: each of these statistics can have 95% Confidence Interval (CI) (see Statistics 102). For RR if CI includes 1 then no statistical difference between groups

Absolute Risk Reduction &Number Needed to Treat/Harm

Absolute Risk Reduction (ARR) = difference in risk (control – intervention)

or = difference in risk (treatment1 - treatment2) Note: 10%=>0.10Number Needed to Treat (NNT)

= 1/ARR “the number of patients who need to be treated to prevent one outcome event from occurring in specified time”

Number Needed to Harm (NNH)Same but instead of benefit you get harm (e.g. a side effect)

Note: each of these statistics can have 95% Confidence Interval (CI) (see Statistics 102). For ARR if CI includes (crosses) zero then no statistical difference between groups

Example (I)

An oncology trial testing a new treatment with 4-year follow-up for mortality

(risk of death, (-) outcome)

experimental treatment: 30% (risk in intervention)

control group: 50% (risk in control)

What are the RR, RRR, ARR, NNT?

Example (II)RR = risk of death in intervention/risk of death in control groups

= 30%/50% = 0.6 or 60%

RRR = 1 - RR = 1-0.6 = 0.4 or 40%

ARR = risk of death in control – experimental groups

= .50 -.30 = 0.2 or 20%

NNT =1/ARR = 1 ÷ 0.2 = 5 = 5 patients treated with the experimental therapy

to prevent one death at 4 years

Relative vs. Absolute Increases

Relative Increase: On July 3, 2002 Worldcom stock rose

120% => Great!

Absolute Increase On July 3, 2002 Worldcom stock rose

from $0.10 to $0.22 => Not so great!

Relative RR vs. Absolute RR

Risk 10/100 5/100 RRR = 50% ARR = 5% NNT = 20

Risk 1/100 0.5/100RRR = 50% ARR = 0.5% NNT = 200

Risk 0.1/100 0.05/100RRR = 50% ARR = 0.05% NNT = 2000

Control Treatment



Study Design 101 Case Series: 1 series of patients with outcome of interest, no control group, +/-

protocol, weakest design (“anecdote”) E.g. last 10 toddlers with asthma and their histories

Cohort Trial: 2 groups (cohorts), protocol is “exposure of interest” vs. not, follow FORWARD for outcome of interest, can’t control for UNKNOWN factors

E.g. 10 infants born to mothers who smoked vs. 10 infants born to mothers who did not and seeing (looking FORWARD) whether or not they develop asthma

Case Control Trial: 2 groups, protocol is outcome of interest (“cases”) vs. those without (“controls”), look BACK for “exposure of interest”, can only match cases/controls for KNOWN factors (e.g. age/gender)

E.g. 10 toddlers with asthma and 10 similar toddlers without asthma looking BACK to see if their mothers smoked

Randomized Controlled Trial Protocol is treatment (intervention) randomly assigned, avoids bias by ensuring

KNOWN and UNKNOWN factors (confounders) that determine outcome are evenly distributed between treatment and control groups (or treatment 1 vs treatment 2), often called “gold standard” study design. Double blinding is when neither investigator nor study subjects know who got what intervention.

E.g. 100 toddlers with asthma and randomly giving 50 one treatment and 50 another treatment and seeing if one group does better/worse

Systematic Review/Meta-analysis Summary of medical literature using explicit protocol to search the literature

and critically appraise and combine studies E.g. 20 studies looking at asthma treatments in toddlers

Levels of evidence

EBM, 2005, Straus et al

Statistics 102: Confidence Intervals (I)

95% Confidence Interval (CI): there is a 95% probability the “true” effect of treatment (in the whole population, not the study population) lies within the stated range

Can be calculated for most treatment statistics (RR, NNT, ARR, etc.) “Treatment 1 relative risk of death was 50% that of treatment

2 (RR 95% CI 40%-60%)” 95% chance the “true” (population) RR is in the 40-60% range

Study 1: 95% CI: 40-60% vs. Study 2: 95% CI: 45-55% Study 2 is more precise (narrower 95% CI) Could be due more patients studied in Study 2 or less variability

between patients in Study 2 95% CI can also tell you about significance (next slide)

See: http://www.acponline.org/journals/ecp/sepoct01/primerci.pdf

Statistics 102: Confidence Intervals (II) Likely no statistically significant difference

Likely statistically significant differenceRed line is “no difference”

Red line is “no difference”

Statistics 102: Confidence Intervals (III)

Example: Average weight of students in this classroom

“Truth”: entire population, weigh everybody and calculate average (the vertical line)

“Sample”: 10 randomly chosen students, calculate average and 95% CI for average

Figure: 100 samplings with their 95% CI, note 5 marked with xx do NOT include “truth”

The narrower the CI is, the more certain we can be that the experimental value is close to the “true value”.

Larger samples tend to decrease CI (increase precision)

See: http://www.acponline.org/journals/ecp/sepoct01/primerci.pdf



Applying Statistics to a Scenario Previously health 43 year old male is diagnosed with

a bacterial pneumonia. Prior episode of sinusitis treated with “Z-pack” (Azithromycin). Recommendation to treat with levofloxacin.

Question: how do these treatments compare in terms of clinical cure rates and cost?

Search on terms “community acquired pneumonia treatment levaquin outpatient” finds following: “Novel, single-dose microsphere formulation of

azithromycin versus 7-day levofloxacin therapy for treatment of mild to moderate community-acquired Pneumonia in adults.” Antimicrob Agents Chemother. 2005 October; 49(10): 4035–4041.

Assessing the Document in ScenarioSearch Result Comparison

Problem: Community Acquired Pneumonia

Problem: Community Acquired Pneumonia

Are they really the same?Yes

Patient characteristics: 43yo previously health

Population characteristics 18-95yo, mean 48y

Is patient similar enough to population studied? Yes

Intervention: Levofloxacin Intervention: Levofloxacin Are they the same? Yes

Comparison: Azithromycin Comparison: Azithromycin Are alternatives studied those of interest? Yes

Outcomes – Clinical response, cost

Outcomes – Clinical response

Are outcomes studied those of interest? Partly

Number of subjects Does study have enough subjects to trust results?

Study design hoped for Statistics – study design and statistical results

Is study design good? What do results mean?

Sponsor – who paid for study

Is there potential bias?

Abstract of Document (adapted)This randomized, double-blind, noninferiority study was designed to demonstrate that a single 2.0-g oral dose of a novel microsphere formulation of azithromycin was at least as effective as 7 days of levofloxacin, 500 mg/day, in the treatment of adult patients community acquired pneumonia. Clinical cure rates were 89.7% (156 of 174) for azithromycin microspheres and 93.7% (177 of 189) for levofloxacin (treatment difference, −4.0%; 95% confidence interval, −9.7%, 1.7%). Both treatment regimens were well tolerated; the incidence of treatment-related adverse events was 19.9% and 12.3% for azithromycin and levofloxacin, respectively. A single 2.0-g dose of azithromycin microspheres was at least as effective as a 7-day course of levofloxacin in the treatment of mild to moderate community-acquired pneumonia in adult outpatients.

Calculating Statistics in Scenario

Note: a) control for patient in scenario is azithromycin and intervention is levofloxacin (opposite of paper), b) “risk” in this situation is in fact rate of cure ((+) outcome)

Relative Risk = risk in intervention group = 93.7/89.7=1.04

(RR) risk in control group Absolute Risk Reduction (ARR) = diff in risk (control – intervention)=0.897-0.937=-0.04Number Needed to Treat (NNT)

= 1/ARR = 1/0.04=25 In this scenario the number of patients who need to be treated in order to achieve apparent levofloxacin benefit

Using 95% CI in ScenarioThis randomized, double-blind, noninferiority study was designed to demonstrate that a single 2.0-g oral dose of a novel microsphere formulation of azithromycin was at least as effective as 7 days of levofloxacin, 500 mg/day, in the treatment of adult patients community acquired pneumonia. Clinical cure rates were 89.7% (156 of 174) for azithromycin microspheres and 93.7% (177 of 189) for levofloxacin (treatment difference, −4.0%; 95% confidence interval, −9.7%, 1.7%). Both treatment regimens were well tolerated; the incidence of treatment-related adverse events was 19.9% and 12.3% for azithromycin and levofloxacin, respectively. A single 2.0-g dose of azithromycin microspheres was at least as effective as a 7-day course of levofloxacin in the treatment of mild to moderate community-acquired pneumonia in adult outpatients.=> Note cost of azithromycin generic at Bartell’s is $5 and cost of levofloxacin is $ 106

Small Group Wednesday Oct 29th Small group leads to give examples of recent

clinical situations where they had to evaluate one or more documents related to treating a particular treatment

Group to review and discuss assignment with multiple short examples related to treatments, focusing on:

Confidence intervals RRR ARR NNT

AND/OR: group to search for treatment article(s) on a topic of interest and assess results

QUESTIONS? Context for Assessing a Treatment Document Statistics 102 Treatment Statistics Small Group Portion If any midcourse feedback e-mail pth@u.

Documents

Session 3: Assessing a Document on Treatment