Mark Pletcher 6/10/2011 Quantifying Treatment Effects

Mark Pletcher6/10/2011

Quantifying Treatment Effects

Rationale

Any treatment involves tradeoffs Weigh benefits against risks/costs

Benefit$$ Harm

Rationale

Sometimes the decision is difficult!

Benefit $$ Harm

Rationale

Benefit $$ Harm

How big is this box?

And this one?

Rationale

Tests can help us understand who is most likely to benefit from a treatment

Benefit $$ Harm

How big is this box?

And this one?

Rationale

Tests can help us understand who is most likely to benefit from a treatment Rapid strep to decide who will benefit

from penicillin BNP to decide who will benefit from

furosemide CRP to decide who will benefit from

statins

Rationale

The utility of a test depends on:

How beneficial the treatment is How harmful the treatment is How much the test tells us about

these benefits and harms in a given individual

Risk of harm from the test itself

Rationale

The utility of a test depends on:

How beneficial the treatment is How harmful the treatment is How much the test tells us about

these benefits and harms in a given individual

Risk of harm from the test itselfThe topic for this lecture

Outline

Is an intervention really beneficial? How beneficial is it? Pitfalls Examples

Is the intervention beneficial?

Randomized trials compare an outcome in treated to untreated persons MI in 10% vs. 15% Duration of flu symptoms 3 vs. 5 days


Randomized trials compare an outcome in treated to untreated persons MI in 10% vs. 15% Duration of flu symptoms 3 vs. 5 days

*Statistics* are used to decide if should reject the “null hypothesis” and accept that the intervention is beneficial


But statistics cannot help us interpret effect size

Quantifying the Benefit Effect size

How do we summarize and communicate this?

What is really important for clinicians and policymakers?

Quantifying the Benefit Effect size

How do we summarize and communicate this?

What is really important for clinicians and policymakers?

Example: MI in 10% vs. 15% Q: What do we do with these two

numbers?

Quantifying the Benefit

Two simple possibilities:

10% / 15% = 0.66 15% - 10% = 5%


Two simple possibilities:

10% / 15% = 0.66 15% - 10% = 5%

Relative Risk (RR)

Absolute Risk Reduction (ARR)


Relative risk as a measure of effect size

RR = 0.66 – is this big or small?



RR = 0.66 – is this big or small? MI: 10% vs. 15% in

10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime



RR = 0.66 – is this big or small? MI: 10% vs. 15% in

10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime

Medium

Big

Small



RR = 0.66 – is this big or small? MI: 10% vs. 15% in 10 years Death: 50% vs. 75% in 3 years Basal Cell CA: 2% vs. 3% in lifetime

RR is NOT the best measure of effect size


Absolute risk reduction (ARR) is better

ARR = Risk difference = Risk2 – Risk1


Absolute risk reduction (ARR) is better

RR ARRMI: 10% vs. 15% in 10 years .66

5%Death: 50% vs. 75% in 3 years .66 25%

Basal Cell CA: 2% vs. 3% in lifetime .66 1%

Q: What does the 34% reduction mean?

Nimotop® Ad Graph

22% 33%

Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% = .66 ARR = 33% - 22% = 11%

Nimotop® Ad Graph

22% 33%

Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% = .66 ARR = 33% - 22% = 11%

What is 34%?

Nimotop® Ad Graph

22% 33%

Risk1 = 61/278 = 21.8% Risk2 = 92/276 = 33% RR = 22%/33% = .66 ARR = 33% - 22% = 11%

Relative risk reduction (RRR) =

1 – RR = 1-.66 = .34 or 34%


RRR is no better than RR

RR RRRMI: 10% vs. 15% in 10 years .66

34%Death: 50% vs. 75% in 3 years .66 34%



RRR is ALWAYS bigger than ARR (unless untreated risk is 100%)


BEWARE of risk reduction language!!

ARR or RRR? “We reduced risk by 34%” “Risk was 34% lower”


BEWARE of risk reduction language!!

ARR or RRR? “We reduced risk by 34%” can’t tell “Risk was 34% lower” can’t tell

Very hard to be unambiguous!


Another reason that ARR is better:

Translate it into “Number Needed to Treat”

NNT = 1/ARR

Why is NNT = 1/ARR?

67 no stroke anyway

22 strokes with Nimotop®

11 strokes prevented

22 strokes with with treatment

33 strokes with no treatment

100 SAH patients treated

R2

R1

Why is NNT 1/ARR?

Treat 100 SAH patients prevent 11 strokes

Ratio manipulation:

100 treated 1 treated 9.1 treated11 prevented .11 prevented 1

prevented

= =

Why is NNT 1/ARR?

Treat 100 SAH patients prevent 11 strokes

Ratio manipulation:

100 treated 1 treated 9.1 treated11 prevented .11 prevented 1

prevented

= =

1/ARR = NNT

Why is NNT 1/ARR?

NNT best expressed in a sentence:

“Need to treat 9.1 persons with SAH using nimodipine to prevent 1 cerebral infarction”


NNT calculation practice

RR ARR NNT?

MI: 10% vs. 15% in 10 years .665%

Death: 50% vs. 75% in 3 years .66 25%




RR ARR NNT?

MI: 10% vs. 15% in 10 years .665% 20

Death: 50% vs. 75% in 3 years .66 25% Basal Cell CA: 2% vs. 3% in lifetime .66 1%



RR ARR NNT?

MI: 10% vs. 15% in 10 years .665% 20

Death: 50% vs. 75% in 3 years .66 25% 4




RR ARR NNT?

MI: 10% vs. 15% in 10 years .665% 20

Death: 50% vs. 75% in 3 years .66 25% 4

Basal Cell CA: 2% vs. 3% in lifetime .66 1% 100


NNT expression practice

RR ARR NNT?

MI: 10% vs. 15% in 10 years .665% 20

Death: 50% vs. 75% in 3 years .66 25% 4

Basal Cell CA: 2% vs. 3% in lifetime .66 1% 100

Statins

Chemo

Sunscreen every day


NNT expression practice

“Need to treat 20 patients with statins for 10 years to prevent 1 MI”

“Need to treat 4 patients with chemo for 3 years to prevent 1 death”

“Need to treat 100 patients with sunscreen every day for their whole life to prevent 1 basal cell”

Example 1

Randomized controlled trial of the effects of hip replacement vs. screws on re-operation in elderly patients with displaced hip fractures.

Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.

Example 1Re-

operationNo Re-

operation

Hip Replacement 12 217 229

Internal Fixation with Screws 90 136 226

Parker MH et al. Bone Joint Surg Br. 84(8):1150-1155.

Example 1Re-

operationNo Re-

operation Risk


12/229 = 5.2%


90/226 =

39.8%

Example 1Re-

operationNo Re-

operation Risk


12/229 = 5.2%


90/226 =

39.8%

RR = R1/R2 = 5.2% / 39.8% = .13

RRR = 1-RR = 1-.13 = 87%

ARR = R2 – R1 = 39.8% - 5.2% = 34.6%

NNT = 1/ARR = 1/.346= 3

“Need to treat 3 patients with hip replacement instead of screws to prevent 1 from needing a re-do operation”

Example 2

JUPITER: Randomized controlled trial of high dose rosuvastatin in patients with LDL<130 and CRP>2.0

Ridker et al. NEJM 2008; 359:2195-207

Example 2

Ridker et al. NEJM 2008; 359:2195-207

Example 2

Ridker et al. NEJM 2008; 359:2195-207

Example 2

HR = (R1/R2) (from regression) = .56

RRR = 1-HR = 1-.56 = 44%

ARR = R2 – R1 = 1.36 - 0.77 = .59 / 100py*

= .0059 / py

NNT = 1/ARR = 1/.0059 = 100/.59 = 169 pys

“Need to treat 169 patients for a year to prevent 1 CVD event”

Or better:

“Need to treat 85 patients for 2 years to prevent 1 CVD event”

(average treatment duration in trial was 1.9 years)

* py = person-years

Example 4

Warfarin vs. placebo for atrial fibrillation

Warfarin Placebo

Risk of major bleed (/yr) 1.2% 0.7%

Ann Intern Med 1999; 131:492-501

Example 4

Warfarin vs. placebo for atrial fibrillation

RR = R1/R2 = 1.2% / .7% = 1.7

RR (flipped) = R2/R1 = .7% / 1.2% = .59

RRR (flipped) = 1-RR = 1 - .59 = 41%

ARR = R2 – R1 = .7% - 1.2% = -.5%

“ARI” – Absolute risk increase = 0.5%

NNT = 1/ARR = 1/-.5% = -200

“NNH” – Number needed to harm = -NNT = 1/ARI = 200

“If you treat 200 Afib patients with warfarin, you will cause 1 major bleed”

Circling back to test utility… Tests help determine:

If the RR applies Treatment for a disease doesn’t help if you don’t

have the disease! Interactions (RR is higher or lower than average)

Statins more effective if CRP is high? Patients with gene XYZ more likely to have a side

effect

Baseline risk The higher the risk, the larger the ARR, the

smaller the NNT

Key Concepts Test utility depends on how good the

treatment is RR and p-values good for hypothesis

testing/statistics ARR and NNT (and NNH) better for

interpreting clinical importance ARR = risk difference NNT = 1/NNT

Beware RRR and ambiguous language

Documents

Mark Pletcher 6/10/2011 Quantifying Treatment Effects