HARMHARM

Indonesia Clinical epidemiology and Indonesia Clinical epidemiology and Evidence Based Medicine (ICEEvidence Based Medicine (ICE--EBM)EBM)

HARMHARM

Jarir At Thobari (FK UGM)Jarir At Thobari (FK UGM)

EfficaciousEfficacious

Medical interventionMedical interventionHarmHarm

HARM

“Unintended physical injury resulting from

or contributed to by medical care, that

requires additional monitoring, treatment

or hospitalization, or that results in death

which is hospital acquired.”which is hospital acquired.”

Definition modified from the IHI definition of Harm by the QUEST

Harm Workgroup

“Do oral contraceptives cause breast “Do oral contraceptives cause breast cancer?”cancer?”

“Do calcium“Do calcium antagonists increase the antagonists increase the risk of death or cancer?”risk of death or cancer?”

HarmHarm

risk of death or cancer?”risk of death or cancer?”

“Do vasectomies increase the risk of “Do vasectomies increase the risk of prostate cancer?”prostate cancer?”

“Do“Does Rosiglitazone cause myocardial es Rosiglitazone cause myocardial infactioninfaction?”?”

Summary estimates of risk of breast cancer in premenopausal women and women younger than 50 years associated with ever use of oral contraceptives

Kahlenborn C et al. Mayo Clin Proc. 2006;81:1290-1302

♦ Meta analysis OR=1.19; 95% CI, 1.09-1.29

– Parous OR=1.29; 95% CI: 1.20-1.40

– Nulliparous OR=1.24; 95% CI: 0.92-1.67

♦ Parous women, OCs were used

Risk of breast cancer in premenopausal women and women younger than 50 years associated with ever use

of oral contraceptives

♦ Parous women, OCs were used

– Before FFTP* OR=1.44 (1.28-1.62)

– after FFTP OR=1.15 (1.06-1.26)

♦ used OCs 4 or more years before FFTP (OR=1.52 (1.26-1.82).

*first full-term pregnancy

Mayo Clin Proc. 2006;81(10):1290-1302

Primary and Other Outcomes: DREAM

Rosiglitazone

group

(n=2635)

Placebo

group

(n=2634)

HR (95% CI) p

Cardiovascular events composite* 75 (2.9%) 55 (2.1%) 1.37 (0.97-1.94) 0.08

Myocardial infarction 15 (0.6%) 9 (0.3%) 1.66 (0.73-3.80) 0.2

Stroke 7 (0.3%) 5 (0.2%) 1.39 (0.44-4.40) 0.6

Cardiovascular death 12 (0.5%) 10 (0.4%) 1.20 (0.52-2.77) 0.7

Confirmed heart failure‡ 14 (0.5%) 2 (0.1%) 7.03 (1.60-30.9) 0.01

New angina 24 (0.9%) 20 (0.8%) 1.20 (0.66-2.17) 0.5

Revascularization 35 (1.3%) 27 (1.0%) 1.29 (0.78-2.14) 0.3

Myocardial infarction, stroke, or

cardiovascular death

32 (1.2%) 23 (0.9%) 1.39 (0.81-2.37) 0.2

DREAM investigators. Lancet 2006;368:1096-1105.

Data are number (%). *Rows are not mutually exclusive for components of the

composite—if a participant had more than one component of the composite then they are

counted in the relevant row. †Regression implies achieving a normal fasting glucose

concentration (as defined in both rows) and 2-h plasma glucose level. ‡Defined as acute

treatment with at least two of the following criteria: typical signs and symptoms, typical

radiological evidence, use of diuretics, vasodilators, or inotropes.

Adverse Events, Hospitalization, and Death: ADOPT

Variable

Rosiglitazone (N=1456) Metformin (N=1454) Glyburide (N=1441)

Serious Events

Total Events

Serious Events

Total Events

Serious Events

Total Events

Adverse events — number of patients (%)

Total events 346 (23.8) 1338 (91.9) 331 (22.8) 1341 (92.2) 308 (21.4) 1321 (91.7)

Cardiovascular disease 49 (3.4) 62 (4.3) 46 (3.2) 58 (4.0) 26 (1.8)† 41 (2.8)

Myocardial Infarction

Fatal 2 (0.1) 2 (0.1) 2 (0.1) 2 (0.1) 3 (0.2) 3 (0.2)

Nonfatal 22 (1.5) 25 (1.7) 18 (1.2) 21 (1.4) 11 (0.8) 15 (1.0)Nonfatal 22 (1.5) 25 (1.7) 18 (1.2) 21 (1.4) 11 (0.8) 15 (1.0)

Congestive heart failure(investigator-reported) 12 (0.8) 22 (1.5) 12 (0.8) 19 (1.3) 3 (0.2)† 9 (0.6)†

Stroke 13 (0.9) 16 (1.1) 17 (1.2) 19 (1.3) 12 (0.8) 17 (1.2)

Peripheral vascular disease 7 (0.5) 36 (2.5) 6 (0.4) 27 (1.9) 4 (0.3) 31 (2.2)

Gastrointestinal events 8 (0.5) 335 (23.0) 7 (0.5) 557 (38.3)‡ 3 (0.2) 316 (21.9)

Death and HospitalizationHospitalization for any reason

Patients — no. (%) 169 (11.6) 172 (11.8) 150 (10.4)

Events — no. 251 267 203

Deaths from any cause—no. 34 31 31

ADOPT study group. N Engl J Med 2006;355:2427-2443.

† P<0.05 for the comparison between this treatment group and the rosiglitazone group; ‡ P<0.01 for the

comparison between this treatment group and the rosiglitazone group.

HOME | SUBSCRIBE | CURRENT ISSUE | PAST ISSUES | COLLECTIONS | HELP | Search Term Advanced Search

Institution: Syrian Arab Republic | Sign In as Individual | Contact Subscription Administrator at Your

Institution | FAQ

.100):1(357;2007A correction has been published: N Engl J Med

Institution | FAQ

Effect of Rosiglitazone on the Risk of Myocardial

Infarction and Death from Cardiovascular Causes

Steven E. Nissen, M.D., and Kathy Wolski, M.P.H

Number 24June 14, 2007Volume 356:2457-2471

Search NEJMSEA RC H

Meta Analysis: Rates of Myocardial Infarction and Death from Cardiovascular Causes

Rosiglitazone

Group

Control

Group

Odds Ratio

(95% CI) P Value

no. of events/total no. (%)

Myocardial Infarction

Small trials combined 44/10,285 (0.43) 22/6106 (0.36) 1.45 (0.88–2.39) 0.15

DREAM 15/2,635 (0.57) 9/2634 (0.34) 1.65 (0.74–3.68) 0.22

ADOPT 27/1,456 (1.85) 41/2895 (1.42) 1.33 (0.80–2.21) 0.27

Overall 1.43 (1.03–1.98) 0.03

Death from

Cardiovascular Causes

Small trials combined 25/6,845 (0.36) 7/3980 (0.18) 2.40 (1.17–4.91) 0.02

DREAM 12/2,635 (0.46) 10/2634 (0.38) 1.20 (0.52–2.78) 0.67

ADOPT 2/1,456 (0.14) 5/2895 (0.17) 0.80 (0.17–3.86) 0.78

Overall 1.64 (0.98–2.74) 0.06

Nissen and Wolski. N Engl J Med 2007;356:2457-2471.

Hierarchy of EvidenceHierarchy of Evidence

Systematic

Reviews of

Randomized

Controlled Trials

(Meta-analysis)

Single Randomized Single Randomized

Controlled Trial (RCT)

Systematic Review of

Observational Studies Addressing

Patient-Important Outcomes

Single Observational Study

Addressing Patient-Important Outcomes

Physiologic Studies

Unsystematic Clinical Observations

Level Therapy/Prevention, Aetiology/Harm

Prognosis Diagnosis Differential diagnosis/symptom

prevalence study

Economic and decision analyses

1a SR (with homogeneity*) of RCTs

SR (with homogeneity*) of inception cohort studies; CDR† validated in different populations

SR (with homogeneity*) of Level 1 diagnostic studies; CDR† with 1b studies from different clinical centres

SR (with homogeneity*) of prospective cohort studies

SR (with homogeneity*) of Level 1 economic studies

1b Individual RCT (with narrow Confidence Interval‡)

Individual inception cohort study with > 80% follow-up; CDR† validated in a single population

Validating** cohort study with good††† reference standards; or CDR† tested within one clinical centre

Prospective cohort study with good follow-up****

Analysis based on clinically sensible costs or alternatives; systematic review(s) of the evidence; and including multi-way sensitivity analyses

1c All or none§ All or none case-series Absolute SpPins and SnNouts†† All or none case-series Absolute better-value or worse-value analyses

2a SR (with homogeneity*) of cohort studies

SR (with homogeneity*) of either retrospective cohort studies or untreated control groups in RCTs

SR (with homogeneity*) of Level >2 diagnostic studies

SR (with homogeneity*) of 2b and better studies

SR (with homogeneity*) of Level >2 economic studies

2b Individual cohort study (including low quality RCT;

Retrospective cohort study or follow-up of untreated control

Exploratory** cohort study with good†††reference standards;

Retrospective cohort study, or poor follow-up

Analysis based on clinically sensible costs or alternatives; limited review(s) (including low quality RCT;

e.g., <80% follow-up)follow-up of untreated control patients in an RCT; Derivation of CDR† or validated on split-sample§§§ only

good†††reference standards; CDR† after derivation, or validated only on split-sample§§§or databases

poor follow-up costs or alternatives; limited review(s) of the evidence, or single studies; and including multi-way sensitivity analyses

2c "Outcomes" Research; Ecological studies

"Outcomes" Research Ecological studies Audit or outcomes research

3a SR (with homogeneity*) of case-control studies

SR (with homogeneity*) of 3b and better studies

SR (with homogeneity*) of 3b and better studies

SR (with homogeneity*) of 3b and better studies

3b Individual Case-Control Study

Non-consecutive study; or without consistently applied reference standards

Non-consecutive cohort study, or very limited population

Analysis based on limited alternatives or costs, poor quality estimates of data, but including sensitivity analyses incorporating clinically sensible variations.

4 Case-series (and poor quality cohort and case-control studies§§)

Case-series (and poor quality prognostic cohort studies***)

Case-control study, poor or non-independent reference standard

Case-series or superseded reference standards

Analysis with no sensitivity analysis

5 Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"

Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"

Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"

Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"

Expert opinion without explicit critical appraisal, or based on economic theory or "first principles"

Clinical Scenario

♦Osteoarthritis

– Osteoarthritis is the most common type of

arthritis, especially among older people

– People with osteoarthritis usually have joint – People with osteoarthritis usually have joint

pain and limited movement

– Unlike some other forms of arthritis,

osteoarthritis only affects joints, and not

internal organs

– Usually need long-term analgesics

– Is there any side effects of using long-term

analgesics?

Clinical QuestionClinical Question

PP•Patient and the problem: osteoarthritis

II•Intervention: selective Cox-2 inhibitor (rofecoxib)II (rofecoxib)

CC•Comparison: other analgesic and anti-inflammatory agents

OO•Outcome of interest: congestive heart failure

BMJ 2005; 330:1370

How to assessHow to assess HARM?HARM?

Is there evidence on Is there evidence on cause & effect cause & effect relationship?relationship?relationship?relationship?

ValidValid ImportanceImportanceDirect Direct

relevancerelevance

Establishing ValidityEstablishing Validity

1. Were there clearly defined group of 1. Were there clearly defined group of

patients, similar in all important ways other patients, similar in all important ways other

than exposure to the treatment or other than exposure to the treatment or other

cause?cause?

♦Randomization??

♦ The choice of comparison groups has ♦ The choice of comparison groups has an enormous influence on the credibility of the results

♦ The design of the study determines the comparison groups

Randomized Controlled Trials (RCT)

♦Not suitable for harm studies:

– Unethical

– Expensive (Rare events, long period of – Expensive (Rare events, long period of time)

– Need large samples

♦ Some RCTs for therapy might have shown harm unintentionally (stopped)

Cohort Studies

♦ Prospective, Non-randomized exposure

♦ Useful when subjects cannot be assigned to an

exposure group

♦ Exposed patients might differ from non-exposed ♦ Exposed patients might differ from non-exposed

for important determinants of outcome

(CONFOUNDERS)

♦ Investigator must document the characteristics of

each group and demonstrate comparability, and

use statistical techniques to adjust for known

confounders

Confounders Confounders ((Atthobari et al, Nephrol. Dial. Transp 2006; 21:3106-14))

Case-Control

♦Useful for RARE outcomes that take LONG time to develop

– Ex: DES and vaginal adenocarcinoma in femalefemale

♦ Susceptible to unmeasured confounders

♦Retrospective

♦Recall Bias

♦Can be used to study potentially causal relationships between risk factors & outcomes

♦Measure prevalence (not incidence)

Cross Sectional Studies

♦Measure prevalence (not incidence)

♦ Exposures (risk factors) and outcomes (disease) are measured simultaneously

– Temporal relationship is lost

Case Series and Case Reports

♦No comparison group!

♦Unusual/dramatic outcome (Phocomelia in offsprings of mothers receiving Thalidomide)(Phocomelia in offsprings of mothers receiving Thalidomide)

♦ Sufficient for hypothesis generation (Need more studies)

DesignDesign Starting Starting PointPoint

AssessmentAssessment StrengthsStrengths WeaknessesWeaknesses

CohortCohort Exposure Exposure statusstatus

Outcome event Outcome event statusstatus

Feasible when Feasible when impossible to impossible to randomizerandomize

--Susceptible to Susceptible to biasbias

--Limited validityLimited validity

CaseCase-- Outcome Outcome Exposure statusExposure status --Overcomes Overcomes --Susceptible to Susceptible to CaseCase--

ControlControl

Outcome Outcome event event statusstatus

Exposure statusExposure status --Overcomes Overcomes temporal temporal delaysdelays

--May only May only requires small requires small sample sizesample size

--Susceptible to Susceptible to biasbias

--Limited validityLimited validity

RCTRCT Exposure Exposure statusstatus

Adverse event Adverse event statusstatus

Low Low susceptibility susceptibility to biasto bias

--FeasibilityFeasibility

--GeneralizabilityGeneralizability

2. Were treatment/exposures & clinical outcomes 2. Were treatment/exposures & clinical outcomes

measured in the same ways in both groups?measured in the same ways in both groups?

(Was the assessment of outcomes either objective (Was the assessment of outcomes either objective

or blinded)or blinded)

– How was the exposure

ascertained?

• Recall Bias or

Study hypothesisStudy hypothesis

BlindBlind

• Recall Bias or

Interviewer Bias

• Blinding

– How was the outcome

ascertained?

• Surveillance Bias

3. Was the follow up of the study patients sufficiently 3. Was the follow up of the study patients sufficiently long (for the outcome to occur) and complete?long (for the outcome to occur) and complete?

Short termShort term

Follow upFollow up20% lost to 20% lost to follow upfollow up

Long termLong termcelecoxib, rofecoxib, celecoxib, rofecoxib,

NSAIDNSAID

CHFCHF

No CHFNo CHF

TemporalityTemporality

Strength of associationStrength of association

Biologic PlaucibilityBiologic Plaucibility

CAUSE

4. Do the results satisfy some “diagnostic tests 4. Do the results satisfy some “diagnostic tests for causation”? for causation”?

Biologic PlaucibilityBiologic Plaucibility

SpecificitySpecificity

DoseDose--responseresponse

ConsistencyConsistency

Cessation of exposureCessation of exposureEFFECT

ARE THE VALID RESULTS OF THIS HARM STUDY IMPORTANT?

♦ If the study is valid in showing harm/association,… how bad and how accurate is the harm?

♦ What is the magnitude and precision of the association between exposure and outcome?

…. In other words…

The Estimate of the Magnitude Depends on

the Study Design

Incidence or risk

CohortRCT

Harm No IIexpexp = 20/1000 = 0.02= 20/1000 = 0.02Harm No

Treatment 20 1000

No treatment 2 1000

IIexpexp = 20/1000 = 0.02= 20/1000 = 0.02

IInonnon--expexp = 2/1000 = 0.002= 2/1000 = 0.002

RR=RR= IIexpexp/ I/ Inonnon--expexp== 1010

Exposed patients are 10 times more likely to suffer from the adverse event than non-exposed

Celecoxib

CHFCHF No CHFNo CHF

EXPOSE TO TREATMENT

ADVERSE OUTCOME

aa bb

cc ddNSAID TREATMENT cc dd

RR = RR = a / (a+b)a / (a+b)--------------------c/ (c+d)c/ (c+d)

OR = OR = ad /bcad /bc

Case-Control StudiesOdds Ratio (Relative Odds)

Adverse Outcome

PresentPresent AbsentAbsent TotalsTotals

ExposureExposure aa bb a+ba+b

OR = (a/c)/(b/d) = ad/bc

ExposureExposure

YesYes

aa bb a+ba+b

ExposureExposure

NoNo

cc dd c+dc+d

TotalsTotals a+ca+c b+db+d a+b+c+a+b+c+dd

Are the Results Important?

Case-Control Studies

♦Out of 100 affected cases

– 90 were exposed to the agent (a)

– 10 were not exposed (c)

♦Out of 100 non-affected cases (controls)

– 45 were exposed to the agent (b)

– 55 were not exposed (d)

Case Control Studies – Odds Ratio

Adverse Outcome♦ Out of 100 affected cases

– 90 were exposed

– 10 were not exposed

♦ Out of 100 controls

– 45 were exposed

Present Absent Totals

ExposureYes

90 45 135– 45 were exposed

– 55 were not exposed

♦ Odds Ratio:– OR = (a/c)/(b/d) = ad/bc = (90x55)/(45x10) = 11

The odds of experiencing the adverse event for patients exposed is 11x that of those unexposed

ExposureNo

10 55 65

Totals 100 100 200

Odds Ratio (OR)

♦ OR > 1 represents an increased risk or association

♦ Describes the relative harm of an exposure independent of disease prevalence

♦ When the prevalence of the outcome of interest is rare in the population from which the sample was drawn (often the reason for using a case-control study), the OR closely approximates the RR

Criteria to Examine RR & OR

♦Magnitude– Cohort and Case-Control are both biased

– What if RR or OR’s value preferable?•R.R. > 3

•O.R. > 4Impressive!!

•O.R. > 4

– Recalculate adjusting for a known confounder, if increased or the same after adjustment � more believable

– If the adjustment results in large decline in RR or OR � We should be suspicious of both of them

♦Precision

Impressive!!

Adjusting for Confounders

♦ Assuming the study is attempting to examine the association between SSRIs and Upper GI bleed.

♦ 100 cases with GI bleed, 90 were on SSRIs, 100 controls with no GI bleed, 45 were on 100 controls with no GI bleed, 45 were on SSRIs � OR = 11

♦ If in the study 7 subjects were also on Ibuproffen (confounder) (2 in the control group one on SSRI, 5 in the cases group 3 on SSRI)

♦ You can calculate an adjusted OR

Adjusted OR

Present Absent Totals

ExposureYes

90 45 135

Exposure 10 55 65

87 44 131

8 54 62

7 subjects on Ibuproffen

2 in the control group one on SSRI (b)

Upper GI Bleed

ExposureNo

65

Totals 100 100 200

The adjusted OR = ad/ad/bcbc = (87x54)/(44x8) = 13= (87x54)/(44x8) = 13

What is your conclusion????What is your conclusion????

8 54 62

95 98 193

5 in the cases group 3 on SSRI (a)

Criteria to Examine RR and OR

♦Magnitude

♦Precision

– Is highest when C.I. given is narrow and remains within a clinically important remains within a clinically important increased risk

Examples of C.I. – Is There Association??

♦ SSRIs and Upper GI bleed:

– Adjusted OR 3.0 (95% CI, 2.1-4.4)

♦ Non-SSRIs and Upper GI bleed:

– Adjusted OR 1.4 (95% CI, 1.1-1.9)

♦ NSAIDs and Upper GI bleed:

– Adjusted OR 3.7 (95% CI, 3.2-4.4)– Adjusted OR 3.7 (95% CI, 3.2-4.4)

♦ SSRIs + NSAIDs and Upper GI bleed:

– Adjusted OR 15.6 (95% CI 6.6-36.6)

♦ Long acting Benzodiazepines and risk of fall in elderly patients:

– RR 2.0 (95% CI 1.6-2.5)

♦ Calcium channel blockers and MI:

– OR 1.9 (95% CI 0.9-5.5)

Number Needed to Harm (NNH)

♦ The number of patients who, if they received the experimental treatment, would lead to one additional patient being “harmed”, compared with being “harmed”, compared with patients who received the comparison treatment.

♦ 1/ARI

NNH – For RCT and Cohort

♦One way to report the evidence to your patients

♦ Same formula as NNT♦ Same formula as NNT

– NNH = 1/ARI = 1/ (Iexp- Inonexp) =

1 [a/(a+b)-c/(c+d)]

ARI = Absolute Risk Increase

Number Needed to Harm (NNH)Number Needed to Harm (NNH)

NNHNNH = = aa

a + ba + bcc

c + dc + d

11

1 1 -- [PEER x (1 [PEER x (1 -- OR)]OR)]NNH = NNH = ----------------------------------------------------------

(1 (1 -- PEER) x PEER x (1 PEER) x PEER x (1 -- OR)OR)

1 1 ++ [PEER x (OR[PEER x (OR -- 11)])]NNH = NNH = ----------------------------------------------------------

(1 (1 -- PEER) x PEER x (ORPEER) x PEER x (OR -- 11))If OR > 1If OR > 1

If OR < 1If OR < 1

NNH - For Case Control

♦ PEER = Patient Expected Event Rate = Adverse event rate in non exposed

♦ Table of values available (Do NOT Need to ♦ Table of values available (Do NOT Need to Memorize)

♦ Note: For the same OR, NNH can be different for different PEERs:– Example: For OR = 0.9

• NNH = 2000 for PEER of 0.005

• NNH = 40 for PEER of 0.40

Converting OR to NNH

For OR greater than 11.1 1.25 1.5 1.75 2 2.25 2.5

0.05 212 86 44 30 23 18 16

0.1 113 46 24 16 13 10 9

0.2 64 27 14 10 8 7 6

Calculator available at:

http://www.cebm.utoronto.ca/practise/ca/statscal/orToNnt.htm

PEER

0.2 64 27 14 10 8 7 6

0.3 50 21 11 8 7 6 5

0.4 44 19 10 8 6 5 5

0.5 42 18 10 8 6 6 5

0.7 51 23 13 10 9 8 7

0.9 121 55 33 25 22 19 18

The Flawless Study

♦ A young investigator is designing a case-control study on the association between SSRIs and upper GI bleed. He recruited his cases and controls from a GI clinic. Both cases and controls were interviewed retrospectively for their use of SSRIs. In order to blind

FlawfulFlawful

retrospectively for their use of SSRIs. In order to blind himself to the subjects response, interviews were conducted by the clinic nurse who would ask the subjects a set of questions while checking them in for the visit. The investigator reported a RR of 2.5 (95% CI 0.5-4.0) and concluded that there is a positive association between SSRIs and upper GI bleed and that 1 in 2-3 patients treated with SSRIs will have an upper GI bleed caused by the medication

Study Flaws

♦Controls might have other GI diseases that predisposes them to GI bleed

♦ Interviewer not blinded

♦CI wide and lower end < 1

♦Used RR in a case-control design

♦ Interpretation of RR is wrong

Summary

♦ Harm is unavoidable but can be minimized

♦ Every intervention has some degree of harm

♦ Criteria for establishing validity in a harm study

♦ RCT, while excellent design, might be unethical ♦ RCT, while excellent design, might be unethical in harm

♦ Cohort/ Case-Control (Rare outcome or outcome that takes long time to develop)

♦ Association versus Causation

Summary

♦ Magnitude: – RCT or Cohort � RR = Iexp/Inonexp = [a/(a+b)]/[c/(c+d)]

– Case-Control � OR = (a/c)/(b/d) = ad/bc

♦ Magnitude: RR or OR > 1 (positive association)– Preferable association: (because of confounders)– Preferable association: (because of confounders)

• RR > 3

• OR > 4

– More criteria needed for CAUSATION

– Adjustment for confounders

♦ Precision: C.I. (Narrow and do not straddle “1”)

♦ NNH = 1/ARI = 1/ (Iexp- Inonexp)

THANK YOU