VSM

CHAPTER 6: HARMCHAPTER 6: HARM

Evidence-Based MedicineEvidence-Based Medicine

How to Practice and Teach How to Practice and Teach EMBEMB

VSM

HarmHarmConcerns about potentially harmful Concerns about potentially harmful interventions occur dailyinterventions occur dailyWe must make judgments about We must make judgments about whether our patients may be at riskwhether our patients may be at riskEvaluate evidence about causationEvaluate evidence about causation– ValidityValidity– ImportanceImportance– Relevance to our patientsRelevance to our patients

Goal is to avoid false + and false -Goal is to avoid false + and false -

VSM

Types of Studies on Harm/EtiologyTypes of Studies on Harm/Etiology

Systematic reviewsSystematic reviews– Best evidence about effects of therapyBest evidence about effects of therapy

RCTRCT– Seldom large enough to detect rare Seldom large enough to detect rare

adverse events adverse events – Ill suited in size, duration and ethicsIll suited in size, duration and ethics

Cohort, case-control studies, and Cohort, case-control studies, and cross-sectional studies cross-sectional studies

VSM

Is this evidence about harm valid?Is this evidence about harm valid?1.1. Were there clearly defined groups of Were there clearly defined groups of

patients, similar in all important ways patients, similar in all important ways other than exposure to treatment or other than exposure to treatment or other cause?other cause?

2.2. Were treatments/exposures and clinical Were treatments/exposures and clinical outcomes measured in the same ways in outcomes measured in the same ways in both groups?both groups?

3.3. Was the follow-up appropriate?Was the follow-up appropriate?4.4. Do the results of the harm study fulfill Do the results of the harm study fulfill

some of the diagnostic tests for some of the diagnostic tests for causation?causation?

VSM

Is this evidence about harm valid?Is this evidence about harm valid?1.1. Were there clearly defined groups of Were there clearly defined groups of

patients, similar in all impt ways other patients, similar in all impt ways other than exposure to treatment or cause?than exposure to treatment or cause?

2.2. Were treatments/exposures and clinical Were treatments/exposures and clinical outcomes measured in the same ways in outcomes measured in the same ways in both groups?both groups?

3.3. Was the follow-up long enough?Was the follow-up long enough?4.4. Do the results of the harm study fulfill Do the results of the harm study fulfill

some of the diagnostic tests for some of the diagnostic tests for causation?causation?

VSM

Is it Valid?Is it Valid?BestBestSystematic review or RCTSystematic review or RCT– Randomization would make groups Randomization would make groups

identical for all other causesidentical for all other causes– Ill suited (in size, duration and ethics)Ill suited (in size, duration and ethics)

The validity of the study designs The validity of the study designs used to detect harm is inversely used to detect harm is inversely proportional to their feasibility.proportional to their feasibility.Must often look at other types of Must often look at other types of studiesstudies

VSM

Cohort studyCohort studyObservational studyObservational study

Group of pts who are exposed and group Group of pts who are exposed and group of pts not exposed are followed for of pts not exposed are followed for development of outcomedevelopment of outcome

Not randomized—exposure based on Not randomized—exposure based on patient’s/MD’s preferencespatient’s/MD’s preferences– Yields more confoundersYields more confounders– Multivariable analysis useful, but adjustments Multivariable analysis useful, but adjustments

can only be made for known confounderscan only be made for known confounders– Still may not be useful if outcome rareStill may not be useful if outcome rare

VSM

Case-Control StudiesCase-Control Studies““cases” pts with outcome of interestcases” pts with outcome of interest““controls” those withoutcontrols” those withoutProportion of each groups exposure Proportion of each groups exposure assessed retrospectivelyassessed retrospectivelyStudy design easy for exploring possible Study design easy for exploring possible relationships b/t many exposures and relationships b/t many exposures and outcome of interestoutcome of interest– Even greater potential for confoundersEven greater potential for confounders– If a large number of associations are explored, If a large number of associations are explored,

a statistically significant finding may be due to a statistically significant finding may be due to chance alone.chance alone.

VSM

Cross-Sectional StudiesCross-Sectional Studies

Most common for looking at etiologyMost common for looking at etiology

Look at group with outcome and Look at group with outcome and group without, and compare group without, and compare exposureexposure

Exposures and outcomes measured Exposures and outcomes measured at same timeat same time– which came first?which came first?– Lots of confoundersLots of confounders

VSM

Case Series/Case ReportsCase Series/Case Reports

Look at few pts with adverse Look at few pts with adverse outcome while receiving suspected outcome while receiving suspected treatmenttreatment

Useful in rare and dramatic outcomesUseful in rare and dramatic outcomes

Lack comparison groupsLack comparison groups

Usually only sufficient for hypothesis Usually only sufficient for hypothesis generationgeneration

VSM

Is this evidence about harm valid?Is this evidence about harm valid?1.1. Were there clearly defined groups of Were there clearly defined groups of

patients, similar in all impt ways other patients, similar in all impt ways other than exposure to treatment or cause?than exposure to treatment or cause?

2.2. Were treatments/exposures and clinical Were treatments/exposures and clinical outcomes measured in the same ways in outcomes measured in the same ways in both groups?both groups?

3.3. Was the follow-up long enough?Was the follow-up long enough?4.4. Do the results of the harm study fulfill Do the results of the harm study fulfill

some of the diagnostic tests for some of the diagnostic tests for causation?causation?

VSM

Are outcomes and exposures Are outcomes and exposures measured the same way?measured the same way?

RCT’sRCT’s– Best if exposures and outcomes Best if exposures and outcomes

measured in same way in both groupsmeasured in same way in both groups

Cohort studiesCohort studies– Want outcome assessors blinded to Want outcome assessors blinded to

exposureexposure

Case-control studiesCase-control studies– Want pts and assessors blinded to Want pts and assessors blinded to

outcome and study hypothesisoutcome and study hypothesis

VSM

Is this evidence about harm valid?Is this evidence about harm valid?1.1. Were there clearly defined groups of patients, Were there clearly defined groups of patients,

similar in all impt ways other than exposure to similar in all impt ways other than exposure to treatment or cause?treatment or cause?

2.2. Were treatments/exposures and clinical Were treatments/exposures and clinical outcomes measured in the same ways in both outcomes measured in the same ways in both groups?groups?

3.3. Was the follow-up appropriate?Was the follow-up appropriate?4.4. Do the results of the harm study fulfill some of Do the results of the harm study fulfill some of

the diagnostic tests for causation?the diagnostic tests for causation?1.1. Is it clear the exposure preceded the onset of the Is it clear the exposure preceded the onset of the

outcome?outcome?2.2. Is there a dose-response gradient?Is there a dose-response gradient?3.3. Is the association consistent from study to study?Is the association consistent from study to study?4.4. Does the association make biological sense?Does the association make biological sense?

VSM

Was follow-up appropriate?Was follow-up appropriate?

CompleteComplete– Ideally want no pts lost to follow-upIdeally want no pts lost to follow-up– 20% loss to follow-up cut off20% loss to follow-up cut off

Long enoughLong enough– Want sufficient length of time to ensure Want sufficient length of time to ensure

all outcomes includedall outcomes included

VSM

Is this evidence about harm valid?Is this evidence about harm valid?1.1. Were there clearly defined groups of Were there clearly defined groups of

patients, similar in all impt ways other patients, similar in all impt ways other than exposure to treatment or cause?than exposure to treatment or cause?

2.2. Were treatments/exposures and clinical Were treatments/exposures and clinical outcomes measured in the same ways in outcomes measured in the same ways in both groups?both groups?

3.3. Was the follow-up long enough?Was the follow-up long enough?4.4. Do the results of the harm study fulfill Do the results of the harm study fulfill

some of the diagnostic tests for some of the diagnostic tests for causation?causation?

VSM

Do the results of the harm study fulfill some Do the results of the harm study fulfill some of the diagnostic tests for causation?of the diagnostic tests for causation?

1.1. Is it clear the exposure preceded the Is it clear the exposure preceded the onset of the outcome?onset of the outcome?

2.2. Is there a dose-response gradient?Is there a dose-response gradient?

3.3. Is the association consistent from Is the association consistent from study to study?study to study?

4.4. Does the association make biological Does the association make biological sense?sense?

VSM

Are the results important?Are the results important?

If study passes validity test, we must If study passes validity test, we must decide if association is strong enough decide if association is strong enough for us to act uponfor us to act upon

Magnitude of associationMagnitude of association– Use relative risk or odds ratioUse relative risk or odds ratio

Precision of estimate of associationPrecision of estimate of association– Use confidence interval Use confidence interval

VSM

Relative RiskRelative Risk

RCT and cohort studiesRCT and cohort studies

Relative Risk (RR)Relative Risk (RR)

RR= proportion of those with RR= proportion of those with outcome in exposed and not exposed outcome in exposed and not exposed groupsgroups

VSM

Relative RiskRelative RiskExample:Example:Exposure groupExposure group– 1000 pts receive Tx1000 pts receive Tx– 20 develop outcome20 develop outcome– a/(a+b)a/(a+b)– 20/1000= 2%20/1000= 2%

Not exposed groupNot exposed group– 1000 pts no Tx1000 pts no Tx– 2 experience outcome2 experience outcome– c/(c+d)c/(c+d)– 2/1000=0.2%2/1000=0.2%

RR=(a/(a+b))/(c/c+d)RR=(a/(a+b))/(c/c+d)

RR= 2%/0.2%=10RR= 2%/0.2%=10

PresentPresent

casecaseAbsentAbsent

controlscontrols

exposedexposed aa

2020BB

980980a+ba+b

10001000

Not Not exposedexposed

CC

22DD

998998c+dc+d

10001000

Adverse outcome

Patients receiving treatment are 10x more likely to experience the outcome.

VSM

Odds RatioOdds Ratio

Case-control studiesCase-control studies – Can’t calculate RR in studies because Can’t calculate RR in studies because

the investigator selects the people with the investigator selects the people with the outcomes (rather than the exposure)the outcomes (rather than the exposure)

– Can’t calculate “incidence”Can’t calculate “incidence”– Use “odds ratio” or relative oddsUse “odds ratio” or relative odds

OR=ad/bcOR=ad/bc

VSM

Odds Ratio Odds Ratio

Ie: urge incontinenceIe: urge incontinence– 100pt with urge incontinence100pt with urge incontinence

90 with hx of caffeine exp90 with hx of caffeine exp

10 without caffeine exp10 without caffeine exp

– 100pts without urge incontinence100pts without urge incontinence45 with exposure to caffeine45 with exposure to caffeine

55 without caffeine use55 without caffeine use

OR=ad/bcOR=ad/bc– OR=(90x55)/(45x10)OR=(90x55)/(45x10)– OR=11OR=11

casecase controlscontrols

exposedexposed aa

9090BB

4545

Not Not exposedexposed

CC

1010DD

5555

Odds of experiencing urge incontinence for people with caffeine use is 11 times that of those who did not use caffeine

VSM

OR and RROR and RROR and RR >1OR and RR >1– Increased risk of adverse eventIncreased risk of adverse event

OR and RR=1OR and RR=1– Adverse event is no more likely to occurAdverse event is no more likely to occur

OR and RR <1OR and RR <1– Adverse event is less likely to occur Adverse event is less likely to occur

OR and RR approximate each other OR and RR approximate each other when event rate and treatment when event rate and treatment effect are smalleffect are small

VSM

How big should the RR or OR be How big should the RR or OR be for us to be impressed?for us to be impressed?

Strength of study design/validity Strength of study design/validity determinedetermineProfessor IrwigProfessor Irwig– Suggests we compare the unadjusted measure Suggests we compare the unadjusted measure

of association with one in which at least one of association with one in which at least one known confounder has been adjusted outknown confounder has been adjusted out

– If adjustment produces a large decline in RR or If adjustment produces a large decline in RR or OR, be suspicious of spurious associationOR, be suspicious of spurious association

– If adjustment yields stable OR or RR, or if it If adjustment yields stable OR or RR, or if it rises, our confidence in the validity should be rises, our confidence in the validity should be greatergreater

VSM

Number Needed to Harm (NNH)Number Needed to Harm (NNH)

OR and RR tell about strength of OR and RR tell about strength of associationassociation

NNH=number of pts who need to be NNH=number of pts who need to be exposed to the causal agent to exposed to the causal agent to produce one additional harmful eventproduce one additional harmful event

VSM

RCT and Cohort StudiesRCT and Cohort Studies

Can be calculated directly from trials Can be calculated directly from trials and cohort studies and cohort studies

Reciprocal of difference in adverse Reciprocal of difference in adverse eventsevents

NNH= 1/(a/(a+b))-(c/(c+d))NNH= 1/(a/(a+b))-(c/(c+d))

VSM

Case Control StudiesCase Control StudiesCan’t determine incidenceCan’t determine incidence

ComplexComplex

If OR<1If OR<1– 1-(PEER(1-OR))/PEER(1-PEER)(1-OR)1-(PEER(1-OR))/PEER(1-PEER)(1-OR)

If OR>1If OR>1– 1+(PEER(OR-1))/PEER(1-PEER)(OR-1)1+(PEER(OR-1))/PEER(1-PEER)(OR-1)

PEER= patient expected event ratePEER= patient expected event rate

Different PEERs with same OR can lead to Different PEERs with same OR can lead to different NNHdifferent NNH

VSM

Confidence intervalConfidence intervalIn addition to magnitude of OR or RR In addition to magnitude of OR or RR must look at its precisionmust look at its precision

Credibility is highest when entire CI is Credibility is highest when entire CI is narrow and remains within a narrow and remains within a clinically importantly increased riskclinically importantly increased risk

VSM

Can this valid and important evidence about Can this valid and important evidence about harm be applied to our patient?harm be applied to our patient?

1.1. Is our patient so different from Is our patient so different from those included in the study that its those included in the study that its results cannot apply?results cannot apply?

2.2. What is our patient’s risk of benefit What is our patient’s risk of benefit and harm from the agent?and harm from the agent?

3.3. What are our patient’s preferences, What are our patient’s preferences, concerns, and expectations?concerns, and expectations?

4.4. What alternatives are available?What alternatives are available?