Upload
alvin-hancock
View
213
Download
0
Embed Size (px)
Citation preview
Why Grade Why Grade Recommendations?Recommendations?
• strong recommendationsstrong recommendations– strong methods strong methods – large precise effect large precise effect – few down sides of therapyfew down sides of therapy
• weak recommendationsweak recommendations– weak methodsweak methods– imprecise estimateimprecise estimate– small effectsmall effect– substantial down sidessubstantial down sides
Why Grade Recommendations?Why Grade Recommendations?
• strong recommendationstrong recommendation– one size fits allone size fits all– expect uniform clinician behaviorexpect uniform clinician behavior
• weaker recommendation weaker recommendation – expect action to varyexpect action to vary
Grading recommendations Grading recommendations is not grading studiesis not grading studies
• developing recommendationsdeveloping recommendations– grade (assess) individual studiesgrade (assess) individual studies– grade (assess) overall evidencegrade (assess) overall evidence– grade recommendations that flow from evidencegrade recommendations that flow from evidence
• rating schemes for individual studiesrating schemes for individual studies– prognosis, diagnosis, managementprognosis, diagnosis, management
• recommendations focus on patient outcomerecommendations focus on patient outcome– showing a test, or prognostic tool, is accurate showing a test, or prognostic tool, is accurate
generally not enoughgenerally not enough– RCTs of screening testsRCTs of screening tests
An ideal grading systemAn ideal grading system
• simplesimple• clear and separate consideration of clear and separate consideration of
two issuestwo issues– methodological quality of evidencemethodological quality of evidence
• likelihood of biaslikelihood of bias
– trade-off between benefits and riskstrade-off between benefits and risks
Which grading system to Which grading system to use?use?
• many availablemany available– Australian National and MRCAustralian National and MRC– Oxford Center for Evidence-based MedicineOxford Center for Evidence-based Medicine– Scottish Intercollegiate Guidelines (SIGN)Scottish Intercollegiate Guidelines (SIGN)– US Preventative Services Task ForceUS Preventative Services Task Force– American Heart AssociationAmerican Heart Association
• conflict of interest (point of view)conflict of interest (point of view)– American College of Chest PhysiciansAmerican College of Chest Physicians– Cochrane-related group led by Andy OxmanCochrane-related group led by Andy Oxman
What makes weaker evidence?What makes weaker evidence?
• weaker study designweaker study design– basicbasic– detailed design and executiondetailed design and execution
• inconsistencyinconsistency
• indirectnessindirectness
• reporting biasreporting bias
Methodological QualityMethodological Quality
• study designstudy design– randomizationrandomization– quasi-randomizationquasi-randomization– observational studyobservational study
• detailed design and executiondetailed design and execution– concealmentconcealment– balance in known prognostic factorsbalance in known prognostic factors– intention to treat principle observedintention to treat principle observed– blindingblinding– completeness of follow-upcompleteness of follow-up
ConsistencyConsistency
• evidence weaker if results differevidence weaker if results differ
• if inconsistency, look for if inconsistency, look for explanationexplanation– patientspatients– interventionintervention– outcomeoutcome– methodsmethods
Relative Risk of Conversion to Sinus RhythmAmiodarone vs Placebo or Digoxin or CCB
Favours Control Favours Amiodarone
Cowan 1986 1.11 (0.78 to 1.58)Noc 1990 18.0 (1.17 to 276)
Capucci 1992 0.77 (0.37 to 1.62)Cochrane 1994 1.15 (0.91 to 1.44)Donovan 1995 1.05 (0.69 to 1.60)
Hou 1995 1.29 (0.97 to 1.72)Kondili 1995 1.33 (0.71 to 2.47)Galve 1996 1.13 (0.84 to 1.52)
Kontoyannis 1998 1.42 (1.08 to 1.85)Bellandi 1999 1.41 (1.15 to 1.72)
Cotter 1999 1.43 (1.15 to 1.80)Kochiadakis 1999 1.46 (1.19 to 1.78)
Bianconi 2000 2.04 (0.19 to 22.00)Galperin 2000 33.70 (2.08 to 546)
Hohnloser 2000 3.13 (1.50 to 6.70)Joseph 2000 1.32 (0.95 to 1.80)Natale 2000 5.12 (2.60 to 10.00)
Peukurinen 2000 2.45 (1.49 to 4.02)Vardas 2000 2.01 (1.55 to 2.60)Villani 2000 4.75 (1.60 to 14.00)
Cybulski 2001 1.87 (1.37 to 2.55)0.1 1 10 100
n = 83
n = 95
n = 203
n = 85
n = 120
n = 34
n = 24
n = 40
n = 30
n = 64
n = 39
n = 42
n = 100
n = 42n = 120
n = 100
n = 204
n = 75
n = 62
n = 208
n = 160
Relative Risk of Conversion to Sinus RhythmAmiodarone vs Placebo or Digoxin or CCB
Favours Control Favours Amiodarone
Bianconi 2000 2.04 (0.19 to 22.00)Galperin 2000 33.70 (2.08 to 546)
Hohnloser 2000 3.13 (1.50 to 6.70)Natale 2000 5.12 (2.60 to 10.00)Villani 2000 4.75 (1.60 to 14.00)
Pooled Estimate 4.33 (2.76 to 6.77)
Cowan 1986 1.11 (0.78 to 1.58)Noc 1990 18.0 (1.17 to 276)
Capucci 1992 0.77 (0.37 to 1.62)Cochrane 1994 1.15 (0.91 to 1.44)Donovan 1995 1.05 (0.69 to 1.60)
Hou 1995 1.29 (0.97 to 1.72)Kondili 1995 1.33 (0.71 to 2.47)Galve 1996 1.13 (0.84 to 1.52)
Kontoyannis 1998 1.42 (1.08 to 1.85)Bellandi 1999 1.41 (1.15 to 1.72)
Cotter 1999 1.43 (1.15 to 1.80)Kochiadakis 1999 1.46 (1.19 to 1.78)
Joseph 2000 1.32 (0.95 to 1.80)Peukurinen 2000 2.45 (1.49 to 4.02)
Vardas 2000 2.01 (1.55 to 2.60)Cybulski 2001 1.87 (1.37 to 2.55)
Pooled Estimate 1.40 (1.25 to 1.57)0.1 1 10 100
AF Duration > 48 hrs
AF Duration =/< 48 hrs
n = 83n = 95
n = 203n = 85
n = 120
n = 34
n = 24n = 40
n = 30n = 64n = 39
n = 42n = 100n = 42n = 120n = 100n = 204n = 75
n = 62
n = 208n = 160
Directness of EvidenceDirectness of Evidence
• indirect treatment comparisonsindirect treatment comparisons– interested in A versus Binterested in A versus B– have A versus C and B versus Chave A versus C and B versus C
• alendronate vs risedronatealendronate vs risedronate– both versus placebo, no head-to-headboth versus placebo, no head-to-head
Who are the patients? Who are the patients? If different, weaker evidence If different, weaker evidence
• patients meet trials’ eligibility criteria patients meet trials’ eligibility criteria
• minor question of directnessminor question of directness– slight age difference slight age difference – comorbidity comorbidity – racerace– closely related underlying condition closely related underlying condition
• valvular atrial fibrillationvalvular atrial fibrillation
• serious question about biologyserious question about biology– heart failure trials applicability to aortic stenosisheart failure trials applicability to aortic stenosis
Interventions:Interventions:if differ, weaker evidenceif differ, weaker evidence
• identical interventionidentical intervention– captopril 100 mgcaptopril 100 mg
• minor question of directnessminor question of directness– captopril, lower dosecaptopril, lower dose– other ACE inhibitorother ACE inhibitor
• serious question of biologyserious question of biology– angiotensin receptor blockerangiotensin receptor blocker
Outcome differs, Outcome differs, weaker evidenceweaker evidence
• outcomes of direct interestoutcomes of direct interest– survival of long durationsurvival of long duration– valid measure of HRQLvalid measure of HRQL
• minor question of directnessminor question of directness– follow-up shorter than idealfollow-up shorter than ideal– functional status measurefunctional status measure
• serious question of biology serious question of biology – short duration of follow-upshort duration of follow-up– laboratory exercise capacitylaboratory exercise capacity
Magnitude, Precision, Magnitude, Precision, Reporting BiasReporting Bias
• magnitude not generally part of qualitymagnitude not generally part of quality– but very large magnitude can upgradebut very large magnitude can upgrade
• precision not generally part of qualityprecision not generally part of quality– but sparse data can lower qualitybut sparse data can lower quality
• reporting biasreporting bias– high likelihood can lower qualityhigh likelihood can lower quality
Grading SystemGrading System
• high qualityhigh quality well done RCTwell done RCT
• intermediateintermediate quasi-RCTquasi-RCT
• lowlow well done observationalwell done observational
• insufficient insufficient anything elseanything else
Moving DownMoving Down
• study execution– serious flaws can lower by one level– fatal flaws can lower by two levels
• consistency– important inconsistency can lower by one level
• directness of evidence– some uncertainty re relevance lower by one level– major uncertainty re relevance lower by two levels
• selection bias – strong suspicion lower by 1 level
Moving UpMoving Up
• very strong association, up 2 levels– insulin in diabetic ketoacidosis
• strong, consistent association with no plausible confounders, up 2 levels – fluoride for preventing cavities
• strong association can move up 1 level– ? HRT ?
Risk/Benefit tradeoffRisk/Benefit tradeoff• aspirin after myocardial infarctionaspirin after myocardial infarction
– meta-analysis of high quality RCTsmeta-analysis of high quality RCTs– clear positive treatment effectclear positive treatment effect– unequivocal recommendation to treat, unequivocal recommendation to treat,
uniform practiceuniform practice
• accelerated TPA vs streptokinase after accelerated TPA vs streptokinase after MIMI– very large single trialvery large single trial– clear positive effectclear positive effect– uncertainty whether to treat, varying uncertainty whether to treat, varying
practicepractice
Risk/Benefit tradeoffRisk/Benefit tradeoff
• Aspirin after myocardial infarctionAspirin after myocardial infarction– 25% reduction in relative risk25% reduction in relative risk– side effects minimal, cost minimalside effects minimal, cost minimal– benefit obviously much greater than benefit obviously much greater than
risk/costrisk/cost
• TPA vs streptokinase after MITPA vs streptokinase after MI– 12% reduction in relative risk12% reduction in relative risk– increased rate of intracranial hemorrhageincreased rate of intracranial hemorrhage– large increase in costlarge increase in cost– benefit vs risk/cost a judgment callbenefit vs risk/cost a judgment call
Strength of Strength of RecommendationsRecommendations
Aspirin after MI – do itAspirin after MI – do it
TPA rather than SK in MI TPA rather than SK in MI -- probably do it-- probably do it
-- probably don’t do it-- probably don’t do it
Grade of Grade of RecommendationsRecommendations
•do it or don’t do itdo it or don’t do it– strong recommendationstrong recommendation
•probably do it, or probably don’tprobably do it, or probably don’t– weaker recommendationweaker recommendation
Strong or Weak Recommendation?Strong or Weak Recommendation?
• seriousness of outcomeseriousness of outcome• magnitude of effectmagnitude of effect• precision of treatment effectprecision of treatment effect• risk of target eventrisk of target event• risk of adverse eventsrisk of adverse events• cost of therapycost of therapy• values and preferencesvalues and preferences
Weak recommendationWeak recommendation
• practice will vary practice will vary – according to what?according to what?
• interpretation of evidenceinterpretation of evidence– breast cancerbreast cancer
• patients’ values and preferencespatients’ values and preferences– atrial fibrillationatrial fibrillation
• inclination to gambleinclination to gamble– HRTHRT
ConclusionConclusion• challenges in gradingchallenges in grading
– simple systemsimple system– consider quality of evidenceconsider quality of evidence– consider benefit/risk tradeoffconsider benefit/risk tradeoff
• methodological quality of evidencemethodological quality of evidence– study design, execution, study design, execution, – consistency, directness consistency, directness – reporting biasreporting bias
• balance of benefits and risks/costbalance of benefits and risks/cost– magnitude and precision of effects; magnitude and precision of effects; – values and preferencesvalues and preferences
Way forwardWay forward
• an internationally accepted an internationally accepted grading scheme?grading scheme?
• reliability of grading scheme reliability of grading scheme – will observers come to same grade?will observers come to same grade?
• acceptability and usefulnessacceptability and usefulness– workshop this afternoonworkshop this afternoon