The Science of GuidelinesThe 7th ACCP Conference on Antithrombotic and

Thrombolytic Therapy: Evidence-Based Guidelines

Holger Schünemann, MD, PhD

Italian National Cancer Institute, Rome, Italy McMaster University, Hamilton, Canada

University at Buffalo, NY, USA

Topics for this talk

What makes guidelines evidence basedin 2005?

High- vs low-quality evidence Strong vs weak recommendations Example recommendation Example of the influence of values,

preferences, and cost Grading system

Evidence – recommendation: transparent link

Explicit inclusion criteria Comprehensive search

Standard consideration of study quality

Conduct/use meta-analysis

Grade recommendations

Acknowledge values and preferences underlying recommendations

What makes guidelines evidence based in 2005?

Schünemann J, et al. Chest.

2004;126 Suppl 3:688S-696S.

Background First ACCP guidelines in 1986 (J. Hirsh; J. Dalen) Initially aimed at consensus Group of experts and methodologists formally

convening every 2 to 3 years ~260,000 copies in 2001 7th conference held in 2003 87 panel members 22 chapters Across subspecialities Over 500 recommendations; 230 new Evidence-based recommendations

ACCP = American College of Chest Physicians.

Schünemann HJ, et al. Chest. 2004;126 Suppl 3:174S-178S.

Schünemann HJ, et al. Chest. 2004;126 Suppl 3:174S-178S.

The clinical question

Table 1 Eligibility criteria

Section Inclusion criteria

Population Intervention(s) or exposure

Outcome Methodology

… … … … …

4.1. Patients with unstable

angina, MI, TIA, and non-acute stroke

Any antiplatelet drug compared with placebo

or one or more other antiplatelet drug(s)

Death Stroke or recurrent stroke Other vascular events

RCTs

4.2 Patients with

cardioembolic stroke Oral anticoaluation

Death Stroke or recurrent stroke

RCTs

… … … … …

Albers GW, et al. Chest. 2004;126 Suppl 3:483S-512S.

Transparent link: from evidence to recommendations Explicit inclusion criteria

MI = myocardial infarction; RCTs = randomized controlled trials; TIA = transient ischaemic attack.

Comprehensive search for evidence Use questions to develop search strategy

– e.g. identify all search terms (MeSH and keywords) for antiplatelet drugs or MI

Search– Cochrane Database of Systematic Reviews– Database of Abstracts of Reviews of Effectiveness

– Cochrane Central Register of Controlled Trials – MEDLINE and EMBASE (1966 to December 2002)– ACP Journal Club

Provide search results– use EndNote® software – e.g. 490 citations on thrombolysis in acute stroke

ACP = American College of Physicians; MeSH = Medical Subject Headings.

Schünemann HJ et al. Chest 2004

Schunemann HJ, et al. Chest. 2004;126 Suppl 3:174S-178S

The ACCP grading system:GRADE* approach

Clear separation of 2 issues: Evidence: very low, low, moderate, or high

quality?– methodological quality of evidence– likelihood of bias

Recommendation: weak or strong?– trade-off between benefits and downsides– patient values and preferences

*www.GradeWorking-Group.orgGRADE = Grading of Recommendations Assessment,Development and Evaluation. GRADE Working Group. BMJ. 2004;328:1490-9.

Why grade recommendations?

People draw conclusions about the– quality of evidence and strength of recommendations

Systematic and explicit approaches can help– protect against errors, resolve disagreements

– communicate information

Change practitioner behaviour Strong: apply uniformly

– just do it

Weak: think about it– examine evidence yourself, consider patient circumstances

very carefully and explore with the patient

However, wide variation in approaches (GRADE)

GRADE Working Group. BMJ. 2004;328:1490-9.

Grades of recommendation:methodological quality

High (A): consistent results from RCTs or observational studies with very strong association and secure generalization

Moderate (B): inconsistent results from RCTs or RCTs with methodological limitations

Low (C): unbiased observational studies (e.g. well-executed cohort studies)

Very low (D): other observational studies (e.g. case series)

GRADE Working Group. BMJ. 2004;328:1490-9.

RCT starts high –what moves quality down?

Flawed design and execution Inconsistency Indirectness Imprecision Reporting bias

GRADE Working Group. BMJ. 2004;328:1490-9.

Design and execution

Concealment Intention-to-treat principle observed Blinding Completeness of follow-up Early stopping

GRADE Working Group. BMJ. 2004;328:1490-9.

Moving quality up:observational studies – high or moderate quality?

Strong association– strong association: RR > 2 or RR < 0.5– very strong association: RR > 5 or RR < 0.2

Dose–response relationship– bleeding risk associated with increasing INR

(blood thinning with warfarin)

Plausible confounders would have reduced the effect

INR = International Normalized Ratio;RR = relative risk. GRADE Working Group. BMJ. 2004;328:1490-9.

Grades of recommendation:strength of recommendations

Stronger recommendations (we recommend)– high-quality methods with large, precise effect – benefits much greater than downsides, or downsides much

greater than benefits– do it or don’t do it – we recommend– Grade 1

Weak recommendations (we suggest)– lower-quality methods with imprecise estimate– benefits not clearly greater or smaller than downsides– values and preferences very important– probably do it or probably don’t do it – we suggest– Grade 2

Example: stroke prevention

In patients with history of non-cardioembolic stroke or TIA…, we recommend treatment with an antiplatelet agent (Grade 1A). Aspirin, aspirin + XR dipyridamole, or clopidogrel are all acceptable options for initial therapy.

Clopidogrel: higher cost

If we had to make a choice between aspirin and clopidogrel, what would that choice be?

Albers GW, et al. Chest. 2004;126 Suppl 3:483S-512S. XR =extended release.

CAPRIE trial

Aspirin vs clopidogrel in patients at risk for cardiovascular event

19,185 patients, 3 subgroups with > 6,300 patients each (TIA/stroke; MI; peripheral arterial occlusive disease)

Mean duration of follow-up: 1.9 years Primary outcome: ischaemic stroke, MI,

or vascular death

CAPRIE Steering Committee.Lancet. 1996;348:1329-39. CAPRIE = Clopidogrel versus Aspirin in Patients at Risk of Ischaemic Events.

7.3

23.8

8.7

-3.7

-30

-20

-10

0

10

20

30

40

Relative risk reduction

%Relative risk

increase

Clopidogrel better

(Aspirin better) STROKE MI PAOD Total

p = 0.26 0.66 0.0028 0.043

CAPRIE trial results:relative risk reduction

CAPRIE Steering Committee. Lancet. 1996;348:1329-39. PAOD = peripheral arterial occlusive disease.

CAPRIE trial results:absolute risk

0

2

4

6

8

10

Absolute risk%

Clopidogrel 7.15 5.03 3.71 5.32

Aspirin 7.71 4.84 4.86 5.83

Stroke MI PAOD Total

*p < 0.05

*p < 0.05

NNT 200

CAPRIE Steering Committee. Lancet. 1996;348:1329-39. NNT = number needed to treat.

Which of the following recommendations should be given?

1. Aspirin over clopidogrel in patients with prior history of TIA/stroke?– OPTION 1

2. Clopidogrel over aspirin in patients with prior history of TIA/stroke?– OPTION 2

Audience at a prior thrombosis meeting

57% 43%

0%

20%

40%

60%

80%

100%

Aspirin Clopidogrel

Preferred recommendation

Values and preferences

Underlying values and preferences always present

Sometimes crucial

Important to make explicit

Judgements about recommendations

1. Benefit and downside evaluation

Benefits << downsides

Benefits ? downsides

Benefits ? downsides

Benefits >> downsides

? ?

2. Recommendation (wording)

STRONG

Recommend

don’t do it / should not do it

WEAKSuggestprobably don’t do it / might not do it

WEAKSuggestprobably do it/ might do it

STRONGRecommenddo it / should do it

Example: stroke prevention

In patients with history of non-cardioembolic stroke or TIA…

…we recommend treatment with an antiplatelet agent (Grade 1A). Aspirin, aspirin + XR dipyridamole, or clopidogrel are all acceptable options for initial therapy

…, we suggest use of clopidogrel over aspirin (Grade 2B)

Underlying values and preferences: This recommendation places a relatively high value on a

small absolute risk reduction in stroke rates, and a relatively low value on minimizing drug expenditures

Albers GW, et al. Chest. 2004;126 Suppl 3:483S-512S.

Judgement: benefits vs downsides*

(Quality of evidence) Relative importance of the outcomes

(benefits, harms, and burden) Baseline risk of outcomes Magnitude of the effect (RR) Absolute benefit and harm Precision of the estimates Cost

*Downsides include harm, burden, and cost

Grade of Recommendation Clarity of risk/benefit Quality of supporting evidence Strong recommendation High-quality evidence 1A

Benefits clearly outweigh risk and burdens, or vice versa

Consistent evidence from well-performed RCTs or overwhelming evidence of some other form. Further research is unlikely to change our confidence in the estimate of benefit and risk.

Strong recommendation Moderate-quality evidence 1B

Benefits clearly outweigh risk and burdens, or vice versa

Evidence from RCTs with important limitations (inconsistent results, methodological flaws, indirect or imprecise), or very strong evidence of some other research design.

Strong recommendation Low-quality evidence 1C

Benefits appear to outweigh risk and burdens, or vice versa

Evidence from observational studies, or from RCTs with serious flaws. Any estimate of effect is uncertain.

Strong recommendation Very low-quality evidence 1D

Benefits possibly outweigh risk and burdens, or vice versa

Evidence from unsystematic clinical observations

Weak recommendation High-quality evidence 2A

Benefits closely balanced with risks and burdens

Consistent evidence from well-performed RCTs or overwhelming evidence of some other form. Further research is unlikely to change our confidence in the estimate of benefit and risk.

Weak recommendation Moderate-quality evidence 2B

Benefits closely balanced with risks and burdens, some uncertainly in the estimates of benefits, risks and burdens

Evidence from RCTs with important limitations (inconsistent results, methodological flaws, indirect or imprecise), or very strong evidence of some other research design.

Weak recommendation Low-quality evidence 2C

Uncertainty in the estimates of benefits, risks, and burdens;

Evidence from observational studies, or from RCTs with serious flaws. Any estimate of effect is uncertain.

Very weak recommendation Very low-quality of evidence 2D

Major uncertainty in the estimates of benefits, risks, and burdens;

Evidence from unsystematic clinical observations,

Guyatt G, et al. Chest. 2004;126 Suppl 3:179S-187S.

Summary

Guidelines require evidence-based methods GRADE approach to grading Integration of values and preferences Grade 1: strong recommendation Grade 2: weaker recommendation/suggestion High transparency between evidence and

recommendations

End