Standards for Developing Trustworthy

Clinical Practice Guidelines

Standards for Developing Trustworthy

Clinical Practice Guidelines

Institute of MedicineJanuary 11, 2010

Institute of MedicineJanuary 11, 2010

Sandra Zelman Lewis, PhD

Asst VP, Health & Science Policy

American College of Chest Physicians

Sandra Zelman Lewis, PhD

Asst VP, Health & Science Policy

American College of Chest Physicians

Most Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: Definition

1. Need for a universally accepted definition of evidence-based guidelines (EBGs)

The problem Consensus statements and “even less structured”

documents are often designated as “guidelines” Physicians are misled into thinking that these are

evidence-based and methodologically rigorous Patient care can be adversely impacted

Most Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: Definition

Most Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: DefinitionMost Important Challenge: Definition

Another Important Challenge: FundingAnother Important Challenge: FundingAnother Important Challenge: FundingAnother Important Challenge: Funding

2. Funding guidelines is the biggest challenge that guidelines developers face.

The problem EBGs are very expensive if done correctly Little funding available, difficult to obtain

Charge to this committeeIdentify sources for funding guidelines produced according to the established standards

When Evidence is Nonexistent or Poor QualityWhen Evidence is Nonexistent or Poor QualityWhen Evidence is Nonexistent or Poor QualityWhen Evidence is Nonexistent or Poor Quality

o What does ACCP do when the scientific evidence is absent or poor?

• Sets minimal threshold for evidence: Must be published in a peer-reviewed journal

• Downgrade recs when poor quality evidence

• Allow consideration of indirect evidence if justification (described in text) acceptable but downgrade appropriately

• If evidence is not sufficient, then discuss in text but do not provide recommendation

Disagreements in Interpretations of EvidenceDisagreements in Interpretations of EvidenceDisagreements in Interpretations of EvidenceDisagreements in Interpretations of Evidence

o How do you reconcile disagreements in evidence interpretation among guidelines?

• Assess rigor of methodology/adherence to the evidence

• Request invited associations to review and comment on our guideline recommendations

• Offer competing guideline organizations opportunity to appoint a member to our next edition or update panel (providing he/she passes COI vetting and approval process)

Accommodating SubgroupsAccommodating SubgroupsAccommodating SubgroupsAccommodating Subgroups

o How do guidelines accommodate subgroups (ex. older populations or persons with multi-morbidities) whose treatment outcomes may differ from the average patient?

• If data exist for subgroups, use it.

• If not, use indirect evidence from similar groups and downgrade appropriately

• Always describe patient population in each recommendation based on the patient population in the original studies

• A major challenge is multiple co-morbidities

Setting StandardsSetting StandardsSetting StandardsSetting Standards

o What topics and/or processes do you think the committee should consider in deriving quality standards for clinical practice guidelines? 

• Establish a definition of evidence-based guidelines, possibly with rating scales

• Standard setting topics and criteria. PCPI (Physician Consortium for Performance

Improvement) criteria for guidelines permitted as basis of performance measures

. AGREE instrument (AGREE III is in development)

. COGS (Conference on Guideline Standardization)

. GLIA – for implementation purposes

Panel CompositionPanel CompositionPanel CompositionPanel Composition

o What should the composition of CPG development panels, in particular the balance of methodologists, topical experts, and consumers, look like?

• Depends on structure of panel but should have methodologists reviewing evidence and developing evidence tables or profiles

• Consumers must be educated in EBM

• Content experts provide credibility but all must go through rigorous review, including COIs

• Other considerations: health economists, frontline clinicians, patient preference consultants

Grading SystemGrading SystemGrading SystemGrading System

o Is there an available assessment tool that adequately rates both the level of the scientific evidence and strength of clinical recommendations that should be used as standard practice in guideline development?

• The ACCP Grading System . A modification of GRADE (major difference: restricts

evidence to approved threshold)

. Based on (1) assessment of the quality of the original studies and (2) a balance of the risks to benefits

. Has been adopted by other guideline developers

Grading SystemGrading SystemGrading SystemGrading System

Grade of rec

Benefit versus risk and burdens

Methodologic quality of supporting evidence

1A Benefits clearly outweigh risks/burdens, or vice versa (negative recs)

Consistent evidence from randomized controlled trials without important limitations, or exceptionally strong evidence from observational studies

1B Benefits clearly outweigh risks/burdens, or vice versa (negative recs)

Evidence from randomized controlled trials with important limitations (inconsistent results, methodologic flaws, indirect or imprecise), or very strong evidence from observational studies

1C Benefits clearly outweigh risks/burdens, or vice versa (negative recs)

Evidence for at least one critical outcome from observational studies, case series, or from randomized, controlled trials with serious flaws or indirect evidence

2A Benefits closely balanced with risks/burdens

Consistent evidence from randomized controlled trials without important limitations, or exceptionally strong evidence from observational studies

2B Benefits closely balanced with risks/burdens

Evidence from randomized, controlled trials with important limitations (inconsistent results, methodologic flaws, indirect or imprecise), or very strong evidence from observational studies

2C Benefits closely balanced with risks/burdens

Evidence for at least one critical outcome from observational studies, case series, or from randomized, controlled trials with serious flaws or indirect evidence

Recommendations for PMsRecommendations for PMsRecommendations for PMsRecommendations for PMs

o What methods might be developed for determining which recommendations among those in a guideline should be applied to quality measures or electronic medical record decision prompts?    

• Evidence should dictate direction and strength of recs

• Suggest some 1A and 1B recs for PMs (although not all)

• 1C, 2A, 2B, and 2C recs should generally not be used

• However, all should be evaluated based on feasibility, usability, scientific importance, practicality, and applicability

QI and HarmonizationQI and Harmonization

o What administrative (eg, accreditation) or legal approaches might improve the quality of clinical practice guidelines?• Published guideline quality rating scale

• Listed on NGC Web site • Currency rating also listed in NGC• PMs & CMS policies should be based on highly rated

EBGs

o What explicit approaches might harmonize guideline developers and increase guidelines convergence?• Funding requires multiple societies to collaborate• Require compliance with evidence-based standards,

rating scale

EBG PromotionEBG PromotionEBG PromotionEBG Promotion

o What types of strategies might promote greater utilization of guidelines?

• Requirements for implementation into:. EMRs. Registries. PMs, including PQRI

• Education to allow developers to learn about new techniques and processes (see last slide)

• Healthcare providers need to know how to find good guidelines and good guidelines should address the needs of providers (ask frontline clinicians what they need!)

Other ConsiderationsOther ConsiderationsOther ConsiderationsOther Considerations

o Are there other characteristics of guideline standards you think are important for the committee to consider?

• Most important: address funding• Also should move the field toward incorporating

resource considerations into the EBG recs

Promotion of IOM StandardsPromotion of IOM StandardsPromotion of IOM StandardsPromotion of IOM Standards

The IOM report and new standards should be presented to appropriate audiences:

Guideline developers will be attending the Guidelines International Network conference and guideline methodology course

Guidelines International Network (G-I-N)

2010 Conference - ChicagoAugust 26-28, 2010 – Conference Dates

August 25, 2010 – Pre-meeting Course in Guideline Methodology

Chicago, Illinois, USA

www.GIN2010.org

Host: American College of Chest Physicians