WHO guidelines and the GRC · Applicant submits AD to Secretariat GRC *Evaluation and discussion...

WHO guidelines and the Guidelines Review Committee

Dr Faith McLellan

The Problem

WHO guidelines: not transparent, not evidence based↓ Systematic reviews↓ Transparency about judgements↑ Expert opinion↓ Adaptation of global guidelines to end

users' needs↔ Tension between time taken and when

advice needed↓ Resources

Oxman et al, Lancet 2007;369:1883-9

WHO response

GRCStandards for:

Reporting ProcessesUse of evidence

Revised WHO handbook for guidelinesDifferent types of documents for different purposes

Guideline types

EmergencyResponse to acute need, evidence informed, limited consultation, short use-by date

Standard /focusedLimited topic area, 10-20 'questions', evidence-based, 1 guideline group meeting

ComprehensiveDisease/policy area, evidence-based, 3-4 meetings

TextbooksJoint guidelines

What is a WHO guideline?

"Guidelines are recommendations intended to assist providers and recipients of health care and other stakeholders to make informed decisions. Recommendations may relate to clinical interventions, public health activities, or government policies."

WHO 2003, 2007

But what is it, really?YES: Clinical and public health interventionsNO: Standards (eg pharmacopoeia, food), standard operating procedures, evidence synthesis without recommendations, 'how to' manuals UNCLEAR: compilations of clinical information without clear recommendationsIN ANY CASE: The name is irrelevant

WHO Guidelines Production ProcessGuidelines Review Committee

A WHO Department decides to produce a guideline

Initial publication approvalADG/DGO

Final approval by GRC (see flow chart)

Beginning End

Relevant approvals are obtained (ADG or DGO)

The guideline is produced by the WHO Department (i.e. from a few months to 2/3 years time frame)

GRC SecretariatThroughout the process of production of a guideline,

the WHO Department can access the resources provided by the GRC Secretariat.

Advice and support from members of

the GRC

Advice and support from external

experts on guideline

production

Advice and support from WHO

Collaborating Centres

Advice and support from GRC through

WHO list of technical expert

Advice and support from the GRC Secretariat

R

E

S

O

U

R

C

E

S

Initial approval by GRC (see flow chart)

Application Document (2-4 pages) Index• Copy of the Development of health information product form approved• Type of guideline and Scope • Target audience• Budget•Group members•Proposed outputs, products including translations• Plan for summarising and presenting the evidence, including external technical support• Peer review plan •Implementation plan (linkage with a project, if exists)

Initial approval flow-chart for WHO guidelines

1. Application Document (AD): 2-4 pages (see index proposed)

2. Applicant submits AD to Secretariat GRC

3. GRC reviews AD *

4. GRC meets with Applicant and discusses comments/questions *

6. Applicant reviews AD

9. Approval report is sent to Applicant, cc to relevant Director & ADG

If changes are requested, the AD should be resubmitted to GRC for approval.

8. GRC produces approval report (1 page)

If AD is approved

7. Applicant submits AD to Secretariat GRC

* Evaluation and discussion could require a meeting with the GRC and some technical support from GRC Secretariat. Methodological support to the group for framing, formulating the questions and revising the evidence could be offered.

5. GRC makes written comments/recommendations (1-2 pages)

10. The Applicant is ready to start the process of development of the guideline!

Completed Guideline Document (2-3 pages) Index• Copy of the GRC initial approval of guideline development process• Scope • Summary of DOI from group members• Reference to evidence summaries • Response to peer review • Implementation plan (linkage with a project, if exists)• Plan for update

Final approval flow-chart for WHO guidelines

1. Completed guideline (CG) plus 2-3 pages (see index proposed)

2. Applicant submits CG to Secretariat GRC

3. GRC reviews CG *

4. GRC meets with Applicant and discusses comments/questions *

6. Applicant reviews CG

9. Approval report is sent to Applicant, cc to relevant Director & ADG

If changes are requested, the CG should be resubmitted to GRC for approval.

8. GRC produces approval report (1 page)

If CG is approved

7. Applicant submits CG to Secretariat GRC

* Evaluation and discussion could require a meeting with the GRC and some technical support from GRC Secretariat. Methodological support to the group for framing, formulating the questions and revising the evidence could be offered as methodological support.

5. GRC makes written comments/recommendations (1-2 pages)

10. The Applicant is ready to send the guideline for usual approval (ADG, RD or DGO, as appropriate)

WHO Standards for guidelines

Reporting, process and use of evidence

1. Scoping the document: reasons for choosing the topic, problems with existing guidelines, variations and gaps,

2. Group composition (or consultations)

3. Conflict of interest

4. Formulations of the questions and choice of the relevant outcomes

5. Evidence retrieval, evaluation and synthesis (balance sheet, evidence table)

6. Benefit/risk profile: integrating evidence with values and preferences, equity and costs

7. Formulation of the recommendations

8. Implementation and evaluation of impact

9. Research needs or areas of further research

10. Peer-review process and updating

Title, responsible person, WHO Department- responsible of the clearance process, WHO Departments involved, CC involved,

Standards for evidence: GRADE system

Reporting standard and process

Reporting standard and process

Conflict of interest sample

Standards for evidence

PracticalitiesSynthesis of all available evidenceFormal assessment of quality of evidenceEvidence summaries for group meetings using standard template

Practicalities, cont'd

Consideration of resource use and costsLink evidence to recommendations, explaining reasons for judgements

System for assessing evidence for interventions: GRADE (Grading of Recommendations Assessment, Development and Evaluation)

Quality of evidence

The extent to which one can be confident that an estimate of effect or association is correct. Suggested categories:

HighModerateLowVery low

Quality of evidence

QUESTION: Should active management of the third stage of labour be used by skilled providers for all women to prevent postpartum hemorrhage (PPH)?

7.7low quality++oo

min 41 (26.5, 90.1)max 73 (43.3, 225.5)

0.33 (0.21, 0.51)

min 1.5% (0.6-2.4)max 3.2% (2.0-4.4)

31583126nonesome uncertainty about directness4,5

-1

no important inconsistency

serious limitation2

,3,17

-1

RCT4PW 001

Ad 97Br 88Du 90Hi 98

Blood loss ≥ 1000 ml

6.3low quality++oo

min 8 (6.7, 11.2)max 16 (11.7, 24.7)

0.38 (0.32, 0.46)

min 8.3% (6.3, 10.3) max 17.9% (15.3, 20.5)

31583126nonesome uncertainty about directness4,5

-1

no important inconsistency

serious limitation2

,3,17

-1

RCT4PW 001

Ad 97Br 88Du 90Hi 98

Blood loss ≥ 500 ml

6.4-----------0

Admission to intensive care unit

8.5-----------0

Maternal deaths

Benefits:

NNT(95%CI

)

Relative risk

(95%CI)

Baseline Risk

(95%CI)

Standard procedures

Active management

Other considera

tions

Directness

Consistency

LimitationsDesign

No of studies (Ref)

ImportanceQuality

EffectNo of patients

Summary of findingsQuality assessment

QUESTION: Should active management of the third stage of labour be used by skilled providers for all women to prevent postpartum hemorrhage (PPH)?

7.7low quality++oo

min 41 (26.5, 90.1)max 73 (43.3, 225.5)

0.33 (0.21, 0.51)

min 1.5% (0.6-2.4)max 3.2% (2.0-4.4)

31583126nonesome uncertainty about directness4,5

-1

no important inconsistency

serious limitation2

,3,17

-1

RCT4PW 001

Ad 97Br 88Du 90Hi 98

Blood loss ≥ 1000 ml

6.3low quality++oo

min 8 (6.7, 11.2)max 16 (11.7, 24.7)

0.38 (0.32, 0.46)

min 8.3% (6.3, 10.3) max 17.9% (15.3, 20.5)

31583126nonesome uncertainty about directness4,5

-1

no important inconsistency

serious limitation2

,3,17

-1

RCT4PW 001

Ad 97Br 88Du 90Hi 98

Blood loss ≥ 500 ml

6.4-----------0

Admission to intensive care unit

8.5-----------0

Maternal deaths

Benefits:

NNT(95%CI

)

Relative risk

(95%CI)

Baseline Risk

(95%CI)

Standard procedures

Active management

Other considera

tions

Directness

Consistency

LimitationsDesign

No of studies (Ref)

ImportanceQuality

EffectNo of patients

Summary of findingsQuality assessment

QUESTION: Should active management of the third stage of labour be used by skilled providers for all women to prevent postpartum hemorrhage (PPH)?

7.7low quality++oo

min 41 (26.5, 90.1)max 73 (43.3, 225.5)

0.33 (0.21, 0.51)

min 1.5% (0.6-2.4)max 3.2% (2.0-4.4)

31583126nonesome uncertainty about directness4,5

-1

no important inconsistency

serious limitation2

,3,17

-1

RCT4PW 001

Ad 97Br 88Du 90Hi 98

Blood loss ≥ 1000 ml

6.3low quality++oo

min 8 (6.7, 11.2)max 16 (11.7, 24.7)

0.38 (0.32, 0.46)

min 8.3% (6.3, 10.3) max 17.9% (15.3, 20.5)

31583126nonesome uncertainty about directness4,

5

-1

no important inconsistency

serious limitation2,3,17

-1

RCT4PW 001

Ad 97Br 88Du 90Hi 98

Blood loss ≥ 500 ml

6.4-----------0

Admission to intensive care unit

8.5-----------0

Maternal deaths

Benefits:

NNT(95%CI

)

Relative risk

(95%CI)

Baseline Risk

(95%CI)

Standard procedures

Active management

Other considera

tions

Directness

Consistency

LimitationsDesign

No of studies (Ref)

ImportanceQuality

EffectNo of patients

Summary of findingsQuality assessment

Recommendations versus evidence

Recommendations are judgementsQuality of evidenceTrade off between benefits and harmsCostsValues and preferences

Why bother about grading?

People draw conclusions about thequality of evidencestrength of recommendations

Systematic and explicit approaches can help

protect against errorsresolve disagreementsfacilitate critical appraisalcommunicate information

Strength of a recommendation

Strength of a recommendationStrong vs. weak?

Strength of a recommendationStrong vs weak?

Strong/weakStrong/conditionalStrong/qualified

website

http://intranet.who.int/homes/rpc/grc/.

For assistance, advice, and further informationGuidelines HELP CLINIC

Thursdays, 1400 to 1600, Rooms 4336 and 4341GRC Secretariat contacts:Dr Faith McLellan mclellanf@who.intDr Gunn Vist vistg@who.intMs Silke Walleser wallesers@who.int