To Cochrane or not: that's the question

AUSTRALASIAN COCHRANE CENTRE

To Cochrane or not: that's the question

1

Professor Hesham Al-InanyEditor, Cochrane Menstrual Disorders &

Subfertility Group Editor, British Journal of Obstetrics & gynecology

Cairo University Egypt


Preparing, maintaining and disseminating systematic reviews of the effects of health care

THE COCHRANE COLLABORATION


Antenatal corticosteroids for preterm birth

• Survey: • 1980 RCOG survey: 42%• 1984 Belgium : 32%• 1990 UK centers : 58%• 1992 US Obs: 8-34%

• NIH :• ”corticosteroid Rx is indicated for women at risk of

premature delivery ……….. and will result in a substantial decrease in neonatal morbidity and mortality as well as substantial savings”


Learning objectives

Why are systematic reviews important? Why Cochrane systematic reviews? How to join for a Cochrane systematic review?

4


How do we treat our patients?

Using evidence from high quality research to answer specific questions


For example:

In women with infertility does acupunctureimprove pregnancy outcomes ?


7RCT : Gold standard

Participants

R a

n d

o m

l

y A

s s

i g

n e d

Intervention Group

Control Group

Follow-up

Follow-up

Intervention Group

Control Group


But there are many RCTs on this topic

Some with conflicting results Some with small number of

participants


Why are systematic reviews useful?


Systematic review

Uses tools to summarise the results of a number of trials/studies

Systematic and scientific methodology Helpful in clinical decision making


Advantages

• To increase power, Many individual studies are too small to detect small effects, but when several are combined there is a higher chance of detecting an effect.

• To improve precision: The estimation of an intervention effect can be improved when it is based on more data.


Review: SteffComparison: 01 Absence of evidence and Evidence of absence Outcome: 01 Increasing the amount of data...

Study Treatment Control OR (fixed) OR (fixed)or sub-category n/N n/N 95% CI 95% CI

1 study 10/100 15/100 0.63 [0.27, 1.48] 2 studies 20/200 30/200 0.63 [0.34, 1.15] 3 studies 30/300 45/300 0.63 [0.38, 1.03] 4 studies 40/400 60/400 0.63 [0.41, 0.96] 5 studies 50/500 75/500 0.63 [0.43, 0.92]

0.1 0.2 0.5 1 2 5 10

Favours treatment Favours control

In the example below, as more data is included, the overall odds ratio remains the same but the confidence interval decreases.

It is not true that there is ‘no difference’ shown in the first rows of the plot – there just isn’t enough power to show a statistically significant result.


1. Clear objectives stated with a well formulated clinical question

2. Inclusion and exclusion criteria stated

3. Comprehensive search strategy

4. Systematic approach with all relevant studies

5. Excluded studies listed with reasons

6. Trial results are pooled if possible (meta-analysis)

Features not in narrative reviews


1. Clearly stated objective with clinical question

Participants Intervention Comparison Outcome

In women with infertility does acupunctureimprove pregnancy outcomes ?


2. Inclusion and exclusion criteria pre-statedParticipants Intervention Comparison Outcome

In women with infertility does acupuncture treatment improve pregnancy outcomes ?

• Women <45 yo• Infertility for > 1

yr• No other Rx or

ART?

Acupunture Live birth rateAdverse events

Sham Acupuncture or no treatment


3. Comprehensive search for all studies

All databases are searched to avoid publication bias? MEDLINE, EMBASE, CINHAL and others


Publication Bias

Research with statistically significant results is more likely to be submitted for publication and more likely to be published.


Advanced ovarian cancer

Comparison: combination chemotherapy versus single chemotherapy

Published trials

0.7 1 1.3

Registered trials

Simes 1987

Outcome: mortality


Not restricted to English language


4. Systematic approach

Independent tasks by 2 or more reviewers: Selection of studies for inclusion and

tabulated Data extraction Quality assessment Data entryDisagreements resolved by 3rd party


5. Excluded studies listed

All excluded studies included in a table Reasons given for exclusion Transparency minimises bias


6. Pooling of results using meta-analysis if appropriate and possible

What is a meta-analysis?

A statistical tool to combine results of studies

Using Revman software (free to download)

Valuable way to display a summary of the trials


Why do we need it

To avoid Type II error: Investigators did not detect a difference when a difference actually exists


Review: SteffComparison: 01 Absence of evidence and Evidence of absence Outcome: 01 Increasing the amount of data...

Study Treatment Control OR (fixed) OR (fixed)or sub-category n/N n/N 95% CI 95% CI

1 study 10/100 15/100 0.63 [0.27, 1.48] 2 studies 20/200 30/200 0.63 [0.34, 1.15] 3 studies 30/300 45/300 0.63 [0.38, 1.03] 4 studies 40/400 60/400 0.63 [0.41, 0.96] 5 studies 50/500 75/500 0.63 [0.43, 0.92]

0.1 0.2 0.5 1 2 5 10

Favours treatment Favours control


Graphic Display:ß blockers in secondary prevention after myocardial infarction.


Cumulative Meta-analysis


Meta-analysis vs. a “Mega-study”

Single large studies are liable to:

Long duration

Huge funding

a drug that reduces mortality by 10% from myocardial

infarction may need a study including 10.000 patients


Why are systematic reviews important? Why Cochrane systematic reviews? How to prepare a protocol for a Cochrane

systematic review

28


Advantages of Cochrane reviews

Updating of reviews regularly Methodologically robust Less likely to be biased


How do (Cochrane) systematic reviews reduce bias?

Prestated methods published Limit publication bias (very

comprehensive search) Limit commercial and industry

sponsorship of reviews


Studies with funding bias are more likely to have favourable results (Lundh et al 2012)

Add reviews on this topic


“..we observed far superior reporting standards of Cochrane reviews compared to non-Cochrane therapeutic ones.”

“For therapeutic reviews, all the Cochrane ones reported assessing the quality of included studies whereas only half of the non-Cochrane did.”

“The seven industry supported reviews that had conclusions recommended the experimental drug without reservations, compared with none of the Cochrane reviews, although the estimated treatment effect was similar on average

Better quality reviews


Our structure

• more than 27,000 people in over 100 countries almost all volunteers

• decentralised structure 52 Cochrane Review Groups (CRGs)

• specific areas of health care• first point of contact for authors• international and multidisciplinary

• largely funded by government grants


Cochrane centers all over the world

AUSTRALASIAN COCHRANE CENTREwww.thecochranelibrary.

com


Cochrane Reviews

Randomised controlled trials

Methodology papers





Displaying results graphically

Revman software produces forest plots


there’s a label to tellyou what the comparisonis and what the outcomeof interest is


At the bottom there’sa horizontal line. This is the scale measuringthe treatment effect.


The vertical line in themiddle is where thetreatment and control have the same effect – there is no differencebetween the two


For each study there is an id

The data foreach trial are here, divided into the experimental and control groups

This is the % weightgiven to thisstudy in the pooled analysis


• Each study is given a blob, placed where the data measure the effect.

• The size of the blob is proportional to the % weight • The horizontal line is called a confidence interval and is a measure

of how we think the result of this study might vary with the play of chance. • The wider the horizontal line is, the less confident we are of the

observed effect.

The label above the graph tells you what statistic has been used

The data shown in the graph are also given numerically


The pooled analysis is given a diamond shapewhere the widest bit in the middle is located at the calculated best guess (point estimate), and the horizontal width is the confidence interval

Definition of a 95% confidence interval: If a trial was repeated 100 times, then 95 out of those 100 times, the best guess (point estimate) would lie within this interval.


Reading the graphs

What outcome?

What measure of effect size?

The events

The pooled

ORs

And confidence intervals

weighting?


Reading the graphs

What measure of effect size?

And confidence intervals

The pooled

ORs

Heterogeneity

The events


Limitations ???

Takes long time to publish and huge effort to be done

Authors may lose interest in updating SR

Not considered for PhD as main research (reviewer not author ??)

49


Practical tips for new comers

no specific criteria to be an author Determine which field of interest Choose a very specific point formulate a question make a quick search for RCTs preferably not more than 6 RCTs


The life cycle of a Cochrane review Title

(registered)

Prepare Protocol

Register Protocol

Systematic Review

Review online

Time Frame

6 Months

12 Months

Lifelong

editorial + external review

searchesinclusions

qualitydata extraction

meta-analysis

editorial + external review

outside comments and criticismsnew data

2 or more reviewers


What support is available?

an on-line systematic review training

programmes can be accessed at:

http://www.cochrane-net.org/openlearning

http://www.cochrane.org/training/authors

http://www.cochrane-net.org/openlearning




The end.....

54

http://www.cochrane.org/about-us /newcomers-guide.

http://www.cochrane.org/about-us/newcomers-guide

http://www.cochrane.org/about-us/newcomers-guide

Health & Medicine

To Cochrane or not: that's the question