Upload
kemalmia
View
8
Download
1
Embed Size (px)
DESCRIPTION
evidence based medicine
Citation preview
Systematic Reviews and Meta-Analysis
Kuntjoro Harimurti
Center for Clinical Epidemiology and Evidence-Based Medicine (CE-EBM)Faculty of Medicine UI/Cipto Mangunkusumo Hospital
Jakarta
Reviews on medical literatures
• Overview (review article): • unsystematic
• Systematic review: • in gathering, evaluating, presenting evidence• no formal statistical methods
• Meta-analysis: • systematic review plus formal statistical analysis
Review articleSystematic review
Meta analysis
Integrative Literature
Systematic vs. Unsystematic Reviews
Unsystematic reviews• Not focused on one
interested topic broader• Mechanisms of disease• No criteria for selection of
articles to be reviewed bias
• No assessment of selected articles
• Low applicability in clinical practice
Systematic reviews• Focused on only one or two
topics of interest• Diagnosis, etiology, prognosis,
and treatment of disease• Explicit methods and criteria
for selection (database, keywords, types of articles)
• Assessment of selected articles
• Highly applicable
Narrative/traditional reviews(unsystematic)
Narrative/traditional reviews(unsystematic)
Systematic reviews
Systematic review/Meta-analysis
Why Systematic Reviews?• Information overload
• Not all studies in journals are good in quality (valid/unbiased)
• Result from many studies not conclusive (controversies exist)
Systematically search
Systematically assessed the quality of included studies
Systematically combined to arrive at conclusion
Medical evidence increasing at epidemic rates: we all need EBP skills to keep up-to-date
MEDLINE 20102,000 articles / day
approx 75 new trials published
every day
Bastian, Glasziou, Chalmers (2010) 75 Trials and 11Systematic Reviews a Day: How Will We Ever Keep Up? PLoS Med 7(9)
The research-to-awareness gap
0
500000
1000000
1500000
2000000
2500000
Trials MEDLINE BioMedical
Med
ical
Art
icle
s pe
r Yea
r
5,000?per day
1,500 per day55 per
day
Streptokinase for Myocardial Infarction
Prophylactic antiarrhythmic drug therapy in acute myocardial infarction
Teo et al (JAMA, 1993)• A systematic review of 51 RCTs of anti-
arrhythmic drugs in heart attack involving 23,229 patients
• 660 deaths in patients allocated drugs (lidocaine)
• 571 deaths in patients allocated to control• 89 deaths attributable to drugs
Deadly medicine: why tens of thousands of heart patients died
in America's worst drug disaster • At the peak of their use in the late 1980s, it has been
estimated that these drugs killed asm many Americans every year as were killed during the whole of Vietnam war.
• The vast majority of the victims of these drugs were treated outside controlled trials
Thomas J Moore, 1995
Steps on Systematic Reviews
• Formulating answerable question (PICO)• Extensive and comprehensive searching the evidence
– 2 or more major databases + handsearching– Specific keywords– Selection criteria (inclusion and exclusion): types of
(primary) articles, year of publication, language• Critically appraise the evidence quality assessment
– 2 or more assessors• Synthesis and analysis meta-analysis• Discuss and conclude the results
Components of SR/MA
• Introduction
• Methods
• Results
• Discussions and Conclusions
Introduction
• State the reasons why the review is needed– Based on problems in clinical setting (high volume,
high risk, high cost)• Any controversies in the literatures? In
treatment, choice of diagnostic modalities, determination of prognosis
• Ended with objective of SR based on clinical question (PICO)
Methods
• How the individual studies searched and selected
• How to appraise/assess the individual studies
• How to combine (synthesis)
Searching the literatures
• A comprehensive and reproducible literature search is the foundation of a systematic review
• Search for published studies at least in MEDLINE, EMBASE, and CINAHL.
• Search for unpublished clinical trials in the Cochrane Central Register of Controlled Trials (CENTRAL) or other clinical trial database (www.clinicaltrial.gov)
• Secondary/hand searching• No limitation on years and languages
Literature Search Challenges
• Database Bias - “No single database is likely to contain all published studies on a given subject.”
• Publication Bias - selective publication of articles that show positive treatment of effects and statistical significance.– Hence, it is important to search for unpublished
studies through a manual search of conference proceedings, correspondence with experts, and a search of clinical trials registries.
Literature Review Challenges (cont.)
• English-language bias - occurs when reviewers exclude papers published in languages other than English
• Citation bias - occurs when studies with significant or positive results are referenced in other publications, compared with studies with inconclusive or negative findings
Quality Assessment
• The validity of a systematic review ultimately depends on the scientific method of the retrieved studies and the reporting of data
• In systematic review to assess treatment effect, RCTs are considered to be more rigorous than observational studies– A review based on well-designed RCT will likely be more
valid and accurate than a review based on observational studies or case reports
Quality Assessment (cont.)
• Quality assessment should be performed by at least two assessors
• The most common way to assess and report study quality has been using a composite, numerical scoring instrument (Exp. Jadad score)
Jadad score• Randomization (2 points possible)
– 1 point if study described as randomized– Add 1 point if randomization method described and appropriate
(e.g. random numbers generated) – Deduct 1 point if randomization described and inappropriate
• Double-blinding (2 points possible)– 1 point if study described as double-blinded– Add 1 point if method of double-blinding described and
appropriate – Deduct 1 point if double-blinding described and inappropriate
• Withdrawals (1 point possible)– Give 1 point for a description of withdrawals and drop-outs
Results
• Result of search strategies (flow diagram)
• Result of quality assessment
• Synthesis (summarized the effects)
Flow diagramPotentially relevant RCTs (n= ..)
n RCTs excluded with reason
RCTs retrieved for more detailed information (n= ..)
n RCTs excluded with reason
Potentially appropriate RCTs (n= ..)
RCTs in meta-analysis (n=..)
n RCTs excluded with reason
Quality Assessment
Summarized the effect
• Forest plot:– Effect size with CI of each study represented by
weightened square – Combined effect (and CI) represented by diamond – Vertical line (line of no effect) “1” for OR/RR, “0” for
mean difference– Vertical dotted line crossing combined effect to assess
heterogeneity• Test for heterogeneity
– Cochrane Chi2 (p value), I2 (degree of heterogeneity) – Random effect model vs. Fixed effect model
Study I 1992Study II 1994Study III 1995Study IV 1995Study V 1996Study VI 1997Study VII1 1999Study VIII 2000Combined
0.1 10
OR = 1Favor drug Favor placebo
Meta-analysis of RCTs with nominal outcome
Study I 1992Study II 1994Study III 1995Study IV 1995Study V 1996Study VI 1997Study VII1999Study VIII 2000
Combined
-0.3 +0.3Xe-xc=0
Favor drug Favor placebo
Meta-analysis of RCTs, numerical outcome
Result of summarized effects
Reporting publication bias
• A funnel plot is used as a way to assess publication bias in meta-analysis
CRITICAL APPRAISAL OFSYSTEMATIC REVIEWS/META-ANALYSIS
Is the review any good?Q-FAST appraisal
• Question – Does the systematic review addres focused question (PICO)?
• Finding – Did the search find all relevant evidence?
• Appraisal – Have been the studies critically appraised?
• Synthesis – Have the results been synthesised with appropriate summary tables and plots?
• Transferability of results
Thank You