Systematic Review & Meta-Analysis

系统综述和meta 分析

Xu Xiong, MD, DrPHSchool of Public Health and Tropical Medicine

Tulane University

Narrative Reviews, Systematic Reviews, and Meta-Analysis

• Narrative Review: traditional expert review

Subjective, no formal rules in selecting studies, no standard statistical methods for combining studies

• Systematic Review: review in which there is a comprehensive search for relevant studies on a specific topic, and those identified are then appraised and synthesized according to a predetermined and explicit method.

• Meta-Analysis: systematic review that employs statistical methods (a quantitative summary) to combine and summarize the results of several studies.

Definition: Meta-Analysis• Coined by Glass in 1976 from the Greek prefix

“meta” meaning “after,” “more comprehensive,” or “transcending” and the root, “analysis”

• The statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings (Glass, 1976)

• A statistical synthesis of the data from separate but similar (i.e., comparable) studies, leading to a quantitative summary of the pooled results (Last, 2001)

• Synonyms: Research synthesis, Pooled analysis, Quantitative review, Overview

Number of Papers Referencing Meta-Analysis, 1985-2011

0

1000

2000

3000

4000

5000

6000

7000

Results from MEDLINE search using MeSH or text word “meta-analysis”

Historical Note• 1904 – Karl Pearson derived formulas to combine

correlations from different samples• 1932 – R.A. Fisher developed a method to combine p-

values from different studies• 1976 – Gene Glass coined the term meta-analysis• 1977 – Smith & Glass published the first meta-analysis

paper cited in MEDLINE Meta-analysis of psychotherapy outcome studies (Am

Psychol 1977;32:752-760)

• 1989 – Meta-analysis was adopted by MEDLINE as a subject heading

• 1993 – Meta-analysis was adopted by MEDLINE as a publication type

Strength of Evidence Concerning Efficacy of Treatment

Meta-Analysis

• Meta-analysis differs from: Primary analysis: the original analysis of

data from a research study Secondary analysis: the re-analysis of data

to answer new research questions

• Meta-analysis methods focus on: Contrasting and comparing results from

different studies in anticipation of identifying consistent patterns and sources of disagreements among the results

Why Use Meta-analysis

• To provide a more objective appraisal of the evidence

• To reduce the probability of false negative results

• To test treatment effects in subgroups of patients

• To explore and explain heterogeneity between study results

• To generate research questions to be addressed in future studies

Meta-Analysis Objectives

• Synthetic goal (estimation of summary)• Analytic goal (estimation of differences)

When to Use Meta-Analysis

• When individual trials or studies are too small to give reliable answers

• When large trials or studies are impractical or impossible

• When there have been many trials or studies showing small effects are important

• When trial or study results are inconclusive or conflicting

Potential Limitations of Meta-Analysis

• Problems associated with design or reporting original studies

• Publication bias• Limitations of using published data• Retrospective research• Variation of standard treatments over time• Heterogeneity of studies• Statistical methods

Steps of Meta-Analysis1. Formulate research question

2. Develop a proposal

3. Comprehensive literature search

4. Selection of study

5. Critical appraisal of study

6. Extraction of data

7. Synthesis of data

8. Sensitivity and subgroup analyses if appropriate and possible

9. Preparing a structured report

Formulating a Research Question

• What are the study objectives? To validate results in a large population To guide new studies

• What are the operational definitions? Disease or condition of interest Population and setting Treatment, intervention or exposures (e.g. risk factor,

medication, diagnostic test) Outcomes of interest (both beneficial and harmful)

• What types of study designs? Randomized controlled trials: e.g., Cochrane Review Observational studies

Literature Scoping• Useful for formulating the research question• Preliminary assessment of potentially relevant

literature Search for existing reviews and primary studies

relevant to the topic

• Usually only undertaken in a small range of databases relevant to the topic

Cochrane Controlled Trials Register (CCTR) MEDLINE and EMBASE for medical topics PsycLIT for reviews of psychological and psychiatric

topics

Inclusion Criteria

• Study Design

• Population

• Interventions

• Comparisons

• Outcomes

Practical Considerations in Defining Eligibility for a Meta-Analysis

• Study designs to be included• Years of publication or study conduct• Languages• Choice among multiple publications• Restrictions due to sample size or follow-up

duration• Similarity of treatment and/or exposure• Completeness of information

Comprehensive Data Search

• Need a well formulated and coordinated effort

• Seek guidance from a librarian• Specify language constraints• Requirements for comprehensiveness of the

search depends on the field and question to be addressed

Tulane Medical Library

Literature Search• Computerized bibliographic database (MEDLINE)• Bibliography searches• Current contents• Dissertations• Textbooks• Databases of unpublished work• Citation searches• Expert survey• Meeting proceedings and abstracts• Granting agencies• Trial registries• Industry• Journal hand-searching

Literature Search Challenges

Database Bias - “No single database is likely to contain all published studies on

a given subject.”

Publication Bias - selective publication of articles that show positive treatment

of effects and statistical significance. It is important to search for unpublished

studies through a manual search of conference proceedings, correspondence

with experts, and a search of clinical trials registries.

English-language bias - occurs when reviewers exclude papers published in

languages other than English

Citation bias - occurs when studies with significant or positive results are

referenced in other publications, compared with studies with inconclusive or

negative findings

Unbiased Selection and Extraction Process

• Study Selection Two independent reviewers select studies Based on a priori specification of the

population, intervention, outcomes and study design

Level of agreement: kappa Differences are resolved by consensus Specify reasons for rejecting studies

Data Extraction• Two independent reviewers extract data using pre-

established forms• Should be explicit, unbiased, and reproducible• Include all relevant measures of benefit and harm of the

intervention• Contact investigators of the studies for clarification in

published methods, data• Extract individual patient data when published data do not

answer questions about: intention to treat analyses, time-to-event analyses, subgroups, dose-response relationships

• Methodological quality• Level of agreement: kappa• Differences in data extraction are resolved by consensus

Data Extraction: Study Characteristics

• Types of publication (journal article, abstract or unpublished data)

• Publication year and country of origin• Study participants (sample size, age, gender, race,

health status)• Design details (case-control, cohort, parallel or

cross-over, randomization, blinding)• Nature of treatment and control• Study duration• Measurement of compliance• Definition and measurement of outcome• Other confounders

Data Extraction: Study Outcome

• Continuous variables Mean difference between treatment and

control groups

• Binary variables Odds ratios Relative risks Hazard ratios Absolute risk reduction or number of

patients needed to be treated to prevent one event

Critical Appraisal of Data

• Description of Studies Size of study Characteristics of study participants Details of specific interventions used Details of outcomes assessed

Study Quality Assessment

• Choose a method of assessment of quality of original studies, for example, Chalmers RCT Quality Score (Controlled Clinical Trials 1981;2:31-49)

• Assess quality of each study in uniform, systematic and complete manner

• Identify acceptable studies and give score to their quality• Keep a list of unacceptable studies• Consider weighting each study result by quality score, or

stratifying by quality

Synthesis of Data• Graphic displays

Flow diagram Forest plot

• Pooling data Fixed-effects model Random-effects model

• Test for heterogeneity Subgroup analysis Meta-regression

Active Management of 3rd Stage: Cochrane Review/Meta-Analysis

Cochrane Reviewhttp://www.cochrane.org/

A Forest Plot

Data Analysis• Include all relevant and clinically useful measures of

treatment effect• Perform a narrative, qualitative summary when data are

too sparse, or too low quality or too heterogeneous to proceed with a meta-analysis

• Specify if fixed or random effects model is used• Describe proportion of participants used in final analysis• Use confidence intervals• Include a power analysis• Consider cumulative meta-analysis (by order of

publication date, baseline risk, study quality) to assess the contribution of successive studies

Software for Meta-analysis: e.g., WinPEPI

Subgroup Analyses• Pre-specify hypothesis-testing subgroup analyses and

keep few in number• Label all posteriori subgroup analyses• When subgroup differences are detected, interpret in light

of whether they were: Established a priori Few in number Supported by plausible mechanisms Important (qualitative vs. quantitative) Consistent across studies Statistically significant (adjusted for multiple testing)

Sensitivity Analysis

• Test robustness of results relative to key features of the studies and key assumptions and decisions

• Include tests of bias due to retrospective nature (e.g., with/without studies of lower methodological quality)

• “A funnel plot is used as a way to assess publication bias in meta-analysis.”

Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD. “Clinical Perspective: A Practical Guide to Meta-Analysis.” The Journal of Hand Surgery. Vol.31A No.10 December 2006. p. 1676

Publication bias: A Funnel Plot

Prepare a Structured Report

• Include a structured abstract• Include a table of the key elements of each study• Include a flow diagram detailing the study

selection process• Include summary data from which the measures

are computed• Employ formative graphic displays representing

confidence intervals, group event rates, sample sizes

Interpretation of Findings• Interpret results in context of current health care• State methodological limitations of the individual

studies included and in the meta-analysis• Consider size of effect in studies and meta-

analysis, consistency of effect sizes and any dose-response relationship

• Consider interpreting results in context of temporal cumulative meta-analysis

• Interpret results in light of other valuable evidence• Make recommendations clear and practical• Propose future research agenda (clinical and

methodological requirements)

Summary• A well conducted meta-analysis allows for a more

objective appraisal of the evidence than traditional narrative reviews

• Meta-analysis may resolve uncertainties and disagreements in original research

• Meta-analysis may enhance the precision of estimates of treatment effects

• Exploratory analyses (i.e., subgroups who are likely to respond well to a treatment) may guide cost effective treatment decisions

• Meta-analyses may demonstrate areas where the evidence is inadequate and thus identity areas where further research is needed