16/02/53

1

Systematic Review &

Meta analysisMeta-analysisAmmarin Thakkinstian, Ph.D.

Section for Clinical Epidemiology and BiostatisticsFaculty of Medicine, Ramathibodi Hospital

Tel: 02-201-1269, 02-201-1762 Fax: 02-2011284

e-mail: raatk@mahidol.ac.th

Course outline

• Lecture (20%)– Review methods– Meta-analysis

• Class discussion & tutorial (30%)– Review topics

• Assignments (50%)• Assignments (50%)

16/02/53

2

Topics Score Due DateAssignment I: - Initiating review topic- Literature review

o Background & rationale , o Formulate a review question & objectives

15% Feb 26, 2009

Assignment II: Locate and select studies- Define inclusion & exclusion criteria 12.5% Mar 6 , 2009- Search terms and strategies - Identifying studies- Selecting studiesAssignment III: - Design

o Data extraction formso Quality or risk of bias assessment

- Perform data extraction & assess quality of studies

12.5% Mar 13 , 2009

Assignment IV: Assignment IV: - Data analysis plan - Dummy tables & figures

15% Mar 20, 2009

Assignment V: Data analysis 20% April 17, 2009Assignment VI: Writing up paper (as a manuscript)- Presentation (verbal)- Submit paper

5%25%

May 7, 2009May 8, 2009

Systematic review

• Review methodology• Review methodology• Pooling methods: meta-analysis

– Dichotomous outcome– Continuous outcome

16/02/53

3

Systematic review

A review that has been prepared• A review that has been prepared using a systematic approach to minimise biases and random error

Rationale

• Tool for health care worker• Tool for health care worker, researchers, consumers, and also policy maker who want to keep up with the evidence that is accumulating in their field

16/02/53

4

Rationale• More objective appraisal of the

evidence than traditional narrativeevidence than traditional narrative reviews- Narrative review: subjective, selection

bias, limitation of single or few studies, unhelpful descriptions, e.g., no clear evidence, a weak relationship, a strongevidence, a weak relationship, a strong relationship.

- Systematic review: more transparent appraisal, allow reader to replicate, quantitative conclusion.

• Meta-analysis:

Rationale

– Estimates treatment effects– Leading to reduces probability of false

negative results (increase power of test)– Potentially to a more timely introduction

of effective treatments.of effective treatments.

16/02/53

5

Rationale• Exploratory analyses:

– subgroups of patients who are likely to respond particularly well to a treatment (or the reverse)

– may generate promising new research questions to be addressed in future studies.

• Systematic review may demonstrate th l k f d t id dthe lack of adequate evidence and thus identify area where further studies are needed

Terminology

• Systematic reviewSystematic review• Overview• Meta-analysis• Research synthesis• Summarizing• Summarizing• Pooling

16/02/53

6

Review Methodology• Good Rationale• Clearly state research question• Clearly state research question• Objective• Identify relevant studies/locate studies• Explicitly describe inclusion & exclusion

criteria of studies • Data extraction: study factor outcome• Data extraction: study factor, outcome• Data analysis• Results• Discussion

Review proposal• Introduction & background• Research question/objective

R i th d• Review methods– Locating studies– Selecting studies

• Inclusion/exclusion criteria – Data extraction – Quality/risk assessment – Data analysis

• Results• Discussion

16/02/53

7

Rationale • Why do we need to perform the review• How were results of previous individual p

and review study (if any) – Positive results – Negative results

• Methodologic issues – Sample size/Power of test p– Narrative reviews?

• Selective bias • Pooling effect sizes?

– Quality of the studies

16/02/53

8

Good research question • Evidence-base Medicine (EBM)

– Patient– Intervention– Comparator– Outcome– PICO

16/02/53

9

Research question• Is there association between VDR and

BMD/osteoporosis in women?BMD/osteoporosis in women? – To compare rates of osteoporosis in women

with BB with bb genotypes • Is Rosiglitazone higher risk of developing

myocardial infarction compare with Metformin indiabetic patients?diabetic patients? – To compare incidence of MI between diabetic

patients received Rosiglitazone and Metformin

Locate studies1. Defines source of database

– MEDLINE– MEDLINE- 1949to present• Over 16 million references • Since 2005, between 2,000-4,000 completed references are

added each day Tuesday through Saturday• Cover 5200 worldwide journals in 40 languages- Uses medical subject heading (MeSH) for index

Includes biomedicine and health science journals- Includes biomedicine and health science journals- English abstracts for 79% on references- 90% are English language articles- 47% of journals covered are published in the US

- PubMed available free of charge

From http://www.nlm.nih.gov/pubs/factsheets/medline.html

16/02/53

10

Defines source of databasesEMBASE- Over 12 million records from 1974-present- Over 12 million records from 1974-present - More than 600,000 records added annually- Covers over 4,800 active peer-reviewed journals

published in 70 countries/ 30 languages - uses EMTREE for indexing

includes English abstracts for 80% of references- includes English abstracts for 80% of references- daily update, within two weeks of receipt of the

original journal- Produced by Elsevier, no free version available

Defines source of database

• The Cochrane Controlled Trials Register (CCTR)(CCTR)

• ClinicalTrials.gov • HUGE NET Review• Reference lists

H d hi f k j l• Hand searching of key journals• Personal communication with expert in the

field

16/02/53

11

Locate studies

2 Define the software & version used2. Define the software & version used for searching

- PubMed- OVID version 1.0.2- Silver Platter

National Center for Biotechnology Information

16/02/53

12

National Center for Biotechnology Information

16/02/53

13

16/02/53

14

3. Defines searching terms• Specify period of searching• Specify period of searching• Plan for update searching • Combinations of search terms

• Intervention: treatment/study factor• Outcome of interest • Comparator* • Patient

16/02/53

15

Example• VDR& BMD/Osteoporosis(J Bone Miner Res.

2004;19(3):419-28.)

1. vitamin D receptor or VDR (MeSH)2. genotype(s) or allele(s) or polymorphism(s) (MeSH)3. bone mineral density or BMD or bone density (MeSH)4. low bone mineral density or low density (textword)5. osteoporosis (MeSH)6. fracture (MeSH)6. fracture (MeSH)7. 1 and 2 and 38. 1 and 2 and 49. 1 and 2 and 510. 1 and 2 and 6

Selecting studies • Clearly define inclusion & exclusion criteria• Inclusion criteriaInclusion criteria

– Type of subjects• Treatment or exposure or gene• Comparator (if needed) • Outcome– Study design

• randomized controlled trial, • observational studies (cohort, case-control,

cross-sectional study)– Full paper

•

16/02/53

16

– Languages• English, French, others

– Restrictions due to sample size or other criteria

– Multiple publications of the same studies, choose the recent one or the

h id d l t fone has provided more completeness of data

Exclusion

– Incompleteness of information• Contact authors at least two times for

incomplete data– Not the outcome of interest – Review studies

Narrati e re ie• Narrative review • Systematic review

– Not comparative studies • No control group

16/02/53

17

Selecting studies

• Merge studies identified from databases i f ( E d t )using reference manager (e.g. Endnote)

– Remove duplicates • Two reviewers independently select

studies – Screen title/abstract to remove non-relevantScreen title/abstract to remove non relevant

studies base on eligibility criteria – Access full papers

– Examine other sources of studies C t t th if d d– Contact author if needed

– Final decision• Perform searching every 3 months

while doing a review

16/02/53

18

Thakkinstian et al. Meta-analysis of molecular association studies: vitamin D receptor gene polymorphisms and BMD as a case study J Bone Miner ResBMD as a case study. J Bone Miner Res 2004;19(3):419-28.

Any observational study (cohort, case-control, and cross-sectional study), regardless of sample size, which determined the difference in mean BMD di t th VDR t t t dBMD according to the VDR genotype, or tested the association between osteoporosis or fractureand the VDR genotypes and which fulfilled the following criteria:

16/02/53

19

VDR & BMD/Osteoporosis- The BMD measurements were performed at

lumbar spine or hip by Dual Energy X-raylumbar spine or hip by Dual Energy X-ray Absorptiometry (DEXA) or Dual Photon Absorptiometry (DPA) method.

- sufficient details: mean and SD of BMD, and number of subjects for each VDR genotype, and frequencies of genotype among case and control group were reported for dichotomous outcomes

VDR & BMD/Osteoporosis

• Participants were adult womenParticipants were adult women who were either pre- or post menopausal.

• Study factor & outcome– Bsm/Apa/Taq/Fok– Bsm/Apa/Taq/Fok– BMD/Osteoporosis

16/02/53

20

Data extraction (DE)• At least two reviewers• Design DEF, pilot, & revise DEF

Th ti l– The article• Study ID, • Author, • Year & source of publication

– The study characteristics• Type of studies subjects

th i it– ethnicity, – Adults vs children– Postmenopause, premenopause

• study design (RCT, CS, CC, CrS)• Methods used/criteria for measuring outcomes

DE (cont.)– Patients

• Demographic and clinical features of study's participants that might affect outcomes

–mean age, gender, BMI, smoking, underlying diseases

T bl f t d f t /i t ti– Table of study factors/interventions versus outcomes

16/02/53

21

DE (cont.)• Dichotomous outcome

– Frequencies between study factor/intervention vs outcomefactor/intervention vs outcome

GroupDisease

Yes No N I

Rx (Exp+) a b n1 a/n1Rx (Exp+) a b n1 a/n1

Placebo (Exp-) c d n2 c/n2

- OR (95% CI), RR (95% CI), HR (95% CI)

DE (cont.)

• Continuous outcome n mean (95% CI) G SD– n, mean (95% CI) Group n mean SD

A n1 mean1 SD1

B n2 mean2 SD2

16/02/53

22

Risk of bias in individual studies

• Quality Assessment (QA) • Consider internal & external validity

Risk of bias (cont.)• RCT

TC Chalmers From Egger M Smith GD– TC Chalmers From Egger M, Smith GD, Altman DG. Systematic reviews in health care: Meta-analysis in context. London: BMJ Books, 2001.

• Jadad’s quality assessment scale– The Cochrane Collaboration’s tool for

assessing risk of bias 2009 – Preferred reports of items for systematic

review and meta-analysis-PRISMA guideline

16/02/53

23

Domain Description Review authors’ judgement

Sequence generation. Describe the method used to generate the allocation sequence in sufficient detail to allow an

f h h i

Was the allocation sequence adequately generated?

assessment of whether it should produce comparable groups.

Allocation concealment. Describe the method used to conceal the allocation sequence in sufficient detail to determine whether

Was allocation adequately concealed?

detail to determine whether intervention allocations could have been foreseen in advance of, or during, enrolment.

Blinding of participants, personnel and outcome assessors Assessments should be made for each main outcome (or class of outcomes).

Describe all measures used, if any, to blind study participants and personnel from knowledge of which intervention a participant received. Provide any information relating to

Was knowledge of the allocated intervention adequately prevented during the study?

whether the intended blinding was effective.

Incomplete outcome data Assessments should be made for each main outcome (or class of outcomes).

Describe the completeness of outcome data for each main outcome, including attrition and exclusions from the analysis. State whether attrition and exclusions were reported, the

Were incomplete outcome data adequately addressed?

numbers in each intervention group (compared with total randomized participants), reasons for attrition/exclusions where reported, and any re-inclusions in analyses performed by the review authors.

16/02/53

24

Selective outcome reporting. State how the possibility of selective outcome reporting was examined by the review authors, and what was found.

Are reports of the study free of suggestion of selective outcome reporting?

Other sources of bias. State any important concerns Was the study f fabout bias not addressed in

the other domains in the tool.

If particular questions/entries were pre-specified in the review’s protocol, responses should be provided for each question/entry.

Trial methodology

apparently free of other problems that could put it at a high risk of bias?

Premature trial termination

Post-randomization exclusion

Unbalance baselineStatistical analysis

Unbalance baseline characteristics

Adequately describe methods of data analysis

-use per-protocol analysis, modified ITT

• Observational studies– Thakkinstian A D'Este C Eisman JThakkinstian A, D Este C, Eisman J,

Nguyen T, Attia J. Meta-analysis of molecular association studies: vitamin D receptor gene polymorphisms and BMD as a case study. J Bone Miner Res 2004;19:419-28.;

16/02/53

25

Criteria ScoreA. Representativeness of cases - Consecutive/randomly selected from

cases population with clearly defined random frame

2

1- Consecutive/randomly selected from cases population without clearly defined random frame or with extensively inclusion criteria

- Not describe method of selection

1

0

B. Representativeness of controls- Consecutive/randomly drawn from area

( d/ i ) i h h2

(ward/community) as cases with the same criteria

- Consecutive/randomly drawn from different area as cases

- Not describe

1

0

C. Ascertainment of osteoporosis/fracture- Clearly described objective criteria of

diagnosis of osteoporosis/osteoporotic fracture with proving diagnosis e g measure

2

Criteria Score

fracture with proving diagnosis, e.g. measure BMD using DEXA, X-ray for fracture

- Diagnosis of osteoporosis/osteoporotic fracture by patients history

- Not describe

1

0

D. Ascertainment of control- Controls were proved that they were not

osteoporosis/osteoporotic fracture, i.e. 2

p p ,measured BMD

- Only mentioned that controls were subjects who were not osteoporosis/osteoporotic fracture without proving

- Not describe

1

0

16/02/53

26

E. Ascertainment of VDR examination- Genotyping done under “blind” condition- Unblind or not mention

10

F. Data analysisHWE

Criteria Score

HWE:- Checking goodness of fit in control group

with appropriate statistics - Checking goodness of fit of HWE in case &

control group, or using inappropriate statistics

- Not mentioned

2

1

0

G Association assessment:G. Association assessment: - Appropriate statistic used with adjusting for

confounders e.g. logistic regression, or matched case-control design

- Appropriate statistic used without adjusting - Inappropriate statistic

2

10

H. Response rate- Response rates for both group are the 2

Criteria Score

Response rates for both group are the same or different between groups ≤ 5%

- Response rates are different between 5% - 10%

- Response rates are different 10% or more, or not mention about response rates

2

1

0

Total 15

16/02/53

27

Net work

• Cochrane collaboration – RCT– Diagnostic studies

16/02/53

28

16/02/53

29

16/02/53

30

Statistical analysis

• Unit of analysis is study NOT subject• Describe how data will be pooled

– OR, RR, RD, HR– Mean

• Heterogeneity Test– Test

– Degree (I2)

• Explore source(s) of heterogeneity– Meta-regression– Sub-group analysis

• Publication bias – Graph

Test– Test • Software • P value

16/02/53

31

Dummy tables

• Table 1. study’s characteristics • Table 2. cross-tabulate table

– Treatment vs primary outcome– Treatment effects & 95% CI– Pooled treatment effects

• Table 3 cross tabulate table of subgroup• Table 3. cross-tabulate table of subgroup• Table 4. cross-tabulate table of 2nd

outcome

Dummy figures

• Figure 1. Flow chart of selection of studies• Figure 2. Forest plot of 1st outcome• Figure 3. Forest plot of 2nd outcome