Second Part.MA.Oct202010

Meta-analysis II

Adrian V. Hernandez, M.D., Ph.D.Assistant Professor of Medicine

Quantitative Health Sciences

October 21, 2010

Meta-analysis II l April 10, 2023 l 2

OUTLINE SECOND PART

SECOND PART: 50 MINUTES

• Analysis (models, methods, heterogeneity, publication bias, quality, subgroup analysis)

• Reporting of meta-analysis (PRISMA, MOOSE guidelines)


MODELS

• Fixed effects

-Assumption: Effect is the same across studies and differences due to chance

-Common effect unknown

-Objective: Estimation of common effect with more precision

-Pool studies using weights ↔ sample size


MODELS (2)

• Random effects

-Assumption: Effect is different across studies and there is an average effect

-Average effect unknown

-Objective: Estimation of average effect of studies

-Pool studies using similar weights


When are effects similar between models?

Large effect

Balanced arms

Study sizes similar

Low heterogeneity of effects

HF Fixed MH 1.59 (1.34-1.89)

Random 1.56 (1.32-1.86)

Edema Fixed MH 2.04 (1.85-2.26)

Random 2.41 (1.91-3.04)

Hernandez AV et al. 2010 (submitted)


METHODS TO COMBINE STUDY EFFECTS

• Inverse Variance (IV)

Common, flexible

Binary/continuous data

Log OR, Log RR, log HR, standardized ratios


METHODS TO COMBINE STUDY EFFECTS (2)

• Mantel-Haenzel (MH)

Binary outcomes only

Special cases: sparse outcomes, unbalanced

arms

Correction for zeros in arms

Other situations: Effects similar to IV



• Peto

Binary outcomes only, a few outcomes

Small effect

Balanced arms

No correction for zeros in arms

HF Fixed MH 1.59 (1.34-1.89)

Peto 1.59 (1.34-1.88)






• DerSimonian and Laird

Any type of effect measures

Random model

Larger CI of the pooled effect

More weight to smaller studies


HETEROGENEITY

• Degree of dissimilarity in effects of individual studies

• Why?

Participants

Interventions

Co-interventions

Outcomes

Biases of studies (according to hierarchy), etc.



HETEROGENEITY: Pseudo-tests

• Eyeballing forest plots

• Point estimates

• Significance level

• Confidence intervals


HETEROGENEITY: Test it!

• Cochrane Q test (Х2, p)

AND

• I2: Amount of heterogeneity (0-100%)

It needs 95% CI (<25% Low; 25-50% Mod; >50 High)

(Ioannidis JP. J Eval Clin Pract 2008:14:951-7)

Bad news for both: Low power

Bad news for I2: No software for CIs



HETEROGENEITY: Address it!

• Check data again

• Do not perform a meta-analysis

• Explore vs. ignore

• Use random-effects models

• Change effect measure (e.g. MD →SMD)

• Exclude studies


EXPLORING HETEROGENEITY

• Subgroup analysis

Exploratory only

Low power to detect significant effects

Better pre-specify in protocol

Generates hypotheses

→ Editors and reviewers like subgroup analysis


EXPLORING HETEROGENEITY (2)

• How to perform subgroup analysis?

→ By baseline characteristics (e.g. age, gender)

→ By quality

→ By sample size

→ By follow-up time




Use of thiazolidinediones and risk of heart failure and peripheral edema in patients at high risk of diabetes and

type 2 diabetes:A systematic review and meta-analysis of placebo-

controlled randomized trials

OR (95% CI) RR (95% CI)

Hernandez AV et al. 2010 (Submitted)


EXPLORING HETEROGENEITY (3)

• Meta-regression

→Evaluates factors that explain heterogeneity of

effects

→Bad news: Low power

Lack of data



EVALUATION OF PUBLICATION BIAS

Funnel Plot: Size effect vs. SE/SS ; Asymmetry?

De Luca G et al. Am Heart J 2007; 153:343-53


EVALUATION OF PUBLICATION BIAS (2)

Asymmetrical: Only due to publication bias?

Dentali F et al. Ann Intern Med 2007; 146: 278-88



Pseudo-test: Visual inspection of funnel plot

Test!: Begg-Mazumdar test

Asymmetry regression test

Kendall test, etc

→ Bad news: All have low power



In 95% of MAs, the use of asymmetry regression tests is inappropriate:

→ Highly heterogeneous (I2 > 50%)

→ <10 studies


QUALITY OF STUDIES: Observational

• Design: Prospective cohort

Retrospective cohort

Case-control

• Quality of measurement of factors

• Patient enrollment (consecutive vs no)


QUALITY OF STUDIES: RCTs

• Difficult to define

Design/conduct/analysis?

Clinical relevance?

Reporting?

• Several scales: 39Egger M et al. Systematic reviews in health care. Meta-analysis

in context. 2nd Edition, BMJ London 2001. pp87-108.


QUALITY OF STUDIES: RCTs (2)

TC Chalmers et al. Control Clin Trials 1981; 2: 31-49

30 items, complex

- Internal validity (R, Blinding, Attrition,

stat analysis)

- External validity

- Data presentation/Organizational aspects

→ Low weight to internal validity


QUALITY OF STUDIES: RCTs (3)

AR Jadad et al. Control Clin Trials 1996; 17: 1-12

5 items, 5 points, ≥3 high quality

- Randomization: Description of method? 1

Appropriate? 1

- Double blinding: Description of method? 1

Appropriate? 1

- Description of withdrawal/dropouts? 1

→ More weight to reporting than methodology


REPORTING

• RCTs: PRISMA (Preferred Reporting Items for

Systematic reviews and Meta-Analyses)

• Observational: MOOSE (Meta-analysis Of

Observational Studies in Epidemiology)


PRISMA

Replace and improve the old QUOROM (1999) guidelines

27 items

Title, Abstract, Introduction, Methods, Results, Discussion

and Funding.


PRISMA flow chart


PRISMA guidelines: Improvements

• Clear description of objective (PICOS)

• Improve description of selection of studies (search strategy). Publish at least one.

• Improve evaluation of risk of bias within studies (quality)

• Improve description and evaluation of publication bias.

• Suggest publishing the protocol of the MA


MOOSE

35 items, 1 point to each

Background, Search Strategy, Methods, Results,

Discussion & Conclusion


Documents

Second Part.MA.Oct202010