Introduction to Meta-Analysis Size and Precision... · CAMARADES: Bringing evidence to translational medicine Introduction to Meta-Analysis by Michael Borenstein, Larry V. Hedges

CAMARADES: Bringing evidence to translational medicine

Introduction to Meta-Analysis by Michael Borenstein, Larry V. Hedges , Julian P. T Higgins, and Hannah R. Rothstein

PART 2 Effect Size and Precision

Summary of Chapter 3: Overview Chapter 4: Effect Sizes Based on Means Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables) Chapter 6: Effect Sizes Based on Correlations

Sarah McCann


Chapter 3: Overview

Treatment Effects and Effect Sizes

Terminology:

Effect size – relationship between two variables e.g. difference between males and females

Treatment effect – only appropriate when quantifying the impact of a deliberate intervention e.g. the difference between treated and control groups

Single group summary – estimating the mean or risk or rate of a single population e.g. prevalence of a disease in a particular region or mean scores on a test


Chapter 3: Overview

How to Choose an Effect Size Index

1. The effect sizes from the different studies should be comparable i.e. they measure (at least approximately) the same thing.

2. Estimates of the effect size should be computable from the information in the published reports (should not require re-analysis of raw data, unless they’re known to be available).

3. Need to be able to compute variance and confidence intervals.


Chapter 3: Overview

Summary Data Reported by Primary Studies

Appropriate Effect Size

Means and standard deviations (Chapter 4)

Raw difference in means Standardised difference in means Response ratio

Binary data e.g. events vs. non-events (Chapter 5)

Risk ratio Odds ratio Risk difference

Correlation between two variables (Chapter 6)

Correlation coefficient


Chapter 4: Effect Sizes Based on Means

RAW (UNSTANDARDIZED) MEAN DIFFERENCE D

The outcome is reported on a meaningful scale and all studies in the analysis use the same scale.

Meta-analysis is performed directly on the raw difference in means.



Let µ1 and µ2 be the true (population) means of two groups. The population mean difference is defined as

We can compute an estimate, D, of the population mean using the sample means from studies that use two independent groups, paired groups or matched designs.



Computing D from studies that use independent groups

The variance is calculated using the sample standard deviations (S) and sample sizes (n) of the two groups.



Computing D from studies that use matched groups or pre-post scores

Pairs of participants are matched in some way (e.g. siblings or patients at the same stage of disease) or the same subject acts as their own control (e.g. scores pre-treatment are compared to scores pot-treatment)



STANDARDIZED MEAN DIFFERENCE, d and g

The scale of measurement differs across studies and it is not meaningful to combine raw mean differences.

The mean difference in each study is divided by that study’s standard deviation to create an index that is comparable across studies.



Let µ1 and σ1 and µ2 and σ2 be the true (population) means and standard deviations of two groups. The population mean difference is defined (assuming σ1 = σ2) as:

We can compute an estimate, d, of the population mean using the sample means from studies that use two independent groups, paired groups or matched designs.



Computing d and g from studies that use independent groups

By pooling the two estimates of the standard deviation, we obtain a more accurate estimate of their common value.



d has a slight bias, tending to overestimate the absolute value of δ in small samples.

This bias can be removed with a simple correction, resulting in an unbiased estimate sometimes called Hedges’ g.

To convert from d to Hedges’ g, a correction factor called J is used:

And then:



Computing d and g from studies that use pre-post scores or matched groups



Including different study designs in the same analysis

A systematic review may include some studies that use independent groups and others that used matched groups.

The effect size from each study can be computed using the appropriate formula and all the studies can be combined in the same analysis.

Direction of effect is arbitrary, the researcher must decide on a convention and apply it to all studies, regardless of design.



RESPONSE RATIOS

When the outcome is measured on a physical scale (e.g. length, area, mass) and is unlikely to be zero, the ratio of the two means can be used as the effect size.

Computations are carried out on a log scale.





The response ratio is computed as:

The log response ratio is computed as:



SUMMARY POINTS

The raw mean difference (D) may be used as the effect size when the outcome scale is either inherently meaningful or well known due to widespread use. This effect size can only be used when all studies in the analysis used precisely the same scale.

The standardized mean difference (d or g) transforms all effect sizes to a common metric, and thus enables us to include different outcome measures in the same synthesis.



SUMMARY POINTS

The response ratio (R) is often used in ecology. This effect size is only meaningful when the outcome has a natural zero point, but when this condition holds, it provides a unique perspective on the effect size.

It is possible to compute an effect size and variance from studies that used two independent groups, from studies that used matched groups (or pre-post designs) and from studies that used clustered groups. These effect sizes may then be included in the same meta-analysis.


Chapter 5: Effect Sizes Based on Binary Data (2 x 2 Tables)



RISK RATIO

The ratio of two risks.

Risk of death in treated group is 5/100 and the risk of death in the control group is 10/100, so the ratio of the two risks is 0.50.





ODDS RATIO

The ratio of two odds.

Odds of death in the treated group would be 5/95 or 0.0526 and the odds of death in the control group would be 10/90 or 0.1111, so the ratio of the two odds would be 0.0526/0.1111 or 0.4737.



Why use log transformations?

The log transformation is needed to maintain symmetry in the analysis.

Assume that one study reports that the risk is twice as high in Group A while another reports that it is twice as high in Group B. Assuming equal weights, these studies should balance each other, with a combined effect showing equal risks (a risk ratio of 1.0).

However, on the ratio scale these correspond to risk ratios of 0.50 and 2.00, which would yield a mean of 1.25. By working with log values we can avoid this problem. In log units the two estimates are +0.693 and -0.693, which yield a mean of 0.00.

We convert this back to a risk ratio of 1.00, which is the correct value for this data.



RISK DIFFERENCE

The difference between two risks.

Risk in the treated group is 0.05 and the risk in the control group is 0.10, so the risk difference is -0.05.

No log transformation.



SUMMARY POINTS

We can compute the risk of an event (such as the risk of death) in each group (for example, treated versus control). The ratio of these risks then serves as an effect size (the risk ratio). The difference in these risks can also serve as an effect size (the risk difference).

We can compute the odds of an event (such as ratio of dying to living) in each group (for example, treated versus control). The ratio of these odds then serves as the odds ratio.



SUMMARY POINTS

To work with the risk ratio or odds ratio we transform all values to log values, perform the analyses, and then convert the results back to ratio values for presentation. To work with the risk difference we work with the raw values.

When choosing an effect size index, consider: The risk and odds ratios are relative measures – relatively insensitive to differences in baseline events. The risk difference is an absolute measure – very sensitive to the baseline risk.


Chapter 6: Effect Sizes Based on Correlations

Computing r

Studies report a correlation between two continuous variables.

The correlation coefficient can serve as the effect size index.

Standardised to take into account the different metrics in original scales.


Chapter 6: Effect Sizes Based on Correlations

Documents

Introduction to Meta-Analysis Size and Precision... · CAMARADES: Bringing evidence to translational medicine Introduction to Meta-Analysis by Michael Borenstein, Larry V. Hedges