Upload
julia-logan
View
218
Download
0
Tags:
Embed Size (px)
Citation preview
Schmidt & Hunter Approach to rBare Bones
Statistical Artifacts
Extraneous factors that influence observed effectSampling error*
Reliability
Range restriction
Computational error
Dichotomization of variables
*addressed in the (bare-bones) analysis
Bare Bones r
Find weighted mean and variance:
Note sample size weight.
Note that for unit weights, the weighted variance estimator is the sample, not population, estimate.
i
ii
N
rNr
i
iir N
rrNs
22 )(
Confidence Interval for Mean
i
iir N
rrNs
22 )(
k
srCI r
2
96.1%95
i
ii
N
rNr
There are k studies, with Ni observations.
This is not the only formula they use, but it’s the best one IMHO.
Estimated Sampling Error Variance
The variance of r
Nr
222 )1(
1
)ˆ1(ˆ
222
N
iei
1
)1(ˆ
222
N
re
Estimated variance for a study.
Estimated sampling variance for a meta-analysis. Note mean r is constant. This is the variance of sampling error we expect if all the studies have a common effect estimated by r-bar.
Variance of Rho
ETX 222ETX
er 222er
222er
Classical Test Theory
Sampling Error
A definition
Estimated Variance of rho
222 ˆˆ ers
2ˆ
i
ii
N
rrN 2)(
1
)1( 22
N
r-
To find the variance of infinite-sample correlations, find the variance of r in the meta-analysis and subtract expected sampling error variance. Schmidt would be quick to add that part of the estimated variance of infinite-sample correlations is artifactual.
Note that the variance of rho will be called tau-squared by Hedges
Credibility Interval
96.1%95 rCRThe credibility interval and the confidence interval are quite different things. The CI is a standard statistical estimate (intended to contain rho, or average of rho). The CR is intended to contain a percentage of the values of a random variable – infinite-sample effect sizes. The S&H value forgets that there is also uncertainty in the mean value; the two should be added. There are Bayesian programs that will do this; there is also an approximation called the prediction interval described in Borenstein et al.
Bare-Bones Example (1)Study Ni r
1 200 .20
2 100 .20
3 150 .40
4 80 .40
Mean 132.5 .30 <- Unit weighted mean
Bare-Bones Example (2)
r Ni rNi
.20 200 40
.20 100 20
.40 150 60
.40 80 32
sum 530 152
BB Example (3)
2868.530
152
i
ii
N
rNr
Recall unwighted or unit weighted mean = .30.Why are they different?
BB Example (4)r Ni
.20 200 1.507
.20 100 .753
.40 150 1.922
.40 80 1.025
Sum 530 5.208
2)( rrN i
BB Example (5)
009826.
530
208.5)( 22
i
iir N
rrNs
2868.r 5.132N
006405.15.132
)2868.1(
1
)1(ˆ
22222
N
re
003421.006405.009826.ˆˆ 222 ers
4k
384,.190.4
009826.96.12868.96.1%95
2
k
srCI r
Interpretation
Schmidt says this is a random-effects meta-analysis. It uses a sample of studies to represent a larger population of studies.
People interpret the Credibility Interval, but typically do not recognize that it is poorly estimated.