Embed Size (px)
Citation preview
The analysis of individual and average causal effects: Basic principles and some applications
EUROPEAN SCIENCE FOUNDATIONProgramme ‘Quantitative Methods in the Social Sciences’ (QMSS)
Seminar:‘Theory-Driven Evaluation and Intervention Studies in the Social Sciences’,
28-29 September 2006, Nicosia, Cyprus
Rolf Steyer
Institute of PsychologyDepartment of Methodology and Evaluation ResearchEmail: [email protected]
Rolf Steyer
Institute of PsychologyDepartment of Methodology and Evaluation ResearchEmail: [email protected]
Rolf Steyer
Institute of PsychologyDepartment of Methodology and Evaluation ResearchEmail: [email protected]
Rolf Steyer
Institute of PsychologyDepartment of Methodology and Evaluation ResearchEmail: [email protected]
Rolf SteyerUniversity of Jena (Germany)
Institute of PsychologyDepartment of Methodology and Evaluation ResearchEmail: [email protected]
www.uni-jena.de/svw/metheval
2
Overview
• Individual and average causal effects (Neyman, Rubin)
• Motivation: The Simpson Paradox
• Pre-Post Design with Control Group for the Analysis of Intervention Effects
• Designs for the Analysis of Individual Causal Effects
- Example with an Intervention
- Example with Method Effects
• Conclusions
3
The Simpson Paradox
Table 1. Total Sample
treatment
success
yes (X = 1)
no (X = 0)
total
yes (Y = 1) 500 600 1100
no (Y = 0) 500 400 900
1000 1000 2000
4
The Simpson Paradox
Table 2. Males
treatment
success
yes (X = 1)
no (X = 0)
total
yes (Y = 1) 300 75 375
no (Y = 0) 450 175 625
750 250 1000
5
The Simpson Paradox
Table 3. Women
treatment success
yes (X = 1)
no (X = 0)
total
yes (Y = 1) 200 525 725
no (Y = 0) 50 225 275
250 750 1000
6
The Simpson Paradox Table 2. Males
treatment
success
yes (X = 1)
no (X = 0)
total
yes (Y = 1) 300 75 375
no (Y = 0) 450 175 625
750 250 1000 Table 3. Women
treatment success
yes (X = 1)
no (X = 0)
total
yes (Y = 1) 200 525 725
no (Y = 0) 50 225 275
250 750 1000
7
The Simpson Paradox
0,50,6
0
0,2
0,4
0,6
0,8
1
Gesamtgruppe
ExpositionKontrolle
proportion of success TreatmentControl
total group
0,4
0,8
0,3
0,7
0
0,2
0,4
0,6
0,8
1
Männer
ExpositionKontrolle
Frauen
Anteil Erkrankter
0,4
0,8
0,3
0,7
0
0,2
0,4
0,6
0,8
1
Männer
ExpositionKontrolle
Females
proportion of success
Males
Males
TreatmentControl
8
The single-unit trial
Sample a person u, register her assignment to one of the treatment conditions and observe her outcome y.
In this single-unit trial U, X, and Y have a joint distribution
u1
treatment
y1
y2
y3
y4
y1
y2
y3
y4
control
u2
treatment
control
.
.
.
.
.
.
9
Individual and average causal effects (Neyman, Rubin)
Define the true-outcome variables as follows:
0(u) := E(Y 0 U = u)
and
1(u) := E (Y 1 U = u)
1-0(u) := 1(u) 0(u)
= individual causal effect of unit u
ACE1-0 = E (1-0) = E(1) – E(0)
=: average causal effect
Uni
t
P(U
= u
)
Sam
plin
g pr
obab
ilit
y
0(u
) = E
(Y0
| U =
u)
Tru
e ou
tcom
e un
der
cont
rol
1(
u) =
E (Y
1 | U
= u
) T
rue
outc
ome
unde
r tr
eatm
ent
ICE
1-0(
u) =
E (
Y1
| U =
u)
E
(Y
0 | U
= u
) In
divi
dual
cau
sal e
ffec
t
u1 1/8 68 82 14
u2 1/8 81 89 8
u3 1/8 89 101 12
u4 1/8 102 108 6
u5 1/8 112 118 6
u6 1/8 119 131 12
u7 1/8 131 139 8
u8 1/8 138 152 14
E ( 0) = 105 E ( 1) = 115 ACE1-0 = 10
10
Individual, conditional and average causal effects (Neyman, Rubin)
Per
son
P(U
= u
)
Sam
plin
g pr
obab
ility
0(u
) = E
(Y |
X =
0, U
= u
) T
rue
outc
ome
unde
r co
ntro
l
1(
u) =
E(Y
| X
= 1
, U =
u)
Tru
e ou
tcom
e un
ter
trea
tmen
t
1-0
(u)
= E
(Y |
X =
1, U
= u
)
E
(Y |
X =
0, U
= u
) In
divi
dual
cau
sal e
ffec
t
Indi
vidu
al tr
eatm
ent
Pro
babi
lity
P(X
=1|
U=
u)
u1 1/8 68 82 14 8/9
u2 1/8 81 89 8 7/9
u3 1/8 89 101 12 6/9
u4 1/8 102 108 6 5/9
u5 1/8 112 118 6 4/9
u6 1/8 119 131 12 3/9
u7 1/8 131 139 8 2/9
u8 1/8 138 152 14 1/9
E ( 0) = 105 115 = E ( 1) ACE1-0 := E (1-0) = E ( 1) E ( 0) = 10 PFE1-0 := E(Y | X = 1) E(Y | X = 0) = 13.33
E(Y X = j) = u E(Y X = j, U = u) P(U=u X = j)
E(τj) = u E(Y X = j, U = u) P(U=u)
11
Bias Theorem
Bias Theorem. (i) Let X and Y be the random variables defined in the single-unit trial. Then
PFEj−0 = ACEj−0 + baseline biasj−0 + effect biasj−0, for each j = 1, . . . , J, where
baseline biasj−0 := E(τ 0 |X = j ) − E(τ 0 |X = 0)
and
effect biasj−0 := E(τ j 0 |X = j ) − ACE j 0.
12
Three Design Types
Between-Group Designs
Pre-post Designs (not used at all in the Neyman-Rubin tradition)
Between-Group Designs with Pre-Post Measures (only the between group comparisons are used in the Neyman-Rubin tradition)
13
Utilizing Pre-Post Designs for the Analysis of Individual Effects
Pre-post Designs and Between-Group Designs with Pre-Post Measures can be used to analyze not only average but also individual causal effects.
The crucial asssumption is that the individual pretest distribution is the same as the individual posttest (outcome variable) distribution under control (no treatment).
14
Theorem 1 (Sufficient conditions for unbiasedness of the (conditional) prima facie effects)
If X U, then the regression E(Y X)
and its values E(Y X = x) are unbiased
If X U | Z, then the regression E(Y X, Z)
and its values E(Y X = x, Z = z) are unbiased
(There are also other sufficient conditions for unbiasedness.)
15
Theorem 2
Suppose we have just 2 treatment conditions X = 0 and X = 1, then
E(Y | X, Z) = g0(Z) + g1-0(Z) X If E(Y | X, Z) is unbiased, then the values of g1-0(Z) are the average causal effects of X on Y given Z = z, and E[g1-0(Z)] is the average causal effect.
16
Standard research questions in the analysis of causal effects
The standard research questions are:
• What are the conditional effects of treatment as compared to the control given Z?
Hence, we want to (a) estimate the effect function g1-0(Z) and (b) test H0: g1-0(Z) = 00 (constant) no interaction
• What is the average effect of treatment as compared to the control?
Hence, we want (c) to estimate the average effect E[g1-0(Z)] and (d) test H0: E[g1-0(Z)] = 0 no average effect
17
Generalization to J + 1 Treatment Conditions
For J + 1 treatment conditions, the covariate treatment regression can always be written:
E(Y | X, Z) = g0(Z) + g1-0(Z) IX=1 + ... + gJ-0(Z) IX=J
18
Average effects in the analysis of causal effects
Suppose g1-0(Z) is a linear function 10 + 11 Z
E [g1-0(Z)] = E [10 + 11 Z ] = 10 + 11 E(Z ) H0: E[g1-0(Z)] = 10 + 11 E(Z ) = 0 no average effect
If E(Z ) has to be estimated: this involves a nonlinear hypothesis
19
Types of covariates
The covariate Z can be: manifest discrete manifest continuous latent discrete latent continuous univariate or multivariate stochastic fixed
20
Example (using EffectLite): Intelligence training
21
Pre-Post Design with Control Group for the Analysis of Intervention Effects
22
Between-Group Design, no Pretest
Y0 = τ0 + 0 Y1 = 0 + 1 τ0 + 1
Control
Y0 0
0
Treatment
Y1 1
0
1 0
Uni
t
P(U
= u
)
Sam
plin
g pr
obab
ilit
y
0(u
) = E
(Y0
| U =
u)
Tru
e ou
tcom
e un
der
cont
rol
1(
u) =
E (Y
1 | U
= u
) T
rue
outc
ome
unde
r tr
eatm
ent
ICE
1-0(
u) =
E (
Y1
| U =
u)
E
(Y
0 | U
= u
) In
divi
dual
cau
sal e
ffec
t
u1 1/8 68 82 14
u2 1/8 81 89 8
u3 1/8 89 101 12
u4 1/8 102 108 6
u5 1/8 112 118 6
u6 1/8 119 131 12
u7 1/8 131 139 8
u8 1/8 138 152 14
E ( 0) = 105 E ( 1) = 115 ACE1-0 = 10
23
Pre-Post Design, no Control Group
Y11
0
1 - τ0
Y00
Treatment
Y0 = 0 + 0 Y1 = 0 + (1 0) + 1
= 1 + 1
Uni
t
P(U
= u
)
Sam
plin
g pr
obab
ilit
y
0(u
) = E
(Y0
| U =
u)
Tru
e ou
tcom
e un
der
cont
rol
1(
u) =
E (Y
1 | U
= u
) T
rue
outc
ome
unde
r tr
eatm
ent
ICE
1-0(
u) =
E (
Y1
| U =
u)
E
(Y
0 | U
= u
) In
divi
dual
cau
sal e
ffec
t
u1 1/8 68 82 14
u2 1/8 81 89 8
u3 1/8 89 101 12
u4 1/8 102 108 6
u5 1/8 112 118 6
u6 1/8 119 131 12
u7 1/8 131 139 8
u8 1/8 138 152 14
E ( 0) = 105 E ( 1) = 115 ACE1-0 = 10
24
Identified Individual Effects Model with pretests Y11 and Y21
Control group
0
Y1212
Treatment group
Y2222
Y2121
Y1111
0
Y1212
Y2222
Y2121
Y1111
1 - 0
Treatment
25
Identified Individual Effects Model with pretests Y11 and Y21
Control group
0
Y1212
Treatment group
Y2222
Y2121
Y1111
0
Y1212
Y2222
Y2121
Y1111
1 - 0
x
Explanatory Variables
Treatment
26
Design for the Analysis of Method Effects
27
Y1212
Y2222
Y2121
Y1111h 1
h 2
Y1212
Y2222
Y2121
Y1111h 1
h 2 h 1
Introducing Individual Method Effects
28
Y1221
Y2222
Y2121
Y1111 11
τ21
12
τ22
Y1212
Y2222
Y2121
Y1111 1
2
Introducing Individual Method Effects
τ22 τ12 = τ21 τ11 = IME2-1
29
Y1221
Y2222
Y2121
Y1111 11
τ21 τ11
12
τ22 τ21
Y1221
Y2222
Y2121
Y1111 11
τ21
12
τ22
Introducing Individual Method Effects
τ22 τ12 = τ21 τ11 = IME2-1
30
Y1212
Y2222
Y2121
Y1111 11
IME2 1
12
An identifíed Individual-Method-Effects Model
τ22 τ12 = τ21 τ11 = IME2-1
31
Model in treatment group
IQ9.33
ICE3.85
IME0.42
Y11 1.51
Y21 1.51
Y12 1.51
Y22 1.51
Chi-Square=9.76, df=9, P-value=0.36998, RMSEA=0.025
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
-3.13
-0.51
0.19
Treatment
32
Model in control group
IQ9.50
ICE1.45
IME1.13
Y11 1.51
Y21 1.51
Y12 1.51
Y22 1.51
Chi-Square=9.76, df=9, P-value=0.36998, RMSEA=0.025
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
-2.43
-1.23
0.55
33
Model in treatment group (t-values)
IQ7.38
ICE5.85
IME1.59
Y11 11.77
Y21 11.77
Y12 11.77
Y22 11.77
Chi-Square=9.76, df=9, P-value=0.36998, RMSEA=0.025
-4.34
-1.27
0.68
34
Model in control group (t-values)
IQ7.43
ICE3.84
IME3.31
Y11 11.77
Y21 11.77
Y12 11.77
Y22 11.77
Chi-Square=9.76, df=9, P-value=0.36998, RMSEA=0.025
-4.41
-2.53
2.28
35
Correlation Matrix of ETA in Control group
IQ ICE IME
-------- -------- --------
IQ 1.00
ICE -0.65 1.00
IME -0.37 0.43 1.00
36
Correlation Matrix of ETA in Experimental Group
IQ ICE IME
-------- -------- --------
IQ 1.00
ICE -0.52 1.00
IME -0.26 0.15 1.00
37
The effects of negativ item formulation
good10.64
good20.72
good30.72
good40.65
ME0.14
emot0.05
GUT1 0.10
SCHLECH1 0.20
GUT2 0.08
SCHLECH2 0.13
GUT3 0.12
SCHLECH3 0.10
GUT4 0.15
SCHLECH4 0.10
EMOT1 0.03
EMOT2 0.04
Chi-Square=53.09, df=28, P-value=0.00287, RMSEA=0.042
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.001.00
1.00
1.00
1.00
1.00
0.59
0.19
0.20
0.230.17
0.26
0.29
-0.07
-0.04
-0.05
-0.06
0.08
0.05
0.05
0.04
38
The effects of negativ item formulation
bad10.45
bad20.50
bad30.48
bad40.42
ME0.15
emot0.07
trait0.32
GOOD1 0.15
BAD1 0.15
GOOD2 0.12
BAD2 0.12
GOOD3 0.12
BAD3 0.12
GOOD4 0.12
BAD4 0.12
EMGES4A 0.01
EMGES4B 0.03
Chi-Square=76.70, df=43, P-value=0.00119, RMSEA=0.039
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
1.00
0.78
1.00
1.00
1.00
1.00
-0.03
-0.13
0.10
39
The effects of negativ item formulation (standardized)
bad10.58
bad20.61
bad30.60
bad40.57
ME1.00
emot1.00
trait1.00
GOOD1 0.18
BAD1 0.16
GOOD2 0.14
BAD2 0.12
GOOD3 0.14
BAD3 0.13
GOOD4 0.15
BAD4 0.13
EMGES4A 0.11
EMGES4B 0.43
Chi-Square=76.70, df=43, P-value=0.00119, RMSEA=0.039
0.97
0.43
0.91
0.99
0.43
0.94
0.99
0.43
0.94
0.99
0.45
0.93
0.95
0.76
0.65
0.63
0.63
0.66
-0.26
-0.56
0.70
40
Summary and Conclusion
• We can analyze individual (and average) causal effects in Pre-post Designs
• The causal interpretation rests on assumptions
• These assumptions can be tested
• Latent variables can be constructed from true-scores
• Not a single path in our SEM models represented a causal effect
41
Want More?
Steyer, R. & Partchev, I.. (2006). Causal Effects in Experiments and Quasi-Experiments: Theory (Chapters 1 -5 are available at www.causal-effects.de)
• Symposium on causality in Jena July 7 to 9, 2006 with Tom Cook, Steve West, Don Rubin … (videos available: see www.uni-jena.de/svw/metheval
• Online video of workshop on the analysis of causal effects (same home page)
• Software „EffectLite“ (see: www.statlite.com)
42
Thanks to:
Sven HartensteinUlf KröhneBenjamin NagengastIvailo PartchevSteffi Pohl
