Upload
idalee
View
33
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Propensity score and subgroups: How to find an accurate treatment effect within subgroups when the propensity score is applied to control for selection bias?. Hester van Eeren Erasmus Medical Centre, Rotterdam Halsteren, August 23, 2010. The propensity score (1). - PowerPoint PPT Presentation
Citation preview
1
Hester van EerenErasmus Medical Centre, Rotterdam
Halsteren, August 23, 2010
2
The propensity score (1)The propensity score is “…the conditional
probability of assignment to a particular treatment given a vector of observed covariates.” (Rosenbaum en Rubin, 1983: 41).• Used in non-randomized studies to control for
selection bias• Balance observed pretreatment variables among
patient• Find an estimate of the average treatment effect
But, treatment effect can be different within subgroups
3
The propensity score (2)Univariate propensity score Multivariate propensity score
Bartak and colleagues (2009) Spreeuwenberg and colleagues (2010)
Used for 2 treatment categories Used for > 2 treatment categories
Propensity score used in: • Matching• Stratification• Regression• Inverse probability weight• Combinations of …
Methods in this studyTo find a treatment effect within subgroups, if the
propensity score is applied:• Method 1: Regression analysis with propensity score,
subgroups and interaction with treatment assignment;
• Method 2: Weighted regression analysis with inverse of the propensity score (to weight observations), subgroups and interaction with treatment assignment;
• Method 3: Propensity score applied for groups defined on treatment assignment and subgroups; then, regression analysis with propensity score and dummies for groups
Two treatment categories and two subgroups are used in this study
4
Variable selection for propensity scoreDoes the variable for subgroups has to be included?Discussion about variable selection for propensity score;
• Only variables related to outcome?• Only variables related to treatment assignment?• Both variables…?
In this study; 8 different propensity scores (PS) formulated, based
on: • Variables related to outcome, with and without subgroup• Variables related to treatment assigment, with and without
subgroup• Both variables…, with and without subgroup• Only variables related to both outcome and treatment
assignment, with and without subgroup5
How to test? (1)Real dataset not useful because effects
unknown beforehand; You cannot test whether the effect found is
accurate
Monte Carlo simulation study to test methods and different propensity scores:Simulate data with known treatment effectsEstimate different propensity scores for this dataApply different methods for different propensity scores,
for this data Repeat this process 1000 times
6
How to test? (2)What do you want to know?
If the treatment effect estimated is (almost) equal to the treatment effect you used to simulate the data
Bias of estimator: difference between estimated treatment effect and the true value of parameter
Want to have an unbiased estimate; Less bias indicates a more accurate estimate of the
treatment effect
Bias is estimated for overall treatment effect and for the treatment effect within subgroups
7
Results
8
N=250;ρ=0 N=250;ρ=0.3 N=250; ρ=0.7 N=500;ρ=0 N=500;ρ=0.3 N=500; ρ=0.7 N=1000;ρ=0 N=1000;ρ=0.3 N=1000; ρ=0.7
Row Method Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSERegression1 PS1, func 1 .0736 .0243 .0706 .0265 .0797 .0304 .0844 .0160 .0763 .0159 .0803 0.172 .0812 .0109 .0786 .0113 .0834 .0135
2 PS1, func 2 .0728 .0237 .0724 .0258 .0785 .0288 .0853 .0156 .0774 .0156 .0807 .0169 .0801 .0104 .0779 .0110 .0825 .0130
3 PS1, func 3 -.0060 .0312 -.0050 .0336 -.0014 .0345 .0042 .0149 -.0025 .0162 .0006 .0168 -.0001 .0071 -.0053 .0080 .0029 .0092
4 PS1, subgr. -.0035 .0820 -.0056 .0762 -.0008 .0719 .0030 .0414 .0007 .0382 .0000 .0398 .0006 .0218 .0085 .0193 -.0014 .0178
5 PS2, func 1 .0746 .0262 .0710 .0290 .0798 .0317 .0860 .0175 .0745 .0171 .0804 .0180 .0813 .0116 .0787 .0119 .0838 .01396 PS2, func 2 .0740 .0257 .0730 .0281 .0785 .0298 .0864 .0171 .0755 .0166 .0807 .0175 .0804 .0112 .0778 .0116 .0830 .0135
7 PS2, func 3 -.0049 .0343 -.0055 .0371 -.0023 .0368 .0070 .0169 -.0053 .0183 .0012 .0178 .0002 .0083 -.0061 .0088 .0033 .0098
8 PS2, subgr. -.0030 .0901 -.0025 .0892 .0020 .0765 -.0015 .0467 .0028 .0448 -.0014 .0433 .0006 .0243 .0105 .0220 -.0009 .0196
9 PS3, func 1 .0727 .0237 .0723 .0258 .0787 .0289 .0852 .0156 .0774 .0156 .0807 .0169 .0802 .0104 .0779 .0110 .0824 .0130
10 PS3, func 2 .0740 .0237 0730 .0258 .0785 .0289 .0864 .0156 .0755 .0156 .0807 .0169 .0804 .0104 .0778 .0110 .0830 .0130
11 PS3, func 3 -.0062 .0312 -.0053 .0337 -.0017 .0346 .0039 .0149 -.0028 .0162 .0005 .0168 -.0001 .0071 -.0053 .0080 .0027 .0091
12 PS3, subgr. -.0030 .0819 -.0047 .0763 .0001 .0719 .0034 .0414 .0012 .0383 .0003 .0399 .0008 .0218 .0088 .0190 -.0011 .0178
13 PS4, func 1 .0738 .0257 .0731 .0281 .0782 .0299 .0863 .0170 .0756 .0166 .0808 .0175 .0805 .0112 .0778 .0116 .0830 .0135
14 PS4, func 2 .0739 .0257 .0730 .0281 .0782 .0299 .0862 .0170 .0756 .0166 .0807 .0175 .0805 .0112 .0779 .0116 .0830 .0135
15 PS4, func 3 -.0051 .0343 -.0056 .0372 -.0029 .0370 .0068 .0169 -.0053 .0183 .0011 .0178 .0002 .0083 -.0061 .0088 .0030 .0098
16 PS4, subgr. -.0027 .0901 -.0020 .0894 .0027 .0766 -.0012 .0466 .0033 .0449 -.0012 .0433 .0007 .0243 .0107 .0220 -.0006 .0197
17 PS5, func 1 .0838 .0345 .0855 .0383 .0859 .0357 .0809 .0201 .0785 .0213 .0779 .0196 .0827 .0138 .0768 ..0126 .0812 .0130
18 PS5, func 2 .0836 .0332 .0850 .0370 .0872 .0345 .0806 .0193 .0769 .0202 .0778 .0189 .0824 .0136 .0766 .0123 .0800 .0126
19 PS5, func 3 .0047 .0407 .0027 .0440 .0129 .0396 -.0018 .0209 -.0056 .0209 -.0026 .0183 -.0002 .0104 -.0002 .0100 .0000 .0085
20 PS5, subgr. -.0023 .0898 .0061 .0895 -.0135 .0868 .0064 .0481 .0066 .0421 .0013 .0381 .0067 .0224 -.0081 .0220 -.0006 .0184
21 PS6, func 1 .0836 .0332 .0849 .0369 .0871 .0346 .0805 .0193 .0769 .0202 .0778 .0188 .0823 .0136 .0765 .0123 .0800 .0126
22 PS6, func 2 .0836 .0332 .0850 .0370 .0871 .0345 .0806 .0193 .0768 .0202 .0778 .0189 .0823 .0136 .0765 .0123 .0801 .0126
23 PS6, func 3 .0046 .0407 .0023 .0439 .0126 .0395 -.0019 .0210 -.0059 .0209 -.0027 .0183 -.0003 .0104 -.0004 .0100 -.0001 .0085
24 PS6, subgr. -.0021 .0898 .0069 .0834 -.0127 .0869 .0067 .0481 .0070 .0421 .0016 .0381 .0068 .0224 -.0079 .0220 -.0003 .0184
25 PS7, func 1 .0839 .0320 .1611 .0528 .1663 .0544 .0791 .0180 .1561 .0380 .1584 .0377 .0829 .0130 .1561 .0303 .1632 .0328
26 PS7, func 2 .0836 .0307 .1603 .0518 .1678 .0536 .0784 .0173 .1550 .0369 .1587 .0373 .0827 .0129 .1560 .0301 .1621 .0322
27 PS7, func 3 .0045 .0362 .0790 .0460 .0948 .0459 -.0037 .0180 .0724 .0242 .0786 .0235 .0017 .0093 .0794 .0154 .0827 .0152
28 PS7, subgr. -.0013 .0819 .0037 .0849 -.0162 .0828 .0055 .0435 .0068 .0402 .0005 .0379 .0029 .0207 -.0085 .0205 -.0022 .0185
29 PS8, func 1 .0837 .0307 .1602 .0517 .1679 .0537 .0783 .0173 .1551 .0369 .1587 .0373 .0827 .0129 .1560 .0301 .1621 .0323
30 PS8, func 2 .0836 .0307 .1603 .0517 .1680 .0538 .0784 .0173 .1550 .0369 .1587 .0373 .0827 .0129 .1560 .0301 .1621 .0323
31 PS8, func 3 .0043 .0362 .0787 .0458 .0947 .0459 -.0039 .0181 .0722 .0242 .0785 .0235 .0016 .0093 .0793 .0154 .0826 .0152
32 PS8, subgr. -.0008 .0819 .0046 .0848 .0059 .0436 .0073 .0402 .0009 .0379 .0009 .0379 .0031 .0207 -.0083 .0205 -.0019 .0185
N=250;ρ=0 N=250;ρ=0.3 N=250; ρ=0.7 N=500;ρ=0 N=500;ρ=0.3 N=500; ρ=0.7 N=1000;ρ=0 N=1000;ρ=0.3 N=1000; ρ=0.7
Row Method Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSERegression1 PS1, func 1 .0812 .0272 .1017 .0301 .0987 .0321 .0874 .0173 .1043 .0208 .0960 .0208 .0796 .0113 .1043 .0158 .1019 .0160
2 PS1, func 2 .0780 .0258 .0735 .0247 .0715 .0274 .0851 .0163 .0746 .0149 .0687 .0156 .0796 .0110 .0751 .0104 .0763 .0114
3 PS1, func 3 -.0011 .0322 -.0022 .0296 -.0131 .0358 .0094 .0157 -.0040 .0148 -.0138 .0175 -.0002 .0080 -.0041 .0075 -.0109 .0087
4 PS1, subgr. .0025 .0806 -.0087 .0797 .0253 .0913 -.0103 .0426 -.0009 .0375 .0186 .0441 -.0005 .0202 .0000 .0184 .0306 .0218
5 PS2, func 1 .0824 .0289 .0792 .0280 .0762 .0289 .0866 .0185 .0799 .0175 .0741 .0180 .0781 .0118 .0780 .0119 .0802 .0125
6 PS2, func 2 .0805 .0273 .0813 .0280 .0763 .0293 .0846 .0175 .0767 .0170 .0726 .0169 .0780 .0115 .0802 .0117 .0801 .0124
7 PS2, func 3 -.0009 .0341 .0035 .0347 -.0086 .0380 .0094 .0179 .0011 .0170 -.0102 .0185 -.0006 .0092 .0000 .0086 -.0072 .0098
8 PS2, subgr. .0030 .0863 -.0047 .0968 .0248 .0989 -.0112 .0475 -.0012 .0451 .0183 .0477 -.0031 .0224 .0020 .0208 .0294 .0246
9 PS3, func 1 .0798 .0258 .0790 .0254 .0769 .0280 .0852 .0163 .0733 .0158 .0735 .0163 .0795 .0110 .0803 .0112 .0807 .0121
10 PS3, func 2 .0805 .0258 .0813 .0253 .0763 .0279 .0846 .0163 .0767 .0158 .0726 .0162 .0780 .0110 .0802 .0112 .0801 .0120
11 PS3, func 3 -.0014 .0322 .0018 .0296 -.0117 .0359 .0093 .0157 -.0002 .0148 -.0129 .0175 -.0003 .0080 -.0003 .0075 -.0105 .0088
12 PS3, subgr. .0033 .0806 -.0050 .0806 .0352 .0930 -.0098 .0426 .0027 .0377 .0286 .0451 -.0002 .0202 .0034 .0186 .0400 .0228
13 PS4, func 1 .0804 .0273 .0810 .0277 .0764 .0289 .0847 .0175 .0798 .0170 .0727 .0170 .0780 .0115 .0801 .0117 .0801 .0124
14 PS4, func 2 .0805 .0273 .0810 .0276 .0760 .0287 .0847 .0175 .0797 .0170 .0728 .0169 .0780 .0115 .0800 .0117 .0800 .0124
15 PS4, func 3 -.0010 .0342 .0030 .0347 -.0088 .0376 .0093 .0179 .0010 .0170 -.0102 .0185 -0007 .0092 -.0003 .0085 -.0073 .0097
16 PS4, subgr. .0033 .0863 -.0042 .0972 .0252 .0992 -.0109 .0476 -.0009 .0450 .0189 .0478 -.0029 .0224 .0021 .0209 .0294 .0247
17 PS5, func 1 .0851 .0343 .0769 .0326 .0800 .0330 .0875 .0213 .0784 .0201 .0752 .0191 .0830 .0139 .0787 .0134 .0729 .0118
18 PS5, func 2 .0845 .0329 .0758 .0317 .0786 .0321 .0873 .0204 .0789 .0195 .0744 .0184 .0830 .0136 .0779 .0129 .0743 .0116
19 PS5, func 3 .0062 .0410 -.0024 .0430 -.0029 .0403 .0078 .0201 -.0016 .0208 -.0123 .0185 .0009 .0097 -.0038 .0105 -.0092 .0095
20 PS5, subgr. -.0046 .0945 -.0029 .0899 .0144 .0955 -.0010 .0465 .0031 .0442 .0288 .0450 .0054 .0243 .0063 .0252 .0189 .0236
21 PS6, func 1 .0844 .0329 .0756 .0317 .0786 .0322 .0872 .0204 .0791 .0196 .0743 .0184 .0830 .0136 .0781 .0129 .0742 .0116
22 PS6, func 2 .0844 .0330 .0756 .0317 .0783 .0321 .0872 .0204 .0790 .0195 .0741 .0183 .0830 .0136 .0780 .0129 .0741 .0115
23 PS6, func 3 .0059 .0410 -.0030 .0430 -.0034 .0405 .0075 .0201 -.0016 .0208 -.0128 .0185 .0009 .0098 -.0039 .0105 -.0091 .0095
24 PS6, subgr. -.0041 .0934 -.0019 .0899 .0153 .0958 -.0006 .0465 .0035 .0442 .0296 .0451 .0056 .0243 .0065 .0252 .0185 .0234
25 PS7, func 1 .0831 .0303 .0187 .0592 .1980 .0650 .0875 .0199 .1881 .0478 .1941 .0505 .0816 .0128 .1882 .0420 .1939 .0439
26 PS7, func 2 .0819 .0289 .1426 .0437 .1426 .0447 .0875 .0192 .1461 .0332 .1389 .0318 .0815 .0125 .1448 .0272 .1406 .0256
27 PS7, func 3 .0046 .0359 .0677 .0433 .0672 .0412 .0080 .0185 .0677 .0231 .0585 .0210 -.0006 .0087 .0645 .0137 .0640 .0131
28 PS7, subgr. -.0068 .0852 -.0101 .0820 .0002 .0883 -.0010 .0437 -.0018 .0410 .0136 .0434 .0053 .0220 .0031 .0229 .0026 .0220
29 PS8, func 1 .0818 .0289 .1428 .0438 .1437 .0450 .0876 .0192 .1464 .0333 .1402 .0322 .0815 .0124 .1451 .0273 .1419 .0259
30 PS8, func 2 .0818 .0289 .1429 .0438 .1434 .0450 .0876 .0192 .1464 .0333 .1398 .0320 .0815 .0125 .1451 .0273 .1416 .0258
31 PS8, func 3 .0043 .0358 .0660 .0433 .0626 .0409 .0078 .0185 .0665 .0230 .0537 .0206 -.0006 .0087 .0631 .0135 .0595 .0126
32 PS8, subgr. -.0062 .0853 -.0051 .0821 .0146 .0892 -.0005 .0437 .0022 .0411 .0284 .0444 .0055 .0220 .0071 .0230 .0165 .0224
N=250;ρ=0 N=250;ρ=0.3 N=250; ρ=0.7 N=500;ρ=0 N=500;ρ=0.3 N=500; ρ=0.7 N=1000;ρ=0 N=1000;ρ=0.3 N=1000; ρ=0.7
Row Method Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSE Bias MSEInverse PS1 PS1, func 1 .0831 .0298 .1116 0377 .1402 .0712 .0892 .0181 .1054 .0244 .1127 .0409 .0810 .0118 .1041 .0173 .1102 .0311
2 PS1, func 2 .0835 .0283 .0714 .0295 .0738 .0490 .0881 .0173 .0611 .0165 .0478 .0257 .0816 .0117 .0597 .0101 .0483 .0175
3 PS1, func 3 .0049 .0380 -.0027 .0492 -.0034 .0781 .0120 .0198 -.0171 .0267 -.0311 .0473 .0027 .0099 -.0191 .0136 -.0387 .0313
4 PS1, subgr. .0015 .1177 -.0018 .1667 .0181 .2239 -.0068 .0634 .0032 .0881 .0136 .1394 -.0016 .0307 .0008 .0411 .0279 .0811
5 PS2, func 1 .0921 .0380 .1012 .0524 .1384 .0801 .0940 .0236 .0872 .0356 .1012 .0504 .0831 .0147 .0838 .0207 .0933 .0471
6 PS2, func 2 .0922 .0354 .1058 .0467 .1239 .0646 .0940 .0227 .0889 .0303 .0959 .0376 .0837 .0143 .0856 .0193 .0917 .0338
7 PS2, func 3 .0120 .0494 .0362 .0667 .0450 .0866 .0182 .0282 .0120 .0376 .0154 .0556 .0032 .0140 .0044 .0225 .0011 .0474
8 PS2, subgr. .0076 .1496 -.0095 .2144 .0219 .2375 -.0058 .0818 .0038 .1280 .0181 .1561 .0038 .0418 .0086 .0596 .0391 .1124
9 PS3, func 1 .0818 .0285 .0903 .0329 .1257 .0649 .0871 .0171 .0821 .0196 .1008 .0360 .0811 .0116 .0810 .0130 .1007 .0269
10 PS3, func 2 .0820 .0283 .0905 .0313 .1157 .0496 .0875 .0172 .0823 .0187 .0963 .0286 .0812 .0116 .0812 .0127 .0961 .0214
11 PS3, func 3 .0033 .0384 .0159 .0476 .0391 .0701 .0111 .0198 .0043 .0251 .0120 .0392 .0023 .0099 .0029 .0124 .0117 .0246
12 PS3, subgr. .0014 .1187 -.0026 .1638 .0108 .2112 -.0064 .0635 .0016 .0855 .0024 .1268 -.0015 .0307 -.0016 .0399 .0177 .0716
13 PS4, func 1 .0895 .0368 .1025 .0526 .1384 .0821 .0920 .0229 .0868 .0348 .1006 .0498 .0830 .0143 .0842 .0207 .0931 .0467
14 PS4, func 2 .0903 .0359 .1026 .0455 .1225 .0592 .0932 .0227 .0879 .0293 .0981 .0360 .0833 .0143 .0849 .0187 .0914 .0321
15 PS4, func 3 .0100 .0510 .0327 .0669 .0429 .0815 .0171 .0288 .0109 .0376 .0180 .0525 .0027 .0141 .0035 .0218 .0013 .0449
16 PS4, subgr. .0076 .1534 -.0092 .2170 .0230 .2368 -.0052 .0828 .0038 .1285 .0162 .1545 .0040 .0421 .0087 .0595 .0377 .1125
17 PS5, func 1 .0957 .0458 .0994 .0595 .1335 .0816 .0918 .0273 .0914 .0350 .1102 .0612 .0823 .0162 .0830 .0219 .0972 .0316
18 PS5, func 2 .0983 .0440 .0985 .0523 .1210 .0651 .0936 .0254 .0930 .0312 .1025 .0417 .0830 .0158 .0843 .0199 .0960 .0249
19 PS5, func 3 .0298 .0619 .0213 .0845 .0455 .0861 .0182 .0295 .0075 .0407 .0211 .0487 .0012 .0147 .0027 .0225 .0172 .0256
20 PS5, subgr. -.0204 .1722 .0117 .2172 .0125 .2358 -.0063 .0885 .0235 .1119 .0204 .1491 .0074 .0437 .0092 .0614 .0047 .0745
21 PS6, func 1 .0959 .0446 .0970 .0587 .1326 .0812 .0914 .0262 .0912 .0350 .1097 .0599 .0823 .0160 .0822 .0216 .0974 .0308
22 PS6, func 2 .0972 .0438 .0963 .0523 .1221 .0604 .0925 .0254 .0914 .0307 .1034 .0391 .0826 .0158 .0839 .0198 .0934 .0231
23 PS6, func 3 .0287 .0621 .0191 .0861 .0469 .0813 .0173 .0296 .0058 .0407 .0230 .0450 .0006 .0148 .0024 .0223 .0142 .0245
24 PS6, subgr. -.0208 .1731 .0115 .2208 .0111 .2352 -.0068 .0894 .0237 .1130 .0176 .1505 .0076 .0439 .0088 .0621 .0059 .0749
25 PS7, func 1 .0843 .0318 .1923 .0677 .2242 .1002 .0889 .0204 .1925 .0520 .2136 .0765 .0809 .0131 .1889 .0436 .2088 .0585
26 PS7, func 2 .0851 .0306 .1346 .0473 .1166 .0568 .0901 .0198 .1359 .0325 .1082 .0370 .0813 .0128 .1301 .0245 .1063 .0233
27 PS7, func 3 .0116 .0434 .0576 .0604 .0425 .0760 .0114 .0219 .0520 .0302 .0337 .0395 -.0012 .0107 .0502 .0160 .0332 .0199
28 PS7, subgr. -.0114 .1236 .0041 .1558 .0087 .2113 -.0007 .0648 .0165 .0792 .0022 .1262 .0076 .0325 .0032 .0407 -.0091 .0603
29 PS8, func 1 .0836 .0304 .1499 .0528 .1819 .0787 .0889 .0197 .1519 .0376 .1730 .0571 .0809 .0127 .1466 .0292 .1706 .0418
30 PS8, func 2 .0840 .0304 .1502 .0510 .1746 .0658 .0832 .0197 .1515 .0366 .1670 .0479 .0810 .0127 .1469 .0290 .1646 .0368
31 PS8, func 3 .0102 .0435 .0729 .0610 .1027 .0722 .0103 .0220 .0674 .0313 .0939 .0400 -.0016 .0107 .0672 .0176 .0932 .0243
32 PS8, subgr. -.0111 .1241 .0026 .1530 -.0049 .1924 -.0003 .0651 .0154 .0782 -.0076 .1122 .0079 .0325 .0015 .400 -.0174 .0522
Results (1)Which propensity score is the most accurate within
each method tested (tested with ANOVA):
But, some values for bias per propensity score where not very different from each other…
9
General treatment effect
Treatment effect within subgroups
Method 1
PS with variables related to outcome
PS with variables related to outcome
Method 2
PS with variables related to outcome and variable for subgroups
PS with variables only related to outcome and treatment assignment and variables for subgroups
Method 3
PS with variables related to outcome
NA
Results (2)Which method is most accurate when the most
accurate propensity scores are compared?Decide on partial effect size of method in ANOVA*
For general treatment effect, the partial effect size is 0,028, where method 1 gives the lowest bias (followed by method 3)
For treatment effect within subgroups, the partial effect size is 0,051, where method 1 gives the lowest bias too
Although the effect sizes for method are not very large, regression analysis with treatment assignment, subgroup, interaction between these and the propensity score, which is estimated with variables related to outcome, seems to be the most accurate method to find treatment effects within subgroups
*Effect size – 0,010 = small; 0,059 = medium; 0,138 = large (Cohen, 1988) 10
Discussion (1)Data simulation is done for different settings:
Sample size, correlation between covariates and correlation with covariate for subgroups are changed over simulations
Results for most accurate propensity score are based on sum of bias over all these settings; comparisons between methods for all propensity scores could give more in depth results
The overall bias for different propensity scores was sometimes not very different
Model for simulation of data was simple, linear; the relation between variables and outcome in practice can be more complicated
…. 11
Discussion (2)
Questions?
12