Embed Size (px)
Citation preview
To Prevent Selection Bias
Wenle Zhao, PhD
Medical University of South Carolina, Charleston, SC, 29425, USA
Society for Clinical Trials 36th Annual MeetingArlington, VA, USA - May 17-20, 2015
Minimal Balance is Sufficient
Contents
1. Where does Selection Bias Come From?
2. How to prevent selection bias?
3. How to avoid random serious imbalance?
The Worst Thing in the World of Clinical Trials
Funding?Recruitment?
A Completed Trial with Suspicious SELECTION BIAS.
Defense Measurements against Selection Bias
Random Allocation
Allocation Concealment
Treatment Masking
Subject Enrollmen
t
Treatment Allocation
Outcome Assessme
nt
Real-time Subject
Randomization
The only reliable protection left against selection bias.
Allocation Randomness
1, , , i i iT F R W T X
Target allocation ratio
To balance treatment distributionPermuted Block Randomization
Biased Coin, Urn Design
Random variable ~U(0,1)
To balance baseline covariateStratified Randomization
Minimization
Allo
cati
on R
and
om
ness
Complete Randomization
Permuted BlockRandomization
Minimization
Predictability Defeats Concealment & Masking
Pr( ) 1iT A Deterministic Assignment
50
%
Pro
port
ion o
f D
A
100%
B = 2
33
%
B = 4
25 %
B = 6Permuted Block
Randomization
~ 8
7%
Minimization
2 0 %B =
8
0
Evidence of Selection Bias in Randomized Trials 1. Heparin for myocardial
infarction2. University Group Diabetes Program3. Talc and mustine for pleural
effusions4. Tonsillectomy for recurrent throat infection in children5. Oxytocin and amniotomy for induction of
labor6. Western Washington Intracoronary Streptokinese Trial7. RSV immune globulin in infants and young
children8. A trial to assess episiotomy9. Canadian National Breast Cancer Screening Study10. Surgical Trial11. Lifestyle Heart Trial
12. Coronary Artery Surgery Trial 13. Etanercept for children with juvenile rheumatoid arthritis14. Edinburgh Randomized Trial of Breast Cancer
Screening15. Captopril Prevention Project
16. Göteborg (Swedish) Mammography Trial
17. HIP Mammography Trial18. Hypertension Detection and Follow-up Program
19. Randomized Trial to prevent vertical transmission of HIV-1
20. Effectiveness trial of diagnostic test21. S African trial of high-dose chemotherapy for metastatic breast cancer22. Randomized study of a culturally sensitive AIDS education
program23. Runaway Youth Study24. Cluster randomized trial of palliative care25. Randomized trial of methadone with or without
heroin26. Randomized NINDS trial of tissue plasminogen activator for acute ischemic stroke27. Norwegian Timolol
Trial 28. Laparoscopic versus open appendectomy
29. The Losartan Intervention for Endpoint reduction in Hypertension Study30. The Heart Outcomes Prevention Evaluation
Study
EE
PP
PP
P
EE
PP
EP
EP
PE
EP
P
PE
PP
PP
PP
PP
E Selection bias
evidence identified
P Suspicious election bias
due to p-value <
0.05
Protect Trials Against Selection Bias
P < 0.05 ?
P > 0.2
P > 0.3
Selection bias will result small p-values
Complete randomization may (5% chance) see a p-value < 0.05
The Logic
Real-time complete randomization Eliminates selection bias due to allocation
predictability Eliminates selection bias due to allocation concealment
failure Totally eliminates selection bias Without selection bias, complete randomization may still
have Imbalance in treatment distribution
Power loss is trivial Imbalance is baseline covariate distribution
Adjustment, not balancing, is the solution Serious baseline covariate imbalance with p-value <
0.05 5% chance for any covariate 60% chance for at least one in 10
covariates Suspicion of selection bias Trouble in trial result interpretation
Options We HaveStratified Restricted Randomization
Permuted Block Randomization Biased Coin Design - Efron Urn Design - Wei Big Stick Design – Soares & Wu Maximal Procedure – Berger et al. Block Urn Design – Zhao & Weng
Unnecessarily tighten control imbalances.
Disabled when number of strata getting large.
Minimization
Most assignments are deterministic.
Dynamic Hierarchy Balancing
Hierarchy order is hard to justify.
Minimal Sufficient Balance Procedure
Subject ready for randomization
Any serious imbalance?
Current assignment can effectively
reduce imbalances?
Complete randomization Biased coin assignment
Next subject
N
N
Y
Y
Any p-value < 0.2?
The proportion depends on p-
value threshold and biased coin
probability
T-test for continuous var.
χ2 test for categorical var.
p-value Site NIHSS Age OTT GlucoseStroke
SubtypeSex Fibrinogen Weight
Systolic BP
Diastolic BP
Observed in the Original Study
0.9987
0.1398
0.0289
0.8662
0.7804 0.07330.626
50.1808
0.0111
0.5968 0.2810
Serious imbalances found in 2 of the 11 baseline covariates.
Example : NINDS rt-PA Stroke Study
Example : NINDS rt-PA Stroke Study
NINDS rt-PA Stroke study data. Simulations = 5000
Baseline covariates:• Severity (NIHSS)• Age• Onset to treat• Glucose• Center
Distribution of p-Values for baseline covariate imbalance tests with 11 covariates controlled*NINDS rt-PA data, Imbalance control limit p-Value ≥ 0.3. ξ= 0.65, simulation = 1000/scenario.
p-value Site NIHSS Age OTT GlucoseStroke
SubtypeSex Fibrinogen Weight
Systolic BP
Diastolic BP
Low 2.5% boundary
0.226 0.259 0.237 0.262 0.244 0.262 0.214 0.245 0.239 0.252 0.246
Low 5% boundary
0.276 0.295 0.279 0.288 0.280 0.292 0.277 0.288 0.278 0.281 0.287
Low 10% boundary 0.309 0.341 0.316 0.323 0.315 0.326 0.309 0.322 0.319 0.322 0.330
Median 0.605 0.631 0.626 0.624 0.624 0.638 0.609 0.634 0.626 0.638 0.610
Observed in the Original Study
0.9987
0.1398
0.0289
0.8662
0.7804 0.07330.626
50.1808
0.0111
0.5968 0.2810
Balance 11 baseline covariates
Example : NINDS rt-PA Stroke Study
Summary
Complete randomization eliminates selection bias due to allocation predictability.
Real-time randomization eliminates selection bias due to allocation concealment failures.
Minimization method has the highest proportion of deterministic assignments, and therefore is vulnerable to selection bias.
Power loss due to treatment imbalance is trivial.
Justification, not balancing, is the solution for covariate confounding effects.
Using Minimal Sufficient Balancing to prevent random serious imbalances, while maintaining a high level of allocation randomness.
Some of my works on Randomization
.
Zhao W. A better alternative to stratified permuted block design for subject randomization in clinical trials. Stat Med. 2014 Dec 30;33(30):5239-48. doi: 10.1002/sim.6266. PMID: 25043719
Zhao W. Selection bias, allocation concealment and randomization design in clinical trials. Contemp Clin Trials. 2013 Sep;36(1):263-5. doi: 10.1016/j.cct.2013.07.005. Epub 2013 Jul 19. No abstract available. PMID: 23871796
Zhao W, Weng Y. Block urn design - a new randomization algorithm for sequential trials with two or more treatments and balanced or unbalanced allocation. Contemp Clin Trials. 2011 Nov;32(6):953-61. doi: 10.1016/j.cct.2011.08.004. PMID: 21893215
Zhao W, Hill MD, Palesch Y. Minimal sufficient balance--a new strategy to balance baseline covariates and preserve randomness of treatment allocation. Stat Methods Med Res. 2012 Jan 26. [Epub ahead of print] PMID: 22287602
Zhao W, Ciolino J, Palesch Y. Step-forward randomization in multicenter emergency treatment clinical trials. Acad Emerg Med. 2010 Jun;17(6):659-65. doi: 10.1111/j.1553-2712.2010.00746.x. PMID: 20624149
Zhao W, Weng Y, Wu Q, Palesch Y. Quantitative comparison of randomization designs in sequential clinical trials based on treatment balance and allocation randomness. Pharm Stat. 2012 Jan-Feb;11(1):39-48. doi: 10.1002/pst.493. PMID: 21544929
Zhao W, Durkalski V. Managing competing demands in the implementation of response-adaptive randomization in a large multicenter phase III acute stroke trial. Stat Med. 2014 Oct 15;33(23):4043-52. doi: 10.1002/sim.6213. Epub 2014 May 22. PMID: 24849843
Zhao W, Mu Y, Tayama D, Yeatts SD. Comparison of statistical and operational properties of subject randomization procedures for large multicenter clinical trial treating medical emergencies. Contemp Clin Trials. 2015 Mar;41:211-8. doi: 10.1016/j.cct.2015.01.013. Epub 2015 Jan 29. PMID: 25638754
