Upload
della-roberta-kelley
View
223
Download
2
Tags:
Embed Size (px)
Citation preview
Design & Interpretation of Randomized Trials:A Clinician’s PerspectiveFrancis KL ChanDepartment of Medicine & TherapeuticsCUHK
Common problems of RCTs
Originality
Hypothesis
Allocation concealment & randomization
Evaluation of baseline data
“Intention-to-treat” analysis
Subgroup analysis
Is the study original?
Ground breaking research?
Does this work add to the literature in any way?
Bigger, longer?
More rigorous methodology?
Results add to a meta-analysis of previous studies?
Different population (age, sex, ethnic groups)?
Hypothesis & End pointMany RCTs did not explicitly state their study
hypotheses
“The aim of this study was to compare the efficacy of a new treatment with the standard treatment…”
Hypothesis 1:
Treatment A is superior to the standard treatment
Hypothesis 2:
Treatment A is equivalent to the standard treatment
Hypothesis?
Sample size estimation
None!
Failure to detect a difference
=Equivalence?
Superiority Trial
The new treatment (µN) is superior to the
standard treatment (µS) if the difference
exceeds by a clinically important amount ().
Test hypothesis (H): µN - µS>
Equivalence trial
The new treatment is equivalent to the
standard treatment if the maximal allowable
difference does not exceed by a clinically
important amount.
Equivalence trial
- 0 +
Equivalent
Difference
Not equivalent
Favors new treatment
Not equivalent
Favors standard
treatment
New agent is not inferior to the
standard
Assume non-inferiority if the lower limit of 95% CI is
less than –5%,N=904 per group!
Allocation concealment & randomization
Concealment of allocation
(investigators and patients not knowing the assigned
treatment before randomization)
Was treatment assigned by an independent staff?
What was the method of allocation concealment?
contact with central office
blinded packages
sealed (opaque) envelopes
Allocation concealment
NEJM, JAMA, Lancet, BMJ (N=50, Jul - Sep 97)
44%
56%
YesNo
Comparison of baseline data
Chan et al. Lancet 1997
Does P>0.05 indicate
comparability of treatment groups?
Baseline data
Azathioprine Placebo
Mean age 54.7 54.9
Serum bilirubin (mol/L) 37.2 30.9
Stage I disease % 14 12
Stage II disease % 44 43
Stage III disease % 15 15
Stage IV disease % 27 30
Christensen et al. Gastro 1985
Effect of azathioprine on the survival of patients with primary biliary cirrhosis
Baseline data
Azathioprine Placebo
Mean age 54.7 54.9
Serum bilirubin (mol/L) 37.2 30.9
Stage I disease % 14 12
Stage II disease % 44 43
Stage III disease % 15 15
Stage IV disease % 27 30
Christensen et al. Gastro 1985
Effect of azathioprine on the survival of patients with primary biliary cirrhosis
UnadjustedP=0.10
Adjusted for bilirubin
P=0.01
P=0.04
P=0.02
Columbus Investigators. NEJM 1997
Significant imbalance may not affect outcome
Comparison of baseline data
Non-significant imbalance may affect outcome
Significance tests for baseline differences are inappropriate.
Significance tests for baseline differences
Chan et al. Lancet 1997
INAPPROPRIA
TE
Significant imbalance may not affect outcome
Comparison of baseline data
Non-significant imbalance may affect outcome
Significance tests for baseline differences are inappropriate.
Table of baseline data should focus on factors affecting outcome.
45 baseline factors!
Significant imbalance may not affect outcome
Comparison of baseline data
Non-significant imbalance may affect outcome
Significance tests for baseline differences are inappropriate.
Table of baseline data should focus on factors affecting outcome.
Analysis adjusted for baseline factors that are known to strongly influence the outcome (Covariate-adjusted analysis).
Analysis of covariance for a quantitative outcome
Logistic regression for a binary response
Cox’s-proportional hazard model for time-to-event data
“Intention-To-Treat” Analysis
“…results were analyzed according to the ITT principle.”
Question:
How were missing outcomes/ protocol violators
handled in the so called “ITT” analysis?
“Intention-To-Treat” Analysis
Endpt
Savage et al. NEJM 1997
Minimize missing response on primary outcome
Recommendations for ITT Analysis
Follow up subjects who withdraw early
Investigate & report the effect of missing response
Report all deviations and missing response
Subgroup AnalysisRandomised trial of home-based psychosocial nursing intervention for patients recovering from myocardial infarction. Frasure-Smith et al. Lancet 1997
“…The poor overall outcome for women, and the possible harmful impact of the intervention on women, underlie the need for…”
Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome.
Am J Obstet Gynecol 1981;141:276-87.
Subgroup Analysis
Steroid Placebo P
Pre-ecclampsia 21.2%
(7/33)
27.3%
(9/33)
0.57
No pre-ecclampsia 7.9%
(21/267)
14.1%
(37/262)
0.021
Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome.
Am J Obstet Gynecol 1981;141:276-87.
Subgroup Analysis
Steroid Placebo P
Pre-ecclampsia 21.2%
(7/33)
27.3%
(9/33)
0.57
No pre-ecclampsia 7.9%
(21/267)
14.1%
(37/262)
0.021
Difference 6.1%
Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome.
Am J Obstet Gynecol 1981;141:276-87.
Subgroup Analysis
Steroid Placebo P
Pre-ecclampsia 21.2%
(7/33)
27.3%
(9/33)
0.57
No pre-ecclampsia 7.9%
(21/267)
14.1%
(37/262)
0.021
Difference 6.1%
Difference 6.2%
P value depends on effect size & SE
Evaluation of Subgroup Analysis
Tests of interaction (assess whether a treatment
effect differs between subgroups) rather than
subgroup P values
Diff in Subgroup A – Diff in Subgroup B
SE of the above Diff Z =
Trial of vitamin D supplements in pregnancy to prevent infant hypocalcemia. BMJ 1980;281:11-4.
Infant plasma calcium (mg/100ml)
Mean SEM P
Vit D 9.20 0.085 Bottle-fed
Placebo 8.78 0.076
0.0006
Vit D 9.79 0.146 Breast-fed Placebo 9.64 0.125
0.4
Interaction TestDifference = 0.42
Difference = 0.15
0.42 – 0.15 = 0.27
SE of this Diff = 0.22
Z = Diff / SE = 1.23
P = 0.2No evidence th
at the effe
ct of V
it D is
different
between bottle
-fed and breast-fe
d infants
General points regarding subgroup analysis
Emphasis should remain on overall comparison
More convincing if confined to a limited number of pre-specified subgroup hypothesis
Rely on interaction tests, not P values
View subgroup findings as exploratory (to be confirmed in subsequent trials)