Upload
waseemah-duaa
View
20
Download
0
Tags:
Embed Size (px)
DESCRIPTION
How many patients do I need for my study? Realistic Sample Size Estimates for Clinical Trials. Sample Size Estimation. 1. General considerations 2. Continuous response variable Parallel group comparisons Comparison of response after a specified period of follow-up - PowerPoint PPT Presentation
Citation preview
Sample Size Estimation1. General considerations
2. Continuous response variable– Parallel group comparisons
• Comparison of response after a specified period of follow-up• Comparison of changes from baseline
– Crossover study
3. Success/failure response variable– Impact of non-compliance, lag– Realistic estimates of control event rate (Pc) and event rate
pattern– Use of epidemiological data to obtain realistic estimates of
experimental group event rate (Pe)
4. Time to event designs and variable follow-up
Useful References
• Lachin JM, Cont Clin Trials, 2:93-113, 1981 (a general overview)
• Shih J, Cont Clin Trials, 16:395-407, 1995 (time to event studies with dropouts, dropins, and lag issues) – see size program on biostatistics network
• Farrington CP and Manning G, Stat Med, 9:1447-1454, 1990 (sample size for equivalence trials)
• Whitehead J, Stat Med, 12:2257-2271, 1993 (sample size for ordinal outcomes)
• Donner A, Amer J Epid, 114:906-914, 1981 (sample size for cluster randomized trials)
Key Points• Sample size should be specified in advance (often it is not)
• Sample size estimation requires collaboration and some time to do it right (not solely a statistical exercise)
• Often sample size is based on uncertain assumptions (estimates should consider a range of values for key parameters and the impact on power for small deviations in final assumptions should be considered)
• Parameters that do not involve the treatment difference (e.g., SD) on which sample size was based should be evaluated by protocol leaders (who are blinded to treatment differences) during the trial
• It pays to be conservative; however, ultimate size and duration of a study involves compromises, e.g., power, costs, timeliness.
Some Evidence that Sample Size is Not Considered Carefully: A Survey of 71
“Negative” Trials(Freiman et al., NEJM, 1978)
• Authors stated “no difference”• P-value > 0.10 (2-sided)• Success/failure endpoint• Expected number of events >5 in control and
experimental groups• Using the stated Type I error and control group event
rate, power was determined corresponding to:
– 25% difference between groups– 50% difference between groups
0-9 10-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-990
5
10
15
20
25
Power (1 - ß)
5.63%
25% Reduction
Frequency Distribution of Power Estimates for 71 “Negative” Trials
References: Frieman et al, NEJM 1978.
0-9 10-19 20-29 30-39 40-49 50-59 60-69 70-79 80-89 90-990
5
10
15
20
25
Power (1 - ß)
29.58%
Frequency Distribution of Power Estimates for 71 “Negative” Trials
50% Reduction
References: Frieman et al, NEJM 1978.
Implications of Review by Frieman et al.
• Many investigations do not estimate sample size in advance
• Many studies should never have been initiated; some were stopped too soon
• “Non-significant” difference does not mean there is not an important difference
• Design estimates (in Methods) are important to interpret study findings
• Confidence intervals should be used to summarize treatment differences
Studies with Power to Detect 25% and 50% Differences
6 6
6
6l
l
l
l
0
5
10
15
20
25
30
35
40
45
50
Per
cen
t o
f S
tud
ies
wit
h a
t L
east
80%
Po
wer
6 25% Differencel 50% Difference
1975 1980 1985 1990
Moher et al, JAMA , 272:122-124,1994
These Results Emphasize the Importance of Understanding that the Size of P-Value
Depends on:
• Magnitude of difference (strength of association); and
• Sample size
“Absence of evidence is not evidence of absence”, Altman and Bland, BMJ 1995; 311:485.
Steps in Planning a Study
1) Specify the precise research question
2) Define target population
3) Assess feasibility of studying question (compute sample size)
4) Decide how to recruit study participants, e.g., single center, multi-center, and make sure you have back-up plans
Beginning: A Protocol Stating Null and Alternative Hypotheses Along with Significance Level and
PowerNull hypothesis (HO)
Hypothesis of no difference or no association
Alternative hypothesis (HA)Hypothesis that there is a specified difference (Δ)
No direction specified (2-tailed)A direction specified (1-tailed)
Significance Level (): Type I Error
The probability of rejecting H0 given that H0 is true
Power = (1 - ): ( = Type II Error)
Power is the probability of rejecting H0 when the true difference is Δ
End: Test of Significance According to Protocol
Statistically Significant?
Yes No
RejectHO
Do not rejectHO
Sampling variationis an unlikely
explanation for thediscrepancy
Sampling variationis a likely
explanation for thediscrepancy
Normal Distribution
If Z is large (lies in yellow area), we assume difference in means is unlikely to have come from a distribution with mean zero.
Continuous Outcome Example
Observations: Many people have stage 1 (mild) hypertension (SBP 140-159 or DBP 90-99 mmHg)
For most, treatment is life-long
Many drugs which lower BP produce undesirable symptoms and metabolic effects (new drugs are
needed)
Research Can new drug T adequately control BP for patients Question: with mild hypertension?
Objective: To compare new drug T with diuretic treatment for lowering diastolic blood pressure (DBP)
Parallel Group Design Comparing Average Diastolic BP (DBP) After One Year
Hypothesis HO: DBP after one year of treatment with new drug T equals the DBP for patients given a diuretic (control)
HA: DBP after one year is different for patients given new drug T compared to diuretic
treatment (difference is 4 mmHg or more)
Study Population: Those with mild
hypertension
Drug T DBP at year 1
Diuretic DBP at year 1
Parallel Group Design Comparing Average Difference (Year 1 – Baseline) in DBP.
Hypothesis HO: DBP change from baseline after one year of treatment with new Drug T equals the DBP change from baseline after one year for patients given a diuretic (control)
HA: DBP change from baseline after one year of treatment with new Drug T is different
than the DBP change from baseline after one year for patients given a diuretic (control) treatment (difference is 4 mmHg or more)
Study Population: Those with mild
hypertension
Drug T Change in
DBP(Year 1 – Baseline)
DiureticChange in DBP
(Year 1 – Baseline)
Why Δ= 4 mmHg? An importantdifference on a population-wide basis
Clinical trials (Lancet 1990;335:827-38)
• 14 randomized trials; 36,908 participants
• 5-6 mmHg DBP difference (treatment vs. control)
• 28% reduction in fatal/non-fatal CVD
Observational studies (Lancet 2002;360:1903-13)
• 58 studies; 958,074 participants
• 5 mm Hg lower DBP among those 40-59 years
• 41% (30%) lower risk of death from stroke (CHD)
Considerations in Specifying Treatments Effect (Delta)
• Smallest difference of clinical significance/interest
• Stage of research
• Realistic and plausible estimates based on:– previous research
– expected non-compliance and switchover rates
– consideration of type of participants to be studied
• Resources (compromise)
Delta is a difference that is important NOT to miss if present.
Principal Determinants of Sample Size
• Size of difference considered important (Delta)
• Type I error () or significance level
• Type II error (), or power (1- )
• Variability of response/frequency of event
Constants
Sample Size for Two Groups: Equal Allocation
General Formula
2 x Variability x [Constant (,)]2
Delta2
N PerGroup
=
Delta = Δ = clinically relevant and plausible treatment difference
Sample Size Formula Derivation: One Sample Situation
;H
H
trueis H if 1)( Prob
and
trueis H if )( Prob
:satisfy tohas size Sample
:A
:o
A
o
o
o o
ZZ
ZZ
Sample Size Derivation (cont.)
2
211
2
1
21
A21
21 o
)(N
Nfor solve and :Note
N(0,1)
Prob
Hunder 1 0
Prob
sided)-(2 0.05For
96.1 ifHReject
0
ZZ
ZZ
Z
NZ
N
X
-Z
N
X
ZZN
XZ
0.05 (1.96) 0.80 (0.84) 7.84
0.90 (1.28) 10.50
0.95 (1.645) 13.00
0.01 (2.575) 0.80 (0.84) 11.67
0.90 (1.28) 14.86
0.95 (1.645) 17.81
Typical Values for (Z1-/2 + Z1- )2 Which Is Numerator of Sample Size
Type I Error ()or
Significance Level (Z1-/2)2-sided test
Power (1 - )(Z1-) (Z1-/2 + Z1-)2
Example
Hypertension Study
HO 1 2 1 2: = ; - = 0HA 1 2 1 2: ≠ ; - = 4 mmHg
HO HA
0 4 mmHg
Usually formulated in terms of change from baseline (e.g., Ho = D1 - D2 = 0)
Another Derivation
2
22
222
21
2
2
22
0
under 1Prob
2
under Prob
2
0
ZZN
ZZN
NZ
NZ
dd
HZZ
N
dZ
HZZ
Nnn
N
dZ
A
O
Solve for N using these2 equations and by noting
that Δ = sum of 2 partsfrom the previous figure .
Sources of Variability of BP Measurements
Ref: Rose GA. Standardization of Observers in Blood Pressure Measurement. Lancet 1965;1:673-4.
Variability ofblood pressurereadings
True variations in arterial pressure
Known factors
Unknown factors
Recent physical activityEmotional statePosition of subject and armRoom temperature and season of year
Measurementerrors
Instrument
Observer
Inaccuracy of sphygmomanometer
Cuff width and length
Chiefly affectingthe mean pressureestimate
Distorting the frequency distribution curve (and sometimes affecting the mean)
Mental concentrationHearing acuityConfusion of auditory and visualInterpretation of soundsRates of inflation and deflationReading of moving column
Terminal digit preference
Prejudice, e.g., excess of readings at 120/80
Estimates of Variability for Diastolic Blood Pressure Measurements (MRFIT)
Estimated Using Random-Zero (R-Z) Readings
EstimateVariance Component (mmHg)2
Between Subject 58.4
Within Subjects 36.3
s2
e2
Estimates of Variability for Diastolic Blood Pressure Measurements
Estimated Using Random-Zero (R-Z) Readingsat Screen 2 and Screen 3 in MRFIT
(2 Readings at Each Visit)
EstimateVariance Component (mmHg)2
Between Subject 58.4
Between Visits 26.1
Between Readings 10.2
s2
v2
e2
Within subjectanalyzed further
Consequences on Sample Size of Using Multiple Readings for Defining Diastolic BP
=0.05, 1-=0.90Inter-subject variability=58.4 (mmHg)2
No. of No. ofvisits readings/visit ∆ = 8 ∆ = 4
1 1 31 124
1 2 30 118
2 1 25 100
2 2 24 97
Between visit variability = 26.1 (mmHg)2
Within visit variability = 10.2 (mmHg)2
N per Group
Parallel Group Design Comparing Average DBP After One Year.
Hypothesis HO: DBP after one year of treatment with new Drug T equals the DBP for
patients given a diuretic (control)
HA: DBP after one year is different for patients given new Drug T compared to diuretic
treatment (difference is 4 mmHg or more)
Study Population: Those with mild
hypertension
Drug T DBP at year 1
Diuretic DBP at year 1
Parallel Group StudiesComparing Average DBP After One Year
1 measure, 1 visit (a=0.05, b= .10)
1253.124
4
5.10)7.94(2=n=n
zz2
=n=n=n
4;:H
=:H
2CT
2
2
-1
2-1
2
CT
TCTA
CTO
C
3207.31
8
5.10)7.94(2=n=n
zz2
=n=n=n
8;:H
=:H
2CT
2
2
-1
2-1
2
CT
TCTA
CTO
C
s2=58.4 + 26.1+10.2=94.7
D=4 mmHg D=8 mmHg
Parallel Group Design Comparing Average Difference (Year 1 – Baseline) in DBP.
Hypothesis HO: DBP change from baseline after one year of treatment with new Drug T equals the DBP change from baseline after one year for patients given a diuretic (control) HA: DBP change from baseline after one year of
treatment with new Drug T is different than the DBP change from baseline after one year for patients given a diuretic (control) treatment (difference is 4 mmHg or more)
(2-Tailed)
Study Population: Those with mild
hypertension
Drug T Change in DBP
(Year 1 – Baseline)
DiureticChange in DBP
(Year 1 – Baseline)
Sample Size for Two Groups: Equal Allocation
General Formula
2 x Variability x [Constant (,)]2
Delta2
N PerGroup
=
Delta = Δ = clinically relevant and plausible treatment difference
Estimate of Variability for Change Outcome
• Prior studies (For MRFIT, SD of DBP change after 12 months = 9.0 mmHg [baseline is one visit, 2 readings; follow-up is one visit, 2 readings]. For comparison, SD of 12 month DBP is 9.5 mmHg)
• Use correlation (ρ) of repeat readings for participants to estimate e
2. (For MRFIT, correlation of DBP at baseline
and 12 months is 0.55; note that SD (diff) can be written as
2σT2 (1-ρ) = 2σe
2 = 2(81)(1-0.55) = 72.9 (SD of change ≈ 8.5 mmHg)
• Estimate of SD change using analysis of covariance (regression of change on baseline) (For MRFIT, SD = 7.9 mmHg)
2
22
222
22
222
22
2
22
2
21
122
2
e
es
ees
es
sesBF
tttBF
tyy
tyyBF
BFBBF
es
F
B
yy
so
yy
ppyyc
yyyVaryrVyy
y
y
BF
BF
)())(2()-Var(
,
)(2=)- Var(and
if
)(ov
)cov()()(a=)- Var(
tmeasuremen up-follow
tmeasuremen baseline Let
2
F
2t
Study Population: Those with mild
hypertension
Drug T Washout Period Diuretic
Diuretic Washout Period Drug T
Crossover Group Design Comparing Average Difference (Diuretic – Drug T) in DBP
Hypothesis HO: Average of paired differences for the two treatment sequences differences is zero.
HA: Average is 4 mmHg or more)
Crossover Study Design
1 2 Diff.
I y1 y2 dl
II y1 y2 dll
Period
Var(dl) = 2 e2
Var(dll) = 2 e2
=∆ = TT - TC = E dl + dll
2
– –Dl + Dll
2
With parallel group comparison we had:
or equivalently:
With crossover we have:
HO = TT – TC = 0
HO : mT = mC or HO : DT = DC where
Dl + Dll
2HO = = 0
DT and DC refer to the difference between
follow-up and baseline levels of outcome
Substitution intoSample Size Formula Gives:
n| = n|| = number randomly allocated to each sequence - I (AB) or II (BA).
This follows because the variance of the pooled treatment difference across the 2 sequences is ¼ (22
e + 22 e)
2
2
-1
2-1
2e
IIIc
zzn=n=n
Crossover Sample Size Compared to Parallel Design (no baseline)
)+2(=
zz)+2(
zz
n
n
2e
2s
2e
-12-1
2e
2s
-12-1
2e
c
2
2
2
2
Crossover Sample Size Compared to Parallel Design (no baseline)
But the crossover design will require twice the number of measurements. So, if ρ= 0 then number of measurements are equal, but sample size for crossover is ½.
2
1
12
1
n)(n
+= since
)+2(n
n
+=
c
2e
2s
2e
2e
2s
2ec
2e
2s
2s
Consider an Experimentwith Diastolic BP Response
Type 1 error = 0.05 (2-sided) and Power = 0.95
99=n ,n 5,=
design group parallel
for needed patients more times 5
0.19 = )..(
.
n
n
(mmHg) 36.3 =
(mmHg) 58.4 =
c
c
22e
22s
19
3364582
336
Examples
DBP (mmHg) 58 36 0.62 0.19
Cholesterol (mg/dl) 1200 400 0.75 0.125
Overnight urine 325 625 0.34 0.33excretion Na+(meq/8 hours)
2 overnights 325 312 0.51 0.24
7 overnights 325 90 0.78 0.11
s2 e2
nc
n
2e
2e
2e
III
2eBA
2e
2sBA
2+24
1
2
d+dVar
consider toneed wecrossoverwith
4=)d-Var(d
or )+2(=)y-Var(y
consider to
need webaseline with scomparison group parallelWith
crossover to
compared baseline uses whichdesign group
parallel for required patients more times 4
is or whatof Regardless
4
1=
)zz)((
)zz(
baseline) withn(parallel
)(crossovern
-12-1
-12-1
c
2
2
22
2
22
4
e
e
e
Sample size for = .05 (2-sided) and = .05
ParallelNumber/group 163 72 41 26(no baseline)
ParallelBaseline 80 36 20 12number/group(r=0.75)
Crossover(Number/seq.) = 0.00 82 36 21 13 = 0.25 62 27 15 10 = 0.50 41 18 10 7 = 0.75 20 9 5 3
0.8 1.00.4 0.6
Key Points
• Sample size should be specified in advance
• Sample size estimation requires collaboration
• Often sample size is based on uncertain assumptions, therefore estimates should consider a range of values for key parameters (i.e., investigate the impact on power if sample size and treatment effect is not achieved)
• Parameters on which sample size is based should be evaluated during the trial
• It pays to be conservative; however, ultimate size and duration of a study involves compromises, e.g., power, costs, timeliness.
Power
Prob (rej Ho | when HA is true) = 1-b
1-b = (rej Ho | m1-m2 = D)
= Prob x 1 - x 2 - 0
s 12
n1
+ s 22
n2
³ Z1 -a 2
m1 -m2 = D
+ x 1- x 2 - 0
s12
n1
+s 2
2
n2
£ -Z1-a 2
m1 -m2 = D
Assume s 12 = s 2
2 = s 2 and n1=n2 =n
A measure of how likely the study will detecta specific treatment difference (∆), if present.
Power (cont.)
1-b = Prob x 1 - x 2 - D
2s 2
n
³ Z1-a 2
+0-D
2s 2
n
m1-m2 = D
é
ë
ê ê ê ê ê
ù
û
ú ú ú ú ú
+ Prob x 1 - x 2 -D2s 2
n
£ -Z1-a 2
+ 0 -D2s 2
n
m1-m2 = D
é
ë
ê ê ê ê ê
ù
û
ú ú ú ú ú
=Prob Z ³ Z1-a 2
-D
2s 2
n
é
ë
ê ê ê ê ê
ù
û
ú ú ú ú ú
+
Prob Z £ -Z1-a 2
- D2s 2
n
Usually one of these probabilities will be very close to zero, depending on whether ∆ is positive or negative.
Sensitivity of Power to Variations in Other Sample Size Parameters
(Assume s2 = 100)
0.05 4 100 -0.87 0.81
0.01 4 100 -0.25 0.60
0.05 6 100 -2.28 0.99
0.01 6 100 -1.67 0.95
0.05 4 200 -2.04 0.98
0.01 4 200 -1.425 0.92
a ∆ n Zc Power
Unequal Sample Sizes
%.k For
/)(2 1:k versus 1:1 for size sample Relative
)(Nc
)allocation 1:(kkNc Ν
)(.Nc
SE(diff)
)allocation 1:(2Nc Ν
T
T
5124542
41
11
51
2
11
2
2
22
2
2
/. k
k
ZZk
ZZ
NN cc
Another Formulation: Unequal Allocation
Comparison of means (Treatment C vs. E):
PC and PE= fraction of patients assigned control (C) and experimental treatment (E); PC+ PE = 1
Total N =
s 2 1P
C
+ 1P
E
æ
è ç ç
ö
ø÷ ÷ Z
1-a
2
+ Z1- b
æ
è ç ç
ö
ø÷ ÷
2
Δ2
Total Sample Size for Different Allocation Ratios
Allocation Ratio (E:C)
∆ (mm Hg) 1:1 2:1 3:1 1:2
4 250 280 332 280
8 64 70 84 70
0.90 β)-1(Power
test)sided-(2 05.
)Hg (mm 7.94 22
Sample sizes rounded up
Multiple Treatments and Unequal Allocation
Example: m experimental treatments and control; comparison of means
Problem: Find n which minimizes variance
mn-N
+n
=XXV
E(x)=
treatment
alexperiment each to response = X
treatment control to response = X
arm control on patients of no. = mm - N
treatment
alexperiment each on patients of no. = n
treatments alexperiment no. = m Let
2c
2e
ce
e
c
Solution: Take derivative with respect to n and set = zero
Then,
No. of patients in control group = no. of patients in experimental times square root no. of treatments
mn
mn
ce
e
c
=mn-N
if
mnN
Other Issues with Multiple Groups
• Multiple comparisons (a adjustment)
• Interim analyses – possible early termination of some, but not all, treatment groups
Minimum Clinically Important Difference (MCID)
For a given sample size (N) the null hypothesis (HO: difference in means = 0) will be rejected if the observed difference (d)
NZ
221 / is d
MCID of 61% is d if 0.05P 0.10, 0.05, For
)Z
( MCIDd then
(MCID))(
2N that so determined wasN If
12/1
2/1
2
212/12
ZZ
ZZ
Chuang-Stein C et al. Pharmaceutical Stat 2010.
d can be smaller than MCID and p<0.05!
Sample Size for Dual Criteria: Statistical Significance and Clinical Significance
• In some cases, you may want to establish with high probability that the treatment effect is as large as MCID
– For example, a new HIV vaccine might be assumed to have 60% efficacy but the study is designed to have sufficient power to rule out efficacy lower than 30%
– This will require a larger sample size
– For example, if Δ=2MCID, then sample size is 4 times greater
Summary (General)
• It is important that sample size be large enough to achieve the goals of the study – too many studies are conducted which are under-powered.
• Sample size assumptions are frequently very rough so they should be re-evaluated as the study progresses.
• A good knowledge of the subject matter (background on disease and intervention, outcomes, and target population) is necessary to estimate sample size.
Summary (Crossover versus Parallel Group)
• Efficiency of crossover increases as increases.
• Design using change from baseline as response is better than design which just uses follow-up responses if > 0.50.
• With multiple measurements on each patient, to establish baseline and follow-up levels, sample size can be reduced.