Upload
hilda-watkins
View
230
Download
0
Tags:
Embed Size (px)
Citation preview
1
Chapter 10Chapter 10
Data Monitoring, Monitoring Data Monitoring, Monitoring Committee Function & Committee Function &
Statistical MethodsStatistical Methods
2
Some ReferencesSome References• Texts/Chapters
1. Friedman, Furberg & DeMets (1998) 3rd edition,Fundamentals of Clinical Trials, Springer-Verlag, NY, NY
2. Pocock (1983) Clinical Trials, Wiley.
3. Ellenberg S, Fleming T and DeMets D: Data Monitoring Committees in Clinical Trials: A Practical Perspective. John Wiley & Sons, Ltd., West Sussex, England, 2002.
4. Jennison C and Turnbull B (2000) Group Sequential Methods with Application to Cinical Trials. Chapman & Hall, NY.
5. DeMets DL (1998) Data and Safety Monitoring Boards. In: Encyclopedia of Biostatistics. John Wiley and Sons, West Sussex, England, Vol. 2, pp. 1067-71.
6. DeMets and Lan. The alpha spending function approach to interim data analysis. In, Recent Advances in Clinical Trials Design and Analysis. Kluwer Academic Publishers, Boston, MA, 1995.
3
Some ReferencesSome References• Review Papers
1. Greenberg Report:Organization, review, and administration ofcooperative studies. Controlled Clinical Trials 9:137-148, 1988.
2. DeMets and Lan: (1994) Interim analyses: The alpha spendingfunction approach. Statistics in Medicine, 13(13/14):1341-52, 1994.
3. Lan and Wittes. The B-value: A tool for monitoring data. Biometrics 44:579-585, 1988.
4. Task Force of the Working Group on Arrhythmias of the European Society of Cardiology: The early termination of clinical trials:
causes, consequences, and control. Circulation 89(6):2892-2907, 1994.
5. Fleming and DeMets: Monitoring of clinical trials: issues and recommendations. Controlled Clin Trials 14:183-97, 1993.
6. DeMets, Ellenberg, Fleming, Childress, et al: The Data and Safety Monitoring Board and AIDS clinical trials. Controlled Clin Trials 16:408-21, 1995.
7. Armstrong and Furberg: Clinical trial data and safety monitoring boards: The search for a constitution. Circulation 1, Sess:6, 1994.
4
Data MonitoringData Monitoring
RationaleRationale
1. Ethical
2. Scientific
3. Economic
5
A Brief HistoryA Brief History
• A 40-year history
• Greenberg Report (1967)
• Coronary Drug Project (1968)
• NIH Experience and Guidelines
• Industry and ICH Guidelines
• Department of Health & Human Services Policy (Shalala, 2000)
6
Greenberg Report Greenberg Report Recommendations Recommendations
• Develop a mechanism to terminate early if– Question already answered
– Trial can’t achieve its goals
– Unusual circumstances
– Hypothesis no longer relevant
• Sponsor decision to terminate should be based on advice of external committee
7
Coronary Drug Project (CDP)Coronary Drug Project (CDP)
• References– Design (Circulation, 1973)
– Monitoring Experience (CCT, 1981)
– Major Outcome (JAMA 1970, 1972, 1973, 1975)
• Tested several lipid lowering drugs in post MI patients
• Multicenter study• Mortality as primary outcome• Began recruitment in 1965
8
Coronary Drug ProjectCoronary Drug Project
• First trial to benefit from Greenberg Report
• Policy Advisory Board– Senior Investigators, External Experts, NIH– Initially reviewed interim data
• Data Coordinating and Statistical Center
• Safety Monitoring Committee formed (1968), after trial was underway
9
Early NHLBI CT ModelEarly NHLBI CT Model
Policy Advisory
Board
Data and SafetyMonitoring
Board
Central Lab(s)Data Coordinating Center
Data ManagementStatistical Analysis
MultipleClinics
Working Committees
Steering Committee
Funding Agency
10
NHLBI CT ModelNHLBI CT Model
Data Monitoring Committee
Central Lab(s)
CoordinatingData Center
Clinics Working Committees
Steering Committee
Funding Agency
11
NIH DMC ActivityNIH DMC Activity• Ref: Statistics in Medicine (1993)
• CDP (Coronary Drug Project) became model for National Heart, Lung, and Blood Institute (NHLBI)– heart, lung, blood disease trials
• National Eye Institute (NEI) (1972)– Diabetic Retinopathy Study
• National Institute Diabetes, Digestive and Kidney (NIDDK)– Diabetes Complication and Control Trial (1980)
• National Cancer Institute (NCI)– Prevention Trials, Cooperative Group Therapeutic Trials
• National Institute Allergy and Infectious Disease (NIAID)– AIDS Clinical Trial Group (ACTG) (1986)
12
Industry/FDA/ICHIndustry/FDA/ICH
• Industry sponsorship of RCTs expanded dramatically since 1990 in several disease areas (e.g. cardiology, cancer, AIDS)
• Industry use of DMCs growing as well
• FDA 1989 guidelines very brief mention of data monitoring and DMCs
• International Conference on Harmonization (ICH)– ICH/E9
Section 4.5 Interim Analyses Section 4.6 Independent DMCs
– ICH/E6
13
Independent DMCsIndependent DMCsWhen are they Needed?When are they Needed?
• Department of Health and Human Services Policy– Shalala (NEJM, 2000): All NIH FDA trials must
have a monitoring plan, for some a DMC may be required
• NIH policy (1998)– all sponsored trials must have a monitoring
system– safety, efficacy and validity– DMC for Phase III trials
• FDA guidelines (Nov 2001)
14
Need for Independent DMCsNeed for Independent DMCs• Phase I Trials (dose)
– Monitoring usually at local level
• Phase II Trials (activity)– Most monitoring at local level– Some randomized, blinded, multicenter Phase II
trials may need IDMC
• Phase III & IV (effectiveness, risk, benefit)– Most frequent user of IDMC
• Structure of monitoring depends on risk (e.g. Phase I-IV)
15
Data Monitoring CommitteeData Monitoring Committee
• FDA suggests a need for an
Independent DSMB for
– Pivotal Phase IIIs
– Mortality or irreversible
morbidity outcome
Central Units (Labs, …)
Clinical Centers
Patients
Data Management Center
(Sponsor or CRO)
Pharmaceutical Industry Sponsor
Institutional Review Board
Independent Data Monitoring
Committee (IDMC)
Steering Committee
Statistical Analysis Center
Regulatory Agencies
Industry-Modified NIH ModelIndustry-Modified NIH Model
17
DMC RelationshipsDMC Relationshipsand Responsibilitiesand Responsibilities
• Patients
• Study Investigators
• Sponsor
• Local IRBs
• Regulatory Agencies
18
Early Administrative AnalysisEarly Administrative AnalysisDMC and Executive CommitteeDMC and Executive Committee
1. Recruitment/Entry Criteria
2. Baseline Comparisons
3. Design Assumptions
a. Control only
b. Combined groups
19
Design ModificationsDesign Modifications
1. Entry Criteria
2. Treatment Dose
3. Sample Size Adjustment
4. Frequency of Measurements
20
DMC Data ReviewDMC Data ReviewInterim AnalysisInterim Analysis1. Recruitment
2. Baseline Variables
-Eligibility
-Comparability
3. Outcome Measures
-Primary
-Secondary
4. Toxicity/Adverse Effects
5. Compliance
6. Specified Subgroups
21
DMC RecommendationsDMC Recommendations
1. Continue Trial / Protocol Unmodified
2. Modify Protocol
3. Terminate Trial
22
Reasons for Early TerminationReasons for Early Termination
1. Serious toxicity
2. Established benefit
3. Futility or no trend of interest
4. Design, logistical issues tooserious to fix
23
DMC Decision Making DMC Decision Making Process Complex (1)Process Complex (1)
• Recruitment Goals
• Baseline risk and comparability
• Compliance
• Primary and secondary outcomes
• Safety
24
DMC Decision Making DMC Decision Making Process Complex (2)Process Complex (2)
• Internal consistency
• External consistency
• Benefit/Risk
• Current vs future patients
• Clinical/Public impact
• Statistical issues
25
DMC Decision Making RoleDMC Decision Making Role• DMC makes recommendations, not final decisions
• Independent review provides basis for DMC recommendations
• DMC makes recommendations to
– Executive Committee who recommends to sponsor, or
– Sponsor
• DMC may, if requested, debrief Executive Committee and/or sponsor
• Rarely are DMC recommendations rejected
26
DMC Meeting FormatDMC Meeting Format• Open Session
– Progress, blinded data
– Sponsor, Executive Committee, DMC, SAC
• Closed Session– Unblinded data
– DMC, SAC
– Sponsor Rep? (Not recommended)
• Executive Session – DMC only
• Debriefing Session– DMC Chair, Sponsor Rep, Executive
Committee Rep
27
DMC RelationshipsDMC Relationships• Regulatory Agencies (e.g. FDA)
– Could perhaps brief DMC about specific concerns at Open Session
– Should not participate in DMC Closed Sessions
– Should be briefed about DMC recommendations/decisions ASAP following Executive Committee
28
DMC MembershipDMC Membership• Monitoring is complex decision process
and requires a variety of expertise• Needed expertise
– Clinical– Basic science– Clinical trial methodology– Biostatistics– Epidemiology– Medical ethics
• Helpful expertise– Regulatory
• Some experience essential
29
DMC ConfidentialityDMC Confidentiality• In general, interim data must remain
confidential– DMC may rarely release specific/limited interim
data (e.g. safety issue)
• Members must not share interim data with anyone outside DMC
• Leaks can affect– Patient Recruitment– Protocol Compliance– Outcome Assessment– Trial Support
30
DMC LiabilityDMC Liability
• Recent events (eg Cox-IIs, Vioxx) have raised the potential for litigation (訴訟 )(Vioxx or COX-IIs (painkillers) can raises the risk of heart attack, stroke and death and were withdrawn from the market)
• Members have been gotten a subpoena (傳票 )
• DMC Charters (設立 ) for industry trials now often cover indemnification clauses (賠償條款 )
• No indemnification yet for NIH trials
31
DMC Needs “On-Line”DMC Needs “On-Line”Data Management and AnalysisData Management and Analysis
• DMC reluctant to make decisions on “old data”
• Minimize data delay and event verification
• Be prepared from start
• Focus on key variables, not complete case reports (delays can be problematic)
32
Levels of IndependenceLevels of Independence
• Totally Inhouse Coordinating Center
• Internal DM, Internal SAC, External DMC
• Internal DM, External SAC, External DMC
• External DM(e.g. CRO), External SAC, External DMC
33
DMC SummaryDMC Summary• NIH Clinical Trial Model - long history
of success
• Adaptation for industry can be made
• SC, DMC, SAC or DM are critical components
• Independence of DMC essential
• Best way to achieve this goal is for external SAC and external DMC
34
Data Monitoring ProcessData Monitoring Process1. DMC and the decision process
2. A brief introduction to statistical monitoring methods
a. Group Sequential
b. Stochastic Curtailment
3. Examples
Ref: BHAT, DeMets et al. Controlled Clin Trials,1984
35
Decision FactorsDecision Factors1. Comparability
2. Bias
3. Compliance
4. Main effect vs. Potential side effects
5. Internal Consistency
a. Outcome measures
b. Subgroups
c. Centers
6. External Consistency
7. Impact
8. Statistical Issues/Repeated Testing
36
Beta-blocker Heart Attack Trial Beta-blocker Heart Attack Trial (BHAT)(BHAT)
Preliminary Report. JAMA 246:2073-2074, 1981
Final Report. JAMA 247:1707-1714, 1982
Design Features
Mortality Outcome 3,837 patients
Randomized Men and women
Double-blind 30-69 years of age
Placebo-controlled 5-21 days post-M.I.
Extended follow-up Propranolol-180 or 240 mg/day
37
BHATBHATAccumulating Survival DataAccumulating Survival Data
Date Data Monitoring
Committee Meeting Propranolol Placebo Z(log rank)
May 1979 22/860 34/848 1.68
Oct 1979 29/1080 48/1080 2.24
March 1980 50/1490 76/1486 2.37
Oct 1980 74/1846 103/1841 2.30
April 1981 106/1916 141/1921 2.34
Oct 1981 135/1916 183/1921 2.82*
June 1982
* Data Monitoring Committee recommended termination
38
Beta-Blocker Heart Attack TrialBeta-Blocker Heart Attack Trial October 1, 1981October 1, 1981LIFE-TABLE CUMULATIVE MORALITY CURVESLIFE-TABLE CUMULATIVE MORALITY CURVES
39
Beta-Blocker Heart Attack TrialBeta-Blocker Heart Attack TrialBaseline ComparisonsBaseline Comparisons
Propranolol Placebo
(N=1,916) (N=1,921)Average Age (yrs.) 55.2 55.4
Male (%) 83.8 85.2
White (%) 89.3 88.4
Systolic B.P. 112.3 111.7
Diastolic B.P. 72.6 72.3
Heart rate 76.2 75.7
Cholesterol 212.7 213.6
Current smoker (%) 57.3 56.8
40
Beta-Blocker Heart Attack TrialBeta-Blocker Heart Attack TrialTotal MortalityTotal Mortality
(Average 24-Month Follow-Up)(Average 24-Month Follow-Up)
Propranolol Placebo
Age 30-59 5.9% 7.1%
60-69 9.6% 14.4%
Sex Male 7.0% 9.3%
Female 7.1% 10.9%
Race White 6.7% 9.0%
Black 11.0% 15.2%
41
Beta-Blocker Heart Attack TrialBeta-Blocker Heart Attack TrialTotal MortalityTotal Mortality
(Average 24-Month Follow-Up)(Average 24-Month Follow-Up)
Propranolol Placebo
Risk Group I 13.5% 16.9%
Risk Group II 7.8% 11.4%
Risk Group III 5.2% 7.1%
42
DMC Interim AnalysisDMC Interim Analysis
• Ethical, scientific and financial reasons
• Repeated analysis of accumulating data causes a statistical problem
43
Data MonitoringData Monitoring
44
Classical Sequential AnalysisClassical Sequential Analysis
• Observations are taken sequentially
• After each observation – Decide whether to stop sampling (one group
is significantly better, or worse, than the other)
– Or take another observation
• Originally developed by Wald (1947)
• Applied to the clinical trial by Armitage (1975)
45
Why Sequential Analysis? Why Sequential Analysis? (Armitage, 1975)(Armitage, 1975)
• Data reduction
• Estimation with desired precision
• Medical ethics
46
Repeated Significance TestsRepeated Significance Tests
• Assume X 1 , X 2 , ~ N(, 1)
• Let S n = X 1 + +X n
• N is the maximum sample size
• Testing H 0 : = 0 vs H A : 0
• Nominal significance level is 0.05
47
Repeated Significance TestsRepeated Significance Tests
• For each n N , we assess if | Sn |
1.96 n
– Stop sampling and reject H0 at the first n
N , if any, such that | Sn | 1.96 n
– Otherwise, stop sampling at N and do not
reject H0
48
Probability of Type I ErrorProbability of Type I Error
*N = P {| Sn | 1.96 n for some n N |
H0 }
• By the law of the iterated logarithm,
eventually reject H0 when in fact it is true
*N might be large for some N
49
The Type I Error Probability when the Maximum Number of Observations is N
N *N
1 0.050 2 0.083 3 0.107 4 0.126 5 0.142
10 0.193 15 0.225 20 0.248 25 0.266 30 0.280
50
The Required Critical Values and Nominal Level Giving a Type I Error Probability 0.05 for Various Values of N
N Critical Value Nominal Level
1 1.96 0.050
5 2.42 0.015
10 2.56 0.010
15 2.64 0.008
20 2.68 0.007
50 2.80 0.005
100 2.88 0.004
200 2.96 0.003
51
Group Sequential ProceduresGroup Sequential Procedures
• Repeated significance tests after every observation are not easy to conduct
• Apply the significance test at longer intervals
• Compute summary statistic at each interim analyses, based on additional group of new subjects (events)
• Compare statistic to a conservative critical value such that α=0.05 overall
52
Group Sequential ProceduresGroup Sequential Procedures
Boundaries
• Haybittle-Peto (1971,1976)
• Pocock (1977)
• O’Brien-Fleming (1979)
• Lan-DeMets (1983)
• Slud-Wei (1982)
53
54
Group Sequential BoundariesGroup Sequential Boundaries
55
Pocock's boundary
N = 0.05 = 0.01
1 1.96 2.58
2 2.18 2.77
3 2.29 2.87
4 2.36 2.94
5 2.41 2.99
56
Lan-DeMets ProcedureLan-DeMets Procedure
Criticism of “classical” group
Sequential procedure
• Number of interim analyses must be specified in advance
• Equal increments
57
Lan-DeMets ProcedureLan-DeMets Procedure• Specify *(t) spending function
• *(t) defines rate at which Type I error is spent where t is the proportion of information accumulated by calendar time tc
• 0 t 1
• *(t) increasing,
*(0) = 0
*(1) =
58
Lan & DeMets ProcedureLan & DeMets Procedure
• The function * is “arbitrary”
• Examples:
where z/2 is denoted such that (z/2) = 1- ,
and 2*(t) = log{1 + (e - 1)t}
,10 if22
0 if0
2
*1 ttz
tt
2
59
Information and Calendar TimeInformation and Calendar Time
t =proportion of information accumulated by tc
Example: Immediate Response
X1,X2,...,Xn,...,XN
Y1,Y2,...,Yn,...,YN
|tc
t = 2n / (2N) = n / N
60
Information and Calendar TimeInformation and Calendar Time
Example: Failure time (e.g., logrank)
c
c
c
c
T
t
T
tt
dead patientsnumber
dead patientsnumber
]dies Pr[patient
] dies Pr[patient
61
Lan & DeMets ProcedureLan & DeMets Procedure• Assume X1 , X2 , . . . ~ N( , 1)
• Testing H0 : 0 vs H1 : > 0
• Let Zi be the accumulated test statistic at calander time i at which the information time is ti .
• Find boundary values Ci such that
P ( Z1 C1 ) = *( t1 ),
P ( Z2 < C1 , Z2 C2 ) = *( t2 ) - *( t1 ), . . . .
62
Boundary Crossing ProbabilityBoundary Crossing ProbabilityE.g., K = 5, = 0.025
Upper Boundary C1 C2 C3 C4 C5
Pocock (2.41, 2.41, 2.41, 2.41, 2.41)OBF (4.56, 3.23, 2.63, 2.28, 2.04)
Pocock OBF1. P {Z1 > C1} = 0.0079 (0.000)
2. P {Z1 > C1 or Z2 > C2} = 0.0079 + 0.0059 = 0.0138 (0.0006)
3. P {Z1 > C1 or Z2 > C2 or Z3 > C3} = 0.0138 + 0.0045 = 0.0183 (0.0045)
4. P {Z1 > C1, ..., Z4 > C4} = 0.0183 + 0.0036 = 0.0219 (0.0128)
5. P {Z1 > C1, ..., Z5 > C5} = 0.0219 + 0.0031 = 0.0250 (0.0250)
63
64
* (t2) - * (t1)
65
Approximates
1. OBF
2. 2 *(t) = ln { 1 + (e - 1)t } Pocock
3. 3 *(t) = t
•Comparison of Boundaries( = .025, N = 5)
Values C1 C2 C3 C4 C5
1. OBF 4.56 3.23 2.63 2.28 2.04 1
*(t) 4.90 3.35 2.68 2.29 2.03
2. Pocock 2.41 2.41 2.41 2.41 2.41 2
*(t) 2.44 2.43 2.41 2.40 2.38
3. 3*(t) 2.58 2.49 2.41 2.34 2.28
Examples of *(t)
]/[)(* tZt 2 221
66
BHAT GSB
67
Cardiac Arrhythmia Cardiac Arrhythmia Suppression Trial (CAST)Suppression Trial (CAST)
• Ref: NEJM 321(6):406-12, 1989
• Cardiac arrhythmias associated with increased risk of sudden death
• New class of drugs (eg, encainide, flecanide) suppressed arrhythmias
• CAST designed to test effect on sudden death
68
CAST GSBCAST GSB• spending function approach
• *(t) = ½ t t < 1
t = 1
• for benefit = 0.025
• Used symmetric = 0.025 boundary for harm
69
CAST Interim DataCAST Interim DataSudden DeathSudden Death
Time Placebo Drug LogRank ZL ZU
9/01/88 5/576 22/571 -2.82 -3.18 3.01
3/30/89 9/725 33/730 -3.22 -3.04 2.71
Initially expected 100 events/arm
70
CAST Sequential BoundariesCAST Sequential Boundaries
71
Stochastic Curtailed SamplingStochastic Curtailed Sampling
• During study, whether the current trend in the data can lead to the acceptance or rejection of H0 ?
• Group sequential methods focus on existing data
• Curtailed sampling in addition considers the data which have not yet been observed
• Lan, Simon and Halperin (1982)
72
ExampleExample• H0 : = 0.5 (Prob(Heads)) vs. HA : 0.5
• Flip coin 400 times
• S=total number of heads
• Reject H0 if |z| 1.96, where
or when |S - 200| 20
After 350 coin flips and 220 heads, we know for sure we will reject H0 .
5.05.0400
200
S
Z
73
Stochastic CurtailingStochastic Curtailing
• Let Z(T)=statistic at end of trial
Z(t)=current value at time t
R=rejection region
• P [ Z(T) R| H0 ] =
• P [ Z(T) | HA ] =
or P [Z(T) R| HA ] = 1 -
R
74
Stochastic CurtailingStochastic Curtailing
• Lan, Simon, Halperin (1982)
reject when P [Z(T) R| H0 , Z(t)] = 0
shows very positive trend
accept when P [Z(T) | HA , Z(t)] = A
shows negative trend
• P [Type I error] / 0
• P [Type II error] / A
R
75
ExampleExample• Population 1: X ~ N(x , 2 )
Population 2: Y ~ N(y , 2 )
H0 : x = y
HA : x > y
• For design
HA : x - y = 0.1, 2 = 1, = 0.05, 1- = 0.8
Need 1250 subjects per group
Reject if Z 1.645 125011250112501250
YX
Z
76
Example (continued)Example (continued)• During Study
No Z
I 250 0.113 1.26
II 500 0.125 1.98
III 750 0.122 2.26
IV 1000 0.12 2.68
V 1250 ? ?
YX
77
Example (continued)
• Conditional Probability
= 0.12 P 0.999
= 0.03 P 0.98
= 0.00 P 0.95
?,12.0|645.1 yxyxZP
78
B-Value:B-Value:A Method for Computing Conditional PowerA Method for Computing Conditional Power
Lan & Wittes (1988) Lan & Wittes (1988) BiometricsBiometrics
Let t = n/N (or d/D)
Z(t) = current standardized statistic
• Now Z(1) = B(1) and
(= observed + remaining)
))()1(()(Z(1) ttZZttZ
ttZtB )()(
,θtE(B(t))
79
H0: = 0HA: e.g. =
B(1)
Visual AidVisual Aid
B(t)
80
• P[Z(1) Z | Z(t), )]
Conditional PowerConditional Power
)1(/)1(t)( 1 tttZZ
81
Conditional Power Conditional Power
))1t(Z(E 1. Survival D = total events
2. Binomial N = total sample size
)/(Ln4/D TC
qp
NPP
Nqp
PP TCTC
4/)(
)/2/( 2
qp
NPPTC
)(/
21
82
Conditional Power (2)Conditional Power (2)
3. Means N = total sample size
4/NTC
NTC
21/
83
Example: BHATExample: BHAT• Expected Deaths D = 398
• Observed Deaths 183 Placebo d = 318
135 Propranolol D = d + 80 = 398
• Observed logrankZd = 2.82 t = 318/398 = .80
• Compute Conditional Power under H0
= 1 - {- 1.25}
= 0.89
0 with - 1Power Cond H0
2
522961
.
..:
2.52 0.82.82B(.8)