Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Prof. Dr. Karl Broich
EU regulatory concerns about missing data: Issues of interpretation and considerations for
addressing the problem
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 2
• No conflicts of interest
• Views expressed in this presentation are the author's personal views and not necessarily the views of BfArM or EMA
Disclaimer
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 3
• Current regulatory basis
• Sources of missing data
• Missing data and trial validity
• Example: Simulation of a depression trial
• Interpretational issues
• Estimands
• Addressing the problem
• Sensitivity analyses
• Conclusions
Agenda
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 4
• Relevant regulatory documents
• ICH E9 Guideline Statistical Principles for Clinical Trials (1998)
• EMA Guideline on Missing Data in Confirmatory Clinical Trials (2010)
• National Research Council. The Prevention and Treatment of Missing Data in Clinical Trials. Panel on Handling Missing Data in Clinical Trials (2010)
• ICH Concept Paper E9(R1): Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials (2014)
Regulatory discussion on missing data in confirmatory Phase III studies (1)
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 5
• Issues to discuss:
• How to avoid missing data?
• How to consider/address (non-)adherence?
• measuring efficacy assuming
• perfect adherence or
• real adherence or
• in those patients who tolerate treatment
• How to treat missing data?
• how to treat missing data w.r.t. adherence?
• and how to interpret analysis/missing data imputation
Regulatory discussion on missing data in confirmatory Phase III studies (2)
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 6
• Treatment discontinuation
• due to
• adverse events
• lack of efficacy
• others
• often leads to missing data
• follow-up of discontinuing patients avoids missingness
• Study drop-out
• treatment discontinuation and no follow-up
• Intermediate missing data
• usually less relevant
Sources of missing data in confirmatory Phase III studies
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 7
• Substantial amount of drop-outs in
• depression
• drop-out may be up to 50%
• average drop-out rate 20 to 30%
• others: schizophrenia, Alzheimer
• Follow-up
• usually poor follow-up of patients that discontinue treatment
• lack of information on non-adherent patients
• but: estimation of “de-facto” efficacy would require
• follow-up of patients that discontinue treatment
• targeting a “treatment-policy estimand”• treatment benefit in all subjects irrespective of treatment
adherence
Missing data in CNS trials
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 8
• Analysis in completers only• not compliant with ITT principle• treatment dependent patient selection• biased effect estimates and lack of type-1 error control
• invalid conclusions (e.g. false positive decisions ↑)• Missing data imputation
• based on specific assumptions regarding (unknown) missing data• requires definition of a relevant estimation target (estimand), e.g.
• treatment benefit if all patients adheredor
• treatment benefit in all patients regardless of adherenceor
• treatment benefit attributable to the randomized treatment or
• treatment benefit in those who adhere to treatment
Regulatory concerns about missing data in confirmatory clinical trials (1)
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 9
• Missing data imputation
• potential concerns about underlying assumptions and resulting validity
• e.g. LOCF usually invalid in progressive diseases (e.g. dementia)
• potential concerns about target of estimation
• e.g. longitudinal models may target treatment benefit if all patients adhered to treatment• hypothetical target that appears less relevant
• usually several sensitivity analyses required
• to show robustness of the results w.r.t. to underlying assumptions
• to evaluate different estimands
Regulatory concerns about missing data in confirmatory clinical trials (2)
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 10
“De-facto“ and “de-jure“ estimands
time
placebo
active treatment
end of trial
de-facto
(difference in all
randomized patients)
treatment dropout“retrieved” data
de-jure
(difference
if all patients
adhered)
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 11
Example: Simulation of depression trials BfArM research project on missing data and non-adherence
• Longitudinal data (Hamilton Score)
• Non-adherence: Treatment discontinuation
• Some data were collected after treatment discontinuation
• Different drop-out mechanisms
• treatment dropout (TD)
• study dropout (SD)
• SD time ≥ TD time
• “retrieved data” from TD to SD
• Data generation
• according to a two-piece linear mixed model
Leuchs et al (2014). Statistics in Medicine 33
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 12
Example: Simulation of depression trials BfArM research project on missing data and non-adherence
true de-jure effect = 2
(difference if all subjects adhered)
true de-facto (treatment policy)
effect = 0 (difference in all subjects)
Analysis strategies
• 1: Multiple Imputation (Pattern-
Mixture Model)
• 2: Joint Model of drop-out and
outcome
• 3: Mixed Model, all data
• 4: Mixed Model, only data
under treatment
Bias of different analysis strategies for de-jure and de-facto estimands
Leuchs et al (2014). Statistics in Medicine 33Proportion of subjects followed-up
Bia
s f
or
de-f
acto
eff
ect
100% 70% 40% 100% 70% 40%
equal drop-out 30%
unequal drop-out 25% und 35%
Bia
s f
or
de-j
ure
eff
ect
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 13
Example: Simulation of depression trials Conclusions
• Longitudinal Mixed Model analysis of on-treatment data
• targets de-jure estimand
• Longitudinal Mixed Model analysis of all data (off- and on-treatment)
• still shows relevant bias w.r.t. de-facto (treatment policy) estimandif follow-up is poor
• “Joint model” of outcome and time to drop-out
• behaves best
• but would require further investigation on robustness
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 14
Proposed procedure
Primary estimand
Clinical trial design
Analysis method
Sensitivity analyses
Be clear about the trial‘s objective (i.e. primary estimand) before
deciding trial design and analysis
Select a number of different sensitivity analyses
Sensitivity analyses
Customize the design considering the primary estimand
Clinical trial design
Choose a primary analysis applicable for the chosen design and addressing the primary estimand
Analysis method
Leuchs et al (2015). Therapeutic Innovation & Regulatory Science 49.
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 15
Regulatory conclusions on missing data and estimands (1)
Which estimand addresses best clinical relevance ?
• “Treatment policy” estimand
• most likely targets clinical relevance for a given population
• Treatment effect in tolerators
• may be relevant for patients
• but require complex causal inference and assumptions for a valid conclusion (without active run-in)
• “De-jure” like estimands (if all patients adhered)
• are hypothetical parameters difficult to justify
• but: may be most sensitive for non-inferiority conclusions
• Many other options to be discussed
• e.g. composite of different estimands related to reasons for drop-out
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 16
Regulatory conclusions on missing data and estimands (2)
“Treatment policy” estimand
• fails if no or only few “de-facto” (retrieved) data are available
• requiring unverifiable assumptions
• difference between de-facto and de-jure can hardly be substantiated without data
• strong de-facto conclusions require de-facto data
• patient follow-up after drop-out needed
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 17
Sensitivity analyses
… to assess the robustness of trial results!
Robustness of the estimation method Robustness of the estimand Robustness with regard to
generalizability of trial results
Internal validity external validity
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 18
Conclusions
• Missing data highly relevant issue in depression trials
• interpretational issues related to missing data
• Primary estimand to be agreed upon first
• design and analyse accordingly
• Sensitivity analyses relevant to address
• internal validity (concerning underlying assumptions)
• external validity (concerning clinical relevance addressed by different estimands)
• Treatment policy or attributable estimand relevant for population based conclusions
• treatment policy estimand require follow-up of (most) patients
• lack of follow-up result in the need for unverifiable assumptions
K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 19
References• ICH Expert Working Group (1999). Statistical principles for clinical trials (ICH E9). Statistics in
Medicine, 18: 1905-1942.
• EMA (2010). Guideline on Missing Data in Confirmatory Clinical Trials.
• National Research Council of the National Academies (2010). The Prevention and Treatment of Missing Data in Clinical Trials. Washington, D.C.: National Academies Press.
• Mallinckrodt CH et al (2012). A structured approach to choosing estimands and estimators in longitudinal clinical trials. Pharmaceut Statist, 11:456–461.
• O’Neill RT and Temple R (2012). The prevention and treatment of missing data in clinical trials: an FDA perspective on the importance of dealing with it. Clin Pharmacol Ther, 91: 550-554.
• Leuchs AK, Zinserling J, Schlosser-Weber G, Berres M, Neuhäuser M, Benda N (2014). Estimation of the treatment effect in the presence of non-compliance and missing data. Statistics in Medicine, 32:193–208.
• ICH concept paper (2014) E9(R1): Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials
• Leuchs AK, Zinserling J, Brandt A, Wirtz D, Benda N (2015). Choosing appropriate estimands in clinical trials. Therapeutic Innovation & Regulatory Science, 49:584-592.
• Leuchs AK, Brandt A, Zinserling J, Benda N (2016). Disentangling estimands and the intention-to-treat principle. Pharmaceut Statist (accepted for publication).