Upload
duongkien
View
215
Download
0
Embed Size (px)
Citation preview
1
Estimands
Chrissie Fletcher (one of the EFPIA representatives on the ICH E9 Working Group) EFSPI Statistics Leaders 2015
International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use
Disclaimer (Chrissie Fletcher)
• The views expressed herein represent those of the
presenter and do not necessarily represent the
views or practices of Amgen, the views of the other
Industry representatives on the ICH E9 working
group, or the views of the general Pharmaceutical
Industry.
2
3
Agenda
• Introduce the ICH E9 working group
• Estimand definition and framework
• Example
• Case study exercise – hypothetical diabetes trial
• Questions and feedback
4
E9 WG - goals • Develop an addendum (Revision 1) for E9
• Acknowledge this is a new topic and new framework for improved clinical trial planning, conduct, analysis and interpretation. Not only a missing data problem.
• ‘Estimand’ o Need to clarify what measure of treatment effect is being estimated in
a clinical trial. Failure to do so results in inconsistencies in conduct and analysis and confusion in interpretation.
• ‘Sensitivity analysis’ o Current practice may lead to misaligned and uninformative analyses
and confusion for decision makers. Build on existing E9 principles to introduce an improved framework.
5
E9 WG Objectives
• Improved framework for clinical trial planning,
conduct, analysis and interpretation.
• Trial Objective
↓
• (Consequent) Estimand
↓
• (Choice of) Analysis methodology
↓
• (Consequent) Sensitivity Analyses
• Current practice is often not aligned with this
proposed framework.
6
E9 WG - membership • Rapporteur: Rob Hemmings, EU
• Regulatory chair: Estelle Russek-Cohen, FDA
• Represented: EU, EFPIA, MHLW/PMDA, JPMA, FDA,
PhRMA, HC, DoH China, DRA Australia, DRA Brazil o EU: Rob Hemmings (Rappourteur), Frank Petavy
o FDA: Estelle Russek-Cohen (Regulatory Chair), Tom Permutt
o EFPIA: Chrissie Fletcher (Amgen), Frank Bretz (Novartis)
o MHLW/PMDA: Yuki Ando, Hirofumi Minami
o JPMA: Satoru Tsuchiya, Satoru Fukinbara
o PhRMA: Devan Mehrotra, Vladamir Dragalin
o HC: Catherine Njue
• First meeting Nov 2014
7
(
7
Completion Date Deliverable
November 2014 to June 2015 Work towards Step 1 Technical Document:
ICH Meeting
6-11 June 2015
Step 1 Technical Document:
Aim to resolve any disagreements identified and to finalise the Step 1 document.
Aim for Step 2a: Seek agreement of the SC members that there is sufficient scientific consensus on the technical
issues for the technical document to proceed to Step 2b. Alternatively, if a finalized Step I document cannot be
reached, reach agreement on actions needed to resolve divergent positions.
July 2015 to November 2015 Work towards for Step 2b: Continue discussion of methodological issues identified in the technical document and
start drafting the draft Addendum based on the Step 1 technical document.
Draft the outline of a technical appendix. Identify actions that will be undertaken until the ICH meeting in November
2015, and that will contribute to the creation of the Addendum.
ICH Meeting November 2015 Finalise Step 2b: Finalise the draft Addendum and the technical appendix, to be ready for public consultation. Seek
endorsement by the SC. Step 2b is reached when the six Regulatory Parties sign off the draft Addendum.
November 2015 to June 2016 Step 3 Stage I: Publish the Step 2 document for regional regulatory consultation after the November 2015 ICH
Meeting. The duration of public consultation, either 3 or 6 months, is subject to approval by each ICH region.
Initiate Step 3 Stage II: Depending on the duration of the public consultation in each region, start discussion of
regional consultation comments.
ICH Meeting June 2016 Deliverables set for this period will depend on the duration of the public consultation in each region and may be
delayed.
Aim for Step 3 Stage II: Discussion of regional consultation comments.
Finalise Step 3 Stage II: Address the comments received and reach consensus on the Step 3 Experts Draft Addendum.
June to November 2016 Deliverables set for this period will depend on the duration of the public consultation in each region and may be
delayed.
Aim for Step 3 Stage III: finalisation of the Step 3 Experts Draft Addendum.
Aim for Step 4: adoption of the Addendum by the SC.
Work Plan
Estimand – Definition
An estimand reflects what is to be
estimated to address the scientific
question of interest posed by a trial.
The choice of an estimand involves:
• Population of interest
• Endpoint of interest
• Measure of intervention effect
9
Estimand – Definition (cont.)
(A) Population of interest
This is the population of subjects for which we are assessing the scientific question of interest. The
population of interest will inform the study population used for a particular clinical study to answer
the scientific question of interest. The study population is defined through the inclusion/exclusion
criteria of the study.
(B) Endpoint of interest
This is the measurable quantity directly related to the scientific question of interest. The endpoint of
interest is characterized through measurements or observations at a specific time point or within a
time period of interest.
(C) Measure of intervention effect
This is the effect attributed to an intervention that should take into account potential confounding
due to post-randomization events, such as non-compliance, discontinuation of intervention,
treatment switching, or use of rescue medication.
10
Estimand illustrations
NRC report: 1. (Difference in) outcome improvement for all randomized participants. 2. (Difference in) outcome improvement in those who adhere to treatment. 3. (Difference in) outcome improvement if all participants had adhered. 4. (Difference in) areas under the outcome curve during adherence to treatment. 5. (Difference in) outcome improvement during adherence to treatment Mallinckrodt et al, give a further illustration: 6. For all randomized participants at the planned endpoint of the trial attributable to the
initially randomized treatment
11
The Role of Sensitivity Analyses • All sensitivity analyses should address the same primary
estimand, i.e.
o same population,
o same outcome and
o same measure of intervention effect.
• All model assumptions that are varied in the sensitivity analysis should be in line with the estimand of interest.
• If additional estimands are of interest, these could be considered as secondary or exploratory estimands.
• Sensitivity analyses can also be planned for secondary / exploratory estimands and aligned accordingly
Example
Dapagliflozin (Bristol-Myers Squibb / AstraZeneca)
o Anti-diabetic therapy to treat hyperglycemia
o New drug application discussed in 2011 in a public
advisory committee
Dapagliflozin – Primary Endpoint and Statistical Analyses
Primary endpoint: Change in HbA1c from baseline to 24
weeks
Analysis set: modified intention to treat (all randomized patients + at
least one dose + baseline value + at least one post-baseline value)
Data after initiation of rescue medication was considered
as missing
o Interested in the effect of the initially randomized treatment
o Rescue medication can mask or exaggerate effects of the initially
randomized treatments
Primary analysis: ANCOVA using LOCF
Sensitivity analyses:
o ANCOVA using only complete cases
o Mixed-effects model for repeated measures (MMRM) of HbA1c values
Dapagliflozin – Comments by FDA Reviewer
“While FDA has implicitly endorsed LOCF imputation for
diabetes trials in the past, there is now more awareness
in the statistical community of the limitations of this
approach.
Instead I have included a sensitivity analysis in which the
primary HbA1c outcomes are used regardless of rescue
treatment, and no statistical adjustment is made for
rescue.
This approach is also imperfect, but it comes closer to
being a true intent-to-treat (ITT) analysis because it
disregards the non-randomized rescue treatment.”
Sponsor:
Remove data after
initiation of rescue
medication
Attempt to establish the
treatment effect of the
initially randomized
treatments had no patient
received rescue
medication
FDA:
Include all data regardless
of initiation of rescue
medication
Compare treatment
policies ‘dapagliflozin plus
rescue’ versus ‘control
plus rescue’
Dapagliflozin – Sponsor’s Interest versus Regulatory Interest
Implied ‘scientific questions of interest’:
What was done?
Sponsor:
Remove data after
initiation of rescue
medication
Attempt to establish the
treatment effect of the
initially randomized
treatments had no patient
received rescue
medication
FDA:
Include all data regardless
of initiation of rescue
medication
Compare treatment
policies ‘dapagliflozin plus
rescue’ versus ‘control
plus rescue’
Implied ‘scientific question of interest’:
What was done? Implied objectives / scientific
questions of interest differ for both parties.
This is hidden behind the method of
estimation / handling of ‘missing data’.
Need to avoid such ‘miscommunications’.
Dapagliflozin – Sponsor’s Interest versus Regulatory Interest
Case study exercise: hypothetical diabetes trial
Randomized, 2-arm (drug A and drug B) diabetes trial in
patients with type 2 diabetes mellitus (T2DM)
Endpoint is the change of HbA1c levels to baseline after 24
weeks of randomization
HbA1c levels are measured at baseline and at 4, 8, 12, 16, 24
weeks
For ethical reasons, patients are switched to rescue
medication once their HbA1c values are above a certain
threshold
Regardless of switching to rescue medication all (!) patients
are followed up for the whole study duration, i.e.
o there are no missing observations in this study
o patients never discontinue their study medication, unless they start rescue
medication
Potential Estimands of Interest Differ only in their ‘measure of intervention effect’
Estimand 1 Estimand 2 Estimand 3 Population Intended post-approval
population of T2DM patients
Intended post-approval
population of T2DM patients
Intended post-approval
population of T2DM patients
Endpoint Change of HbA1c level
to baseline after 24 weeks of randomization
Change of HbA1c level to
baseline after 24 weeks of randomization
Change of HbA1c level
to baseline after 24 weeks of randomization
Measure of
intervention effect
19
Questions
• What would you propose as the estimand?
Potential Estimands of Interest Differ only in their ‘measure of intervention effect’
Estimand 1 Estimand 2 Estimand 3 Population Intended post-approval
population of T2DM patients
Intended post-approval
population of T2DM patients
Intended post-approval
population of T2DM patients
Endpoint Change of HbA1c level
to baseline after 24 weeks of randomization
Change of HbA1c level to
baseline after 24 weeks of randomization
Change of HbA1c level
to baseline after 24 weeks of randomization
Measure of
intervention effect
Effect regardless of what
treatment was actually
received, i.e.
• effect of treatment
policies ‘drug A until
start of rescue followed
by rescue’ versus ‘drug B
until start of rescue
followed by rescue’.
Potential Estimands of Interest Differ only in their ‘measure of intervention effect’
Estimand 1 Estimand 2 Estimand 3 Population Intended post-approval
population of T2DM patients
Intended post-approval
population of T2DM patients
Intended post-approval
population of T2DM patients
Endpoint Change of HbA1c level
to baseline after 24 weeks of randomization
Change of HbA1c level to
baseline after 24 weeks of randomization
Change of HbA1c level
to baseline after 24 weeks of randomization
Measure of
intervention effect
Effect regardless of what
treatment was actually
received, i.e.
• effect of treatment
policies ‘drug A until
start of rescue followed
by rescue’ versus ‘drug B
until start of rescue
followed by rescue’.
Effect of the initially
randomized treatments
assuming that the
treatment effect
disappears and no
rescue effect occurs
after meeting rescue
criteria, i.e.
• effect of ‘drug A until
intake of rescue followed
by a disappearing drug A
effect’ versus ‘drug B
until intake of rescue
followed by a
disappearing drug B effect’.
Potential Estimands of Interest Differ only in their ‘measure of intervention effect’
Estimand 1 Estimand 2 Estimand 3 Population Intended post-approval
population of T2DM patients
Intended post-approval
population of T2DM patients
Intended post-approval
population of T2DM patients
Endpoint Change of HbA1c level
to baseline after 24 weeks of randomization
Change of HbA1c level to
baseline after 24 weeks of randomization
Change of HbA1c level
to baseline after 24 weeks of randomization
Measure of
intervention effect
Effect regardless of what
treatment was actually
received, i.e.
• effect of treatment
policies ‘drug A until
start of rescue followed
by rescue’ versus ‘drug B
until start of rescue
followed by rescue’.
Effect of the initially
randomized treatments
assuming that the
treatment effect
disappears and no
rescue effect occurs
after meeting rescue
criteria, i.e.
• effect of ‘drug A until
intake of rescue followed
by a disappearing drug A
effect’ versus ‘drug B
until intake of rescue
followed by a
disappearing drug B effect’.
Effect of the initially
randomized treatments
had all patients remained
on their randomized
treatment throughout the
study, i.e.
• effect assuming patients
did not receive rescue medication.
23
Questions
• What would you propose as the (consequent)
analysis methodology?
o What would be your chosen primary analysis?
Potential Estimands of Interest Primary Analyses
Estimand 1 Estimand 2 Estimand 3 Analysis Variable
• Change from
baseline to week 24
in HbA1c
• All HbA1c values are
used, regardless of
treatment
• Change from baseline to
week 24 in HbA1c
• HbA1c values after
intake of rescue
medication are set to
missing
• Change from baseline to
week 24 in HbA1c
• HbA1c values after intake
of rescue medication are
set to missing
Primary
Statistical Model
ANCOVA model • treatment group and
region will be fitted as
factors
• baseline HbA1c will be
fitted as a continuous
covariate.
Potential Estimands of Interest Primary Analyses
Estimand 1 Estimand 2 Estimand 3 Analysis Variable
• Change from baseline
to week 24 in HbA1c
• All HbA1c values are
used, regardless of
treatment
• Change from baseline to
week 24 in HbA1c
• HbA1c values after intake
of rescue medication are
set to missing
• Change from baseline to
week 24 in HbA1c
• HbA1c values after intake
of rescue medication are
set to missing
Primary
Statistical Model
ANCOVA model • treatment group and
region will be fitted as
factors
• baseline HbA1c will be
fitted as a continuous
covariate.
Missing data will be multiply
imputed based on a ‘Copy
Placebo’ controlled
imputation approach.
For every completed data set
fit an ANCOVA model, • treatment group and region
will be fitted as factors
• baseline HbA1c will be fitted
as a continuous covariate.
Overall inference is obtained
by applying Rubin’s rules on
the estimates obtained from
every imputed/completed
data set.
Potential Estimands of Interest Primary Analyses
Estimand 1 Estimand 2 Estimand 3 Analysis Variable
• Change from baseline
to week 24 in HbA1c
• All HbA1c values are
used, regardless of
treatment
• Change from baseline to
week 24 in HbA1c
• HbA1c values after intake
of rescue medication are
set to missing
• Change from baseline to
week 24 in HbA1c
• HbA1c values after intake
of rescue medication are
set to missing
Primary
Statistical Model
ANCOVA model • treatment group and
region will be fitted as
factors
• baseline HbA1c will be
fitted as a continuous
covariate.
Missing data will be multiply
imputed based on a ‘Copy
Placebo’ controlled
imputation approach.
For every completed data set
fit an ANCOVA model, • treatment group and region
will be fitted as factors
• baseline HbA1c will be fitted
as a continuous covariate.
Overall inference is obtained
by applying Rubin’s rules on
the estimates obtained from
every imputed/completed
data set.
The change from baseline
HbA1c values for Weeks 4, 8,
12, 16 and 24 will be analyzed
using a Mixed Model of
Repeated Measurements
(MMRM)
• Treatment group, visit and
region will be fitted as factors
• Baseline HbA1c will be fitted as
a continuous covariate
• Treatment group by visit and
visit by baseline will be
included as interaction terms
• Unstructured covariance
structure
27
Questions
• What would you propose as the (consequent)
analysis methodology?
o What would be your chosen sensitivity analyses?
Potential Estimands of Interest Sensitivity Analyses
Estimand 1 Estimand 2 Estimand 3 Analysis Variable
• Change from baseline
to week 24 in HbA1c
• All HbA1c values are
used, regardless of
treatment
• Change from baseline to
week 24 in HbA1c
• HbA1c values after intake
of rescue medication are
set to missing
• Change from baseline to
week 24 in HbA1c
• HbA1c values after intake
of rescue medication are
set to missing
Sensitivity analyses
Add/remove covariates
and/or interactions.
Use ‘Jump to Placebo’
instead of the ‘Copy
Placebo’ approach.
Use multiple imputation
instead of MMRM
29
Review of hypothetical example
• Was your estimand clear in terms of the measure of
treatment effect?
• Was your analysis methodology aligned to your
estimand?
• Were your sensitivity analyses aligned to your
estimand?
30
Questions & Feedback 1. Does the definition of an estimand make sense?
2. Does the proposed framework in the E9 addendum make sense?
3. Do you have any concerns with the framework?
4. How much of a difference is the proposed framework compared
to what statisticians currently use for designing clinical trials?
5. What do you see as key challenges for introducing the
framework?
6. How do you think clinicians will view estimands and the
proposed framework?
7. Is the ITT (strict) estimand (#1) in the hypothetical example
reasonable?
31
References • ICH concept paper (2014) E9(R1): Addendum to Statistical Principles for Clinical Trials on Choosing
Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials
• Lewis JA (1999) Statistical principles for clinical trials (ICH E9): An introductory note on an
international guideline. Statistics in Medicine, 18: 1903-1904.
• ICH Expert Working Group (1999) Statistical principles for clinical trials (ICH E9). Statistics in
Medicine, 18: 1905-1942.
• National Research Council of the National Academies (2010) The Prevention and Treatment of
Missing Data in Clinical Trials. Washington, D.C.: National Academies Press.
• EMA (2011), Guideline on Missing Data in Confirmatory Clinical Trials.
• O’Neill RT and Temple R (2012) The prevention and treatment of missing data in clinical trials: an
FDA perspective on the importance of dealing with it. Clin Pharmacol Ther, 91: 550-554.
• Little RJ, D’Agostino R, Cohen M, et al. The prevention and treatment of missing data in clinical trials.
N Engl J Med, 367(14): 1355-1360.
• A structured approach to choosing estimands and estimators in longitudinal clinical trials. C. H.
Mallinckrodt et al. Pharmaceut. Statist. 2012, 11 456–461
• Missing Data: Turning Guidance Into Action. C. H. Mallinckrodt et al. Statistics in Biopharmaceutical
Research 2013, Vol. 5, No. 4, 369-382
• Choosing Appropriate Estimands in Clinical Trials. AK Leuchs et al. DIA Therapeutic Innovation &
Regulatory Science. 2015