Upload
stephen-senn
View
279
Download
0
Embed Size (px)
Citation preview
Tokyo First in Man 1
Ethical, practical and statistical considerations in designing first-in-man
studies
Stephen Senn
Outline
• I shall discuss the Royal Statistical Society Working party report in some detail– This made a number of recommendations but
also left many matters open• I shall make some very brief comments
regarding subsequent work
Tokyo First in Man 2
Tokyo First in Man 3
RSS Working Party Members
Dr Dipti Amin, Senior Vice-President, Global Medical & Regulatory Affairs , Quintiles LimitedProfessor R.A. Bailey, Professor of Statistics, Queen Mary, University of LondonProfessor Sheila M. Bird FFPH, Principal Scientist/Statistician, MRC Biostatistics Unit, visiting professor at Department of Statistics and Modelling Science, University of StrathclydeDr Barbara Bogacka, Reader in Probability and Statistics, Queen Mary, University of LondonMr Peter Colman+, Senior Consultant Statistician, Pfizer Global R+D, Statistical ApplicationsDr Andrew Garrett, Vice President Biostatistics, Quintiles LimitedProfessor Andrew Grieve, Professor of Medical Statistics, King’s College LondonProfessor Sir Peter Lachmann, FRS, FMedSci, Emeritus Professor of Immunology, University of CambridgeProfessor Stephen Senn*, Professor of Statistics, University of Glasgow
* Chairman+ Representative of Statisticians in the Pharmaceutical Industry (PSI)
Acknowledgement
Tokyo First in Man 4
TGN1412
• A monoclonal antibody• First-in-man study on 13 March 2006 carried
out by Parexel on behalf of TeGenero• In first cohort 8 volunteers• Six allocated TGN1412 and two allocated
placebo• All six given TGN1412 suffered a cytokine
storm
Tokyo First in Man 5
See. Senn SJ. Lessons from TGN1412. Applied Clinical Trials 2007;16(6):18-22.
Tokyo First in Man 6
A Conventional AnalysisFISHER'S EXACT TEST
Statistic based on the observed 2 by 2 table(x) : P(X) = Hypergeometric Prob. of the table = 0.0357 FI(X) = Fisher statistic = 6.095
Asymptotic p-value: (based on Chi-Square distribution with 1 df ) Two-sided:Pr{FI(X) .GE. 6.095} = 0.0136 One-sided:0.5 * Two-sided = 0.0068
Exact p-value and point probabilities : Two-sided:Pr{FI(X) .GE. 6.095}= Pr{P(X) .LE. 0.0357}= 0.0357 Pr{FI(X) .EQ. 6.095}= Pr{P(X) .EQ. 0.0357}= 0.0357 One-sided:Let y be the value in Row 1 and Column 1 y =6 min(Y) =4 max(Y) =6 mean(Y) = 4.500 std(Y) = 0.5669
Pr { Y .GE. 6 } = 0.0357 Pr { Y .EQ. 6 } = 0.0357
Tokyo First in Man 7
A Slightly Less Conventional AnalysisDatafile: C:\Program Files\Numerical\StatXact-4.0.1\Files\Research\TGN1412.cy3
BARNARD'S UNCONDITIONAL TEST FOR DIFFERENCE OF TWO BINOMIAL PROPORTIONS
Statistic based on the observed 2 by 2 table :
Binomial proportion for column <Yes > : pi_1 = 1.000 Binomial proportion for column <No > : pi_2 = 0.0000 Difference of binomial proportions : Delta = pi_2 - pi_1 = -1.000 Standardized difference of binomial proportions : Delta/Stdev = -2.828
Results:-------------------------------------------------------------------------Method P-value(1-sided) P-value( 2-sided)-------------------------------------------------------------------------Asymp 0.0023 (Left Tail) 0.0047Exact 0.0111 (Left Tail) 0.0113
Tokyo First in Man 8
Conclusions
• “If you need statistics to prove it, I don’t believe it”
• Here the problem is the reverse• You can’t prove it with statistics but
everybody believes• So does this mean statistics is irrelevant• Not if you look more closely…
Tokyo First in Man 9
Further information
• Timing of adverse events• Increasing interest in using this feature in
epidemiological studies– Case series methodology
• Farrington and Whitaker (2006)
• Also if we use background knowledge of risk of cytokine storm we come to quite different conclusions
Tokyo First in Man 10
0 0.2 0.4 0.6 0.8 1
0.005
0.01
0.015
P-value or Likelihood Ratio
Unconditional test
Common probability of reaction
P-va
lue
Barnard
6
8
Tokyo First in Man 11
0 0.01 0.02 0.03 0.04 0.051 10 241 10 231 10 221 10 211 10 201 10 191 10 181 10 171 10 161 10 151 10 141 10 131 10 121 10 111 10 101 10 91 10 81 10 7
L ratio with knowledge of prob adv react placebo
Probability side-effect under placebo
Like
lihoo
d ra
tio
LR 0
0
Tokyo First in Man 12
RSS Recommendations:Themes
• Generic issues• Preparatory work before first in man• Content of protocols (including design)• Risk and information sharing for social
good and reporting standards
Tokyo First in Man 13
Generic
• MHRA to make adequate provision for statistical expertise
• Mandatory insurance of participants• Tertiary care hospitals only if cytokine
storm even a remote possibility
Tokyo First in Man 14
So, were the six healthy volunteers compensated without fuss?
From the protocol
Lawyers acting for the six volunteers who suffered multiple organ failure during the trial of a new drug last month fear the men may not get full compensation because the company accepting liability
for the injuries was underinsured, the Guardian has learned. Martyn Day, solicitor for four of the men, said the German pharmaceutical company TeGenero, which created the drug TGN1412, had insurance of only £2m for the trial but the liability for damages and losses suffered by the men may be much bigger.
Guardian, 26 April 2006
Tokyo First in Man 15
Tokyo First in Man 16
From the informed consent
Tokyo First in Man 17
Herald TribuneTeGenero, the small German biotechnology firm that developed TGN1412 - which it touted as a revolutionary new drug against cancer and arthritis - filed for bankruptcy several weeks ago. ….
Its insurance coverage was not adequate to cover a calamitous outcome, said Martyn Day of Leigh Day & Company in London, the lawyer for four of the six ill men. ….
Day said he would argue that Parexel, which contracts with drug makers to test new medicines, should have made sure that its client had adequate insurance. He also questioned the design and conduct of what it should have seen as a delicate trial, he said.
Elizabeth Rosenthal 30 July 2006
Tokyo First in Man 18
Preparatory• Quantitative justification of dose and risk level
with statements of uncertainty• Classification of studies low, medium high risk• Precautionary approach• Separate document covering these for use by
ethics committees, participants, insurers• Human cell line studies to improve inter-species
scaling• A ‘proper interval’ for dosing to be established for
(or justification as to why not needed)
Tokyo First in Man 19
Tokyo First in Man 20
Content of Protocols• Justification of relevant quantities
– proper interval, dose step, safety, expected number of adverse events etc
• Statistical justification of sample size• Justification of design• Detailed description of intended analysis• Design and analysis should reflect realistic PK models• Plan for blood sampling etc to based on pre-clinical studies• True informed consent: ‘open protocol, hidden allocation’
Tokyo First in Man 21
Tokyo First in Man 22
Questions
1. Why 6 + 2
2. Why four doses?
3. Why quintupling?
4. Why simultaneous treatment in cohorts
Tokyo First in Man 23
The Trial of TGN1412 was also a veiled Trial
• Subjects do not know whether they are getting TGN1412 or placebo
• If they were in the first cohort they knew they could not be receiving higher doses
• This would have had implications for analysis had the trial proceeded normally
• Would you pool all placebo results or not?
Tokyo First in Man 24
To Pool or Not?• ‘Yes’, according to the protocol• But then misleading to describe the trial as
double-blind• Bias variance trade-off
– The proposed analysis would not eliminate the biases blinding is designed to eliminate
• Also what about the analysis at the end of each dose step to guide dose-escalation?
• How would this permit pooling of placebo subjects?
Tokyo First in Man 25
Design Number of subjects Variance of differences between doses, and between placebo and each dose, if
a cohort effect is fitted it is known that there is no cohort effect
1
Dose 0 1 2 3 Cohort 1 2 6 0 0 Cohort 2 2 0 6 0 Cohort 3 2 0 0 6
1 2 3 0 0.67 0.67 0.67 1 1.33 1.33 2 1.33
1 2 3 0 0.33 0.33 0.33 1 0.33 0.33 2 0.33
2
Dose 0 1 2 3 Cohort 1 4 4 0 0 Cohort 2 4 0 4 0 Cohort 3 4 0 0 4
1 2 3 0 0.50 0.50 0.50 1 1.00 1.00 2 1.00
1 2 3 0 0.33 0.33 0.33 1 0.50 0.50 2 0.50
3
Dose 0 1 2 3 Cohort 1 4 4 0 0 Cohort 2 2 2 4 0 Cohort 3 1 1 2 4
1 2 3 0 0.29 0.40 0.65 1 0.40 0.65 2 0.58
1 2 3 0 0.29 0.31 0.39 1 0.31 0.39 2 0.42
Tokyo First in Man 26
Risk and Information Sharing• Public debate regarding maximum acceptable risk• Drug regulators to provide mechanism for
sponsors to share data to improve risk assessment• Statistical reporting to be improved to level
provided by International Conference on Harmonisation for phase 3
• Use of quantitative descriptions to be increased• Mock applications currently available are poor and
need to be improved.
Starting doses
• Two standard relevant concepts – MABEL – minimum anticipated biological
effect level– NOAEL – no observable adverse effect level
• Both need to be considered
Tokyo First in Man 27
Tokyo First in Man 28
Who’s Risk?
• Suppose that the acceptable risk to an individual is 1 in 2000
• But we believe it is 1 in 1000 for this drug• By having one placebo for every active
treatment and randomising we can reduce the risk to 1 in 2000
• Does this make it acceptable?
Tokyo First in Man 29
We Think Not
• Such a device reduces the risk to the individual to acceptable levels
• However, it does not reduce the expected number of side-effects per trial– Nor the risk to the insurer
• Hence we suggest the dual perspective• Acceptable to an individual AND to society
Tokyo First in Man 30
The Future of First-in Man Studies
• More care• More science• More forethought• More transparency• More accountability• To be treated as seriously as phase III• But…..
More Recent Work by Rosemary Bailey
Dose 0 1 2 3 4Cohort 1 5 5 0 0 0
Cohort 2 5 0 5 0 0
Cohort 3 5 0 0 5 0
Cohort 4 5 0 0 0 5
Dose 0 1 2 3 4Cohort 1 5 5 0 0 0
Cohort 2 3 2 5 0 0
Cohort 3 1 1 3 5 0
Cohort 4 1 1 1 2 5
Tokyo First in Man 31
‘Senn’ design ‘Catch-up’ design
NB Dose 0 = placebo
Tokyo First in Man 32
Variance of contrast to placebo
3
0.25
2
0.30
0.35
0.40
4
0.45
0.50
1
0.20
Dose
Variance
'Catch-up' design'Senn' design
However,Bailey assumes that the design will reachCompletion. But the whole point is that the design may not reach completion and we have to decide whether to proceed as each cohort reports.
Tokyo First in Man 33
The dashed lines and asterisks give the variances for each dose contrast at the time that each cohort completes. The final value at the end of the design is labelled F.
The red solid line gives variance at the time the decision has to be made about that dose.
The blue diamond is the value for the ‘Senn’ design.
R
R
R
R
F
F
F
F0.2
1
0.3
3
0.4
0.5
0.6
2 4
Cohort
Variance
Conclusion
• Design theory is important but we have to be careful
• These trials involve dynamic decision making
• We have to consider what may happen not just what we hope will happen
Tokyo First in Man 34
Tokyo First in Man 35
The storm has cleared: lessons from the CD28 superagonist TGN1412 trial
Thomas Hünig The life-threatening cytokine-release syndrome suffered by six volunteers in a Phase I clinical trial following administration of the CD28 superagonist antibody TGN1412 (developed by TeGenero) in March 2006 was completely unpredicted by the preclinical studies. Here, Thomas Hünig, main founder of TeGenero, describes the recent investigations into what went wrong and discusses the lessons learnt for future clinical trials.
NATURE REVIEWS | IMMUNOLOGY VOLUME 12 | MAY 2012 | 317
This is an interesting paper but…
• Discussed– Explanation as to why
the dose was incorrectly calculated
– Why MABEL is preferable to NOAEL
• Not discussed– Basic design– Ethical issues– Monitoring– Insurance
Tokyo First in Man 36
In Conclusion
• We still do not know how to get this right• We still have grounds to be uneasy• We need an ethical debate that is not just
limited to the clinical trial community but involves society more widely
Tokyo First in Man 37