GlobalPharmacometrics

Enhanced Quantitative Decision Making

- Reducing the likelihood of incorrect decisions

Mike K. Smith, Jonathan French, (Pfizer)

Ken Kowalski, (A2PG)

Wayne Ewy (formerly Pfizer, retired).

2

Six Components ofModel-Based Drug Development*

Model-Based Drug Development

PK/PD & Disease Models

Competitor Info. & Meta-Analysis

Design & Trial Execution Models

Data Analysis Model

Decision Criteria

Trial Performance Metrics

* Lalonde et al, Clin Pharm & Ther, 2007; 82: pp21-32

Quantitative Decision Criteria

• “I’ll know it when I see it…”

• “Evidence of an effect…”

• “Reasonable efficacy and safety tradeoffs”

• WRONG!!!

Quantitative Decision Criteria

• 2 points improvement over placebo.– Better. – At least it’s quantitative

• How sure do you want to be?– Mean 2 points?– Lower CI 2 points?– Mean 2 points and lower CI > 0?

P(Criteria|Data)

• Not just P(… | Data)– Data– Prior data, model assumptions, parameter

uncertainties– Trial design– Dropouts, imputation methods etc.– Data analytic method

Truth vs Trial

• For a given set of model parameters / assumptions there will be a “true” outcome against the decision criteria.– What is the chance of achieving 2 points improvement

given current information?– For a given set of parameters we will know whether

we achieve 2 points improvement or not.

• Then for this same set of parameters, apply design, dropout / imputation models, analytic technique and assess decision criteria.

Truth vs. Trial - Formally

is the true (unknown) treatment effect =f(, , ) is specified for a given set of

model assumptions vector of fixed effects

parameters covariance matrix for

between-unit (subject or study) random effects

covariance matrix for within-unit (subject or study) random effects

Truth vs. Trial - Formally

• Define quantitative decision rule under truth () and data-analytic results (T), e.g.,– Truth: Go if TV, No Go if

<TV– Data: Go if TTV, No Go if T<TV

• Note TV denotes the Target Value• Note T could be a point estimate or confidence

limit on estimate/prediction of

11

Operating Characteristics

Correct No Go Incorrect GoP(True No

Go)

Incorrect No Go Correct Go P(True Go)

P(Trial No Go) P(Trial Go) 1.0

Trial No Go Trial Go Total

“Tru

e”

No

Go

“Tru

e”

Go

To

tal

P(correct) P(Go) PTS

Example

• Comparing SC-75416 with ibuprofen in dental pain.– Published in Kowalski, K.G, et al. “Modeling

and Simulation to Support Dose Selection and Clinical Development of SC-75416, a Selective COX-2 Inhibitor for the Treatment of Acute and Chronic Pain”.

• Decision criteria based on 3 point difference from ibuprofen in TOTPAR6 endpoint.

Example

0

500

1000

1500

2000

2500

Fre

qu

en

cy

0 1 2 3 4 5Delta-TOTPAR6

360 mg SC-75416 vs 400 mg Ibuprofen360 mg SC-75416 vs 400 mg Ibuprofen

Obs Mean = 3.3

PTS = P( 3) = 67.2

From Kowalski et al: A model-based framework for quantitative decision-making in drug developmentPresentation at ACOP, Tuscon, AZ 2008.

17

Trial

Truth

Trial No Go

LCL95 0 or Mean<3

Trial Go

LCL95> 0 and Mean3 Total

k<3 2081

20.81%

1199

11.99%

3280

32.80%

k3 1729

17.29%

4991

49.91%

6720

67.20%

Total 3810

38.10%

6190

61.90%

10,000

100%

P(correct) = 70.72% P(Go) = 61.90% PTS = 67.20%

Example

From Kowalski et al: A model-based framework for quantitative decision-making in drug developmentPresentation at ACOP, Tuscon, AZ 2008.

“Nominal” values for OCs

• P(Correct) can be fixed at >=80%• PTS for initiating a new trial depends on

quadrant, portfolio, stage of development.– Perhaps minimal “dignity level” for starting a trial.

• Fixing these two implies P(False GO) and P(False NO GO) must float, depend on P(Correct) and PTS.– Driven by decision criteria.

• E.g. For P(Correct) = 80%, P(Incorrect) = 20%, spent across P(False GO), P(False NO GO).

Iterate / Optimise

• If the operating characteristics “don’t look good”…– Change the data analytic model– Change the design constraints (↑ n /group)– Change the data-analytic decision criteria for the trial.

• If we fix one or more of the above (e.g. n /group) then there is limited other things that can improve OCs.– Change the data analytic model, change data-analytic

decision criteria for the trial.

The components may change over time

• “Truth” model / prior will be refined over time.– P(“True” Go given current knowledge / model)

changes.• Decision criteria may change.

– Commercial viability changes. [This may change both our compound target criteria – truth decision rule, as well as the data-analytic decision rule]

– Acceptable level of confidence for Trial Go decision changes. [This applies only to data-analytic decision rule]

Final Remarks (1)

• Greater collaboration required among kineticists/modelers, statisticians and clinicians

• Kineticists/modelers:– Explicit and transparent about the assumptions and

limitations of their PK/PD and disease models– Think strategically about how model will be used to

influence internal decision-making– Avoid excessive use of NONMEM-jargon and write

reports to broader audience– Calibrate models against data-derived (non-model-

based) statistics of interest

Final Remarks (2)

• Statisticians:– Embrace assumption-rich nonlinear models for

decision-making especially in early clinical development

– Avoid “Phase 3” mentality when designing Phase 2 studies…relying on empirical (assumption-poor) models to make decisions in early clinical development can be costly

• Clinicians:– Quantitatively define clinically relevant effects and

commercial targets– Explicitly and quantitatively defined decision rules

Bibliography1. Kowalski, K.G., Ewy, W., Hutmacher, M.M., Miller, R., and

Krishnaswami, S. “Model-Based Drug Development – A New Paradigm for Efficient Drug Development”. Biopharmaceutical Report 2007;15:2-22.

2. Lalonde, R.L., et al. “Model-Based Drug Development”. Clin Pharm Ther 2007;82:21-32.

3. Kowalski, K.G., Olson, S., Remmers, A.E., and Hutmacher, M.M. “Modeling and Simulation to Support Dose Selection and Clinical Development of SC-75416, a Selective COX-2 Inhibitor for the Treatment of Acute and Chronic Pain”. Clin Pharm Ther, 2008; 83: 857-866.

4. Kowalski, K.G., French, J.L., Smith, M.K., Hutmacher, M.M. “A model-based framework for quantitative decision making in drug development”. Presentation at ACOP, Tuscon, AZ. 2008. http://tucson2008.go-acop.org/pdfs/8-Kowalski_FINAL.pdf