Optimal Drug Development Programs and Efficient Licensing and ReimbursementRegimens

Neil HawkinsKarl Claxton

OverviewSocietal and commercial value of InformationDecision rules incorporating value of informationChallenges

The Quantitative Estimate of the Value of Sample InformationThe value of additional sample information is the value of the increased likelihood of selecting the optimum treatment arising from the reduction in uncertainty regarding treatment effects (and costs).

What is the optimum treatment?The treatment with greatest expected net benefit in terms of costs and effects- Bayesian Decision Rule.

Decision Uncertainty

Expected Net Value of Sample Information (ENVSI)Expectation over potential future samples of: Net benefit from optimum decision made including the additional sample data Net benefit from optimum decision based on existing dataCost of collecting sample

Note: ENVSI < 0, if the optimum decision does not change due to extra sample data

Bayesian Simulation of Future Samples for Binomial Parameter

Sample P from current posterior distribution: Pcurrent ~ beta(a,b)

Simulate Trial DatarT ~ bin(n, PTx )

Calculate new posterior distributionPnew ~ beta(a+r,b+n-r)

Small Sample (n=1)

Large Sample (n=2000)

Example Phase II Trial Results2 Test: 2.0037, 1 df, p = 0.1569

Tx

Placebo

Response

16

8

Deaths

1

1

Total

49

45

Decision Analytic ModelNet Benefit =

P(Resp) x QALYs Gained | Resp x Monetary Value of a QALY - P(Death) x QALYs Lost | Death) x Monetary Value of a QALY -Treatment Cost

Example Parameter EstimatesQALYs Gained | Response : ~ N(0.7,0.12) x 4

QALYs Lost | Death : ~ N(0.7,0.12) x 4

Value of a QALY: 30,000

Treatment Cost Per Course :(

Based on Current DataNew treatment is cost-effectiveNew treatment would not get approval based on a frequentist hypothesis test

Societal Value of Sample Information (Efficacy Trial)

Commercial Value of Sample Information

The value of increased sales due to the increased probability of regulatory and reimbursement approval arising from the extra information

ICH E9: Guidance on Statistical Principles for Clinical Trials Using the usual method for determining the appropriate sample size, the following items should be specified: probability of erroneously rejecting the null hypothesis probability of erroneously failing to reject the null hypothesis

ICH E1A: The Extent of Population Exposure to Assess Clinical Safety100 patients exposed for a minimum of one-year is considered to be acceptable to include as part of the safety data base.

It is anticipated that the total number of individuals treated with the investigational drug, including short-term exposure, will be about 1500.

Probability of Approval (Efficacy Endpoint Current Regulatory Regimen)

Value of Sample Information (Efficacy Endpoint - Current Regulatory Regimen)

What happens if we just use a Bayesian CE decision rule?

Value of Sample Information (Efficacy Endpoint - Bayesian CE Decision Rule)

Societal Value of Sample Information (Utility Study)

Value of Sample Information (Utility Study Current Regulatory Regimen)

ImplicationsUnder current regulatory system we might expect a lack of outcomes and long-term dataWe need to consider uncertainty and resulting VOI when making decisions, not just expectations based on current dataHow should we do this?

Maybe we shouldnt abandon frequentist hypothesis testing just yet?

The FDA view

...A reasonable basis for a claim [of cost-effectiveness] depends on a number of factors relevant to the benefits and costs of substantiating a particular claim. These factors include: the type of product, the consequences of a false claim, the benefits of a truthful claim, the costs of developing substantiation for the claim ...

Potential Industry Responses to Approval based on Value of Information

Reduce cost of uncertainty by research or price reductionTrade-off between:Additional researchCost, delay and uncertain outcomeEntry and free riderPrice reductionReduces EVI (for payoffs > 0) but reduces revenues

Approval Based on Expected Net Value of Sample Information

Approve new (more expensive?) treatment if expected net benefit of treatment is greater than existing treatment and expected net value of further sample Information is zero

Approval Based on Expected Net Value of Sample InformationHard to define set of endpoints, study designs and sample space over which we calculate value of sample information ENVSI is uncertain and will change as data become available. When is ENVSI defined?Many of the parameters required to estimate ENVSI are uncertain and may not be transparentNon-financial capacity restraints on further researchWhat decision do we make in the interim? - Sunk costs, irreversibility and option value

Approval based on Expected Value of Perfect Information

Summary results of the NICE pilot study

Approval based on Expected Value of Perfect InformationApprove new therapy if Expected Value of Perfect Information is below a given threshold at an acceptable cost-effectiveness threshold

Requires an arbritary EVPI threshold for approvalParameters still uncertain. For example; relevant time horizon, future technological change.

Approval based on Decision Uncertainty

Approval based on Decision UncertaintyApprove new therapy if decision uncertainty is below a given threshold at an acceptable cost-effectiveness threshold

Requires an arbritary uncertainty threshold for approval

Some ChallengesHow we consider uncertainty when decision making will influence the availability of evidence How do we frame explicit decision rules incorporating uncertainty?