Oxford Inflammatory Bowel Disease MasterClass

Understanding non-inferiority trial designs

Dr Vipul Jairath Bsc DPhil (Oxon) MRCPNIHR Clinical Lecturer

Translational Gastroenterology UnitNuffield Department of Medicine

University of Oxford

Aim of this talk

To provide a practical guide to the clinician: What they are Why they are conducted Determining an acceptable margin of non-inferiority Design considerations Perils and pitfalls – of which there many! Interpreting the results An IBD trial example

Conventional Designs Parallel group Non-inferiority Equivalence Cluster Factorial Cross-over Multi-arm

Adaptive Designs Sample size re-estimation Dropping treatment arms Change allocation ratio Change primary endpoint Sequential

Alternative Designs Stepped-Wedge Complex Interventions Patient Preference Zelen N of 1 Post marketing surveillance

Conventional and novel trial designs

Biosimilar development pathway

EMA ....... similar clinical efficacy between the similar and reference

product should be demonstrated in adequately powered RCTs, preferably double blind equivalence trials

FDA ....Non-inferiority designs are acceptable which should

demonstrate no clinically meaningful difference in efficacy and shows that the biosimilar poses no more risk in safety or immunogenicity

What are non-inferiority or equivalence trials?

Superiority trial: Designed to prove that E is better than C Lack of difference ≠ equivalence

Non-inferiority trial: Designed to show that E is only marginally inferior to C (-∆) A one sided comparison (interested in degree of detriment)

Equivalence trial: Designed to show that E is not appreciably inferior or superior

to C (a two-sided comparison; -∆, +∆)

Null and alternative hypotheses

Parallel group trial: Designed to show new intervention is superior Null hypothesis = no difference between treatments We reject the null hypothesis, decide one is superior to the

other if there is sufficient evidence

Non-inferiority/equivalence trials: Null hypothesis is reversed Null hypothesis is that there is a difference Aim to show one intervention is not inferior (or is equivalent) to

the other

Null and alternative hypotheses

Non-inferiority trial Equivalence trial

Superiority, non-inferiority or equivalence?

CI does not contain zero

SUPERIORITY TRIAL

EQUIVALENCE TRIAL

CI is in the window of equivalence

NON-INFERIORITY TRIALCI is within Delta (margin of non-inferiority)

Why conduct a non-inferiority or equivalence trial?

To test new form of an existing drug

Not ethical to do a placebo response trial

Even in E is non-inferior to C on the primary efficacy endpoint, it would still need ancillary benefits: Better secondary endpoints Better safety profile Easier route of administration Simpler dosing regimen Cheaper to produce Compliance expected to be better outside of RCT

Equivalence trials largely used for bioequivalence studies

Garattini, Lancet, 2007

Designing the non-inferiority/equivalence trial

Select the non-inferiority margin (∆)

Run the trial comparing E to C

Calculate the confidence interval around the difference between treatments

Look at lower bound of CI

If the lower bound of the CI is within the margin -∆, the new treatment is deemed non-inferior and trial a success

Select ∆, “minimally clinically important difference in the primary endpoint”

Run the trial comparing E to C

Calculate the confidence interval around the difference between treatments

Look at upper and lower bound of CI

If these are within -∆ to +∆, the new treatment is deemed equivalent and trial a success

How is the control arm event rate calculated?

Overestimate this may lead to an underpowered trial

Look at historical event rates in the control arm ideally based on meta-analysis Beware that even these event rates could be outdated

Feasibility data, own experiences

Do a prespecified interim analysis during the trial and adjust the sample size accordingly

Extending trial duration to meet the number of events

How is the NI/equivalence margin chosen?

This is the crux of the trial and not entirely scientific: Overly conservative = inability to detect non-inferiority Overly liberal = risk of claiming non-inferiority

Clinical judgement: Ask experts (e.g. Delphic procedure) or patient groups This is likely to be insufficient for regulators

Choice of NI margin: absolute versus relative risk reduction

Reality: Clinical judgement, statistical (budget)

If inappropriate thresholds set and uncontested this could lead to the uptake of treatments detrimental to patients

Mulla, S et al. JAMA 2012

Preserving 50% of the “minimal treatment effect”

More effective treatments need to preserve > 50%

How is the sample size calculated?

In principle similar to other trials Proportion of patients expected to experience outcome Significance level (α, Type 1 error = “false positive”) Power (1-β, 1- Type 2 error = “false negative”) ∆

90% power, since less than this biases towards non-inferiority/equivalence

The sample size increases with: Greater power Smaller ∆

Sample size examples for an NI trial

E C ∆ N

A 88% 88% -10 592

B 88% 88% -12 414

C 85% 85% -10 716

D 90% 88% -10 382

90% power; 2.5% once sided alpha, 10% dropout rate

What are the important design aspects?

Rigorous methods: poor rigour rewards in NI/equivalence trials; penalises in superiority trials

Eligibility: patients in the trial should be similar to trials which established effectiveness of the standard intervention

Dose: of the standard intervention should be similar to those found to be effective in previous trial Low doses: erroneous equivalence, where both interventions

have no clinical response High doses: can claim equivalence, but excess AEs

Concomitant medications: high response rates may be due to the effect of concomitant medications

Design aspects: Dose selection

How can “Biocreep” be avoided?

A concern particularly with non-inferiority trials

A slightly inferior drug becomes the comparator for the next generation of compounds and so on.

Over time new drugs may only have efficacy close to that of placebo

Will occur if a new drug with lower efficacy than the comparator is approved with a wide ∆

Avoid this by: Using gold standard as the active comparator Incorporate a placebo arm into the trial (?Ethical)

How should the results be analysed and presented?

Intention to treat analysis Preserves randomisation Produced a conservative estimate in superiority trial For a NI/equivalence trial this in not conservative as will make

the groups more similar!

Per-protocol analysis Compromises randomisation Non-adherent often prognostically worse

Both should be presented If consistent = reassuring In not consistent = inference of non-inferiority/equivalence is

weakened

What can the results be from a non-inferiority trial?

Reporting non-inferiority and equivalence trials

Topic item number Additional statement required

Design 1 that the trial was designed to show non-inferiority or equivalence

Background 2 rationale for using a non-inferiority or equivalence design

Participants, interventions and outcomes 3, 4 & 6

whether the eligibility criteria, interventions (e.g. dose), and outcomes are similar to those of any trial(s) which established the efficacy of the standard intervention

Objectives, sample size and analysis method 5, 7 & 12 whether the hypothesis is of non-inferiority or equivalence (one- or two-

sided), and the magnitude of difference used to define this hypothesis

Numbers analysed 16 whether intention to treat or alternative analyses were done

Outcomes 17 for each outcome, 'a figure showing confidence intervals and margins of equivalence may be useful'

Interpretation 20 should take into account the non-inferiority or equivalence hypothesis

An example from an IBD trial

Some take home messages

NI trials are acceptable when a new therapy has a sufficiently favourable property that clinicians/patients are willing to sacrifice some degree of benefit relative to an approved therapy

Goals for the three trials are different Superiority: E is better than C Equivalence: E is not too different from C Non-inferiority: E is not much worse than C

Some take home messages

The margin of ∆ should be prespecified and justified clinically

How was event rate in the control arm estimated

Sample size and power

Check the conduct of the trial Eligibility, dosing, concomitant medication Poor rigour rewards in these designs!

Look at the analysis of the trial PP and ITT analysis and if they concur

Check that only trials planned as non-inferiority/equivalence are reported as such and not claimed retrospectively!

Some take home messages

Equivalence or non-inferiority trials have specific challenges and both designs acceptable dependent on regulator

Methodological rigour to prevent erroneous conclusions and unacceptably inferior products entering the market

Large sample sizes

Less incentive to conduct them well

Emerging biosimilars will employ these designs and we must appreciate the key principles to inform treatment choices

Suggested reading

Readings from text books Wang D, Bakhai A . Clinical Trials: A Practical Guide to Design, Analysis and Reporting. REMEDICA (1

Nov 2005). Chapters 12 & 14

Senn S. Statistical Issues in Drug Development (2nd edition). Wiley, Chichester, 2007, ISBN 978-0-470-

01877-4. Chapter 15

Papers Jones B, Lewis J, Ebbutt E. Trials to assess equivalence: the importance of rigorous methods. BMJ.

1996; 313: 36-9

Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340:c869.

Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006 Mar 8; 295(10): 1152-60.