Sequential Monitoring of Clinical Trials - Session 7 - Special ...rctdesign.org/ShortCourses/uwAugust2013/session7.pdfNon-Inferiority Designs Fixed-sample non-Inferiority designs Regulatory

RCTdesign

Designs meritingspecial consideration

Futility DecisionsStochastic Curtailment:Conditional Power

Non-Inferiority DesignsFixed-samplenon-Inferiority designs

Regulatory guidance

Group sequentialnon-Inferiority designs

UW Grp Seq - Sec 7 - sld 1

Sequential Monitoring of ClinicalTrialsSession 7 - Special Considerations:

Futility and Non-Inferiority Decisions

Presented August 14-16, 2013

Daniel L. GillenDepartment of Statistics

University of California, Irvine

John M. KittelsonDepartment of Biostatistics & Informatics

University of Colorado Denver

c©2013 Daniel L. Gillen, PhD and John M. Kittelson, PhD

RCTdesign




Regulatory guidance



Designs meriting special consideration:Futility decisions

Futility decisions

I Futility designs in the unified familyI On the use of stochastic curtailmentI What is stochastic curtailment?

I Stopping decisions based on conditional powerI Stopping decisions based on predictive power

I Example: sepsis trial

RCTdesign




Regulatory guidance



Futility decisions in the unified family

Futility decisions in the unified family

I One-sided test (ε` = 0, εu = 1) means aj = cj and bj = dj .I Set Pa = Pc < ∞ and Pb − Pd =∞ to get a single aj

stopping boundaryI Choose Pa to control early conservatism

I Properties evaluated as illustrated in session 3.

RCTdesign




Regulatory guidance



On the use of stochastic curtailment

Stochastic curtailment

I Consider the sepsis trial:I Suppose you observe a 5% higher mortality rate with

antibody treatment at the first interim analysis (θ̂ = 0.05).You are considering stopping for lack of effect.

I It might seem relevant to ask: “if I were to continue, would Ichange my mind?"

I More formally, what is the probability of reversing the currentdecision:

Pθ(θ̂J < d∗J |θ̂1 = 0.05)?

This is a form of power. If it is small, then you should stop.I The probability of changing your mind based on the data is

known as stochastic curtailment. There are two types ofapproaches:(a) Conditional power: Power based on some relevant value for

θ.(b) Predictive power: Mean power integrated over a posterior

distribution for θ.

RCTdesign




Regulatory guidance




Conditional power example

I Recall the sepsis trial:I According to the monitoring plan, the trial would stop for

benefit if θ̂J < d∗J = −0.0435 at NJ = 1700 patients.I If θ̂1 = 0.05, then what is the chance (power) that θ̂J < d∗J ?I If we observe θ̂j = x , then the increment between the j th

and Jth analysis is normally distributed:

NJ θ̂J − Njx ∼ N [(NJ − Nj)θ, (NJ − Nj)V ]

where V = 0.23× 0.77 + 0.3× 0.7 = 0.387.

RCTdesign




Regulatory guidance




Conditional power (example)

I It follows that:

Pθ(θ̂J < d∗J |θ̂j = x) < α

⇒ NJd∗J − Njx − (NJ − Nj)θ)p(NJ − Nj)V

< zα

⇒ x > N−1j

hNJd∗J − (NJ − Nj)θ − zα

pNJ − Nj

iThat is, we would be very unlikely (probability less than α)to reverse a futility decision if θ̂j > d∗j where:

d∗j = N−1j

hNJd∗J − (NJ − Nj)θ − zα

pNJ − Nj

i

RCTdesign




Regulatory guidance





I Suppose in the sepsis trial we used the following stoppingcriteria:

I Final critical value is d∗J = −0.0435.I Decide futility as long as the power for changing our mind is

smaller than 10% (zα = −1.28).I Calculate this power under θ = 2× dJ = −0.087.

I The following futility stopping criteria result from the abovecalculations:

d∗1 = 0.1539

d∗2 = 0.0273

d∗3 = −0.0161

d∗4 = −0.0435

Notice the degree conservatism. Is it ethical?

RCTdesign




Regulatory guidance





I Suppose we instead use zα = 0; i.e., a 50% chance ofchanging our mind. Then:

d∗1 = 0.0870

d∗2 = 0.0000

d∗3 = −0.0290

d∗4 = −0.0435

I Notice the degree conservatism. Is it ethical?I Do these stopping rules look familiar?I Is it reasonable to use zα = 0?

RCTdesign




Regulatory guidance




Conditional power (design family)

I It is instructive to construct a conditional futility designfamily in the standardized scale:

I Standardized mean: δ = θ−θ0√V/NJ

I Standardized statistic: δ̂j ∼ N (δ, 1Πj

)

I Decision criteria:

δj > dj ⇒ Decide for superiority

δj < aj ⇒ Decide for lack of superiority

Let dJ = aJ = G.

RCTdesign




Regulatory guidance





I Conditional power decision rules:I Stop for efficacy if Pδ=0(δ̂J < G|δj = dj) = α, which implies

dj = (G + z1−α

p1− Πj)Π

−1j

I Stop for futility if Pδ=δ+(δ̂J > G|δj = aj) = α, which implies:

aj = 2G − (G + z1−α

p1− Πj)Π

−1j

I As in other families, G can be selected to control operatingcharacteristics (see R-code in file session7fut.R).

RCTdesign




Regulatory guidance





I Example: Suppose Πj = 0.25, 0.5, 0.75, 1.0.The values of G satisfying:

J∑j=1

P(δ̂j > dj |δ = 0) = 0.025 (type I error)

J∑j=1

P(δ̂j < aj |δ = 2G) = 1− 0.025 (type II error)

are:Values of dj for selected α

Πj α = 0.10 α = 0.35 α = 0.500.25 12.2815 9.2302 8.01290.50 5.7334 4.4926 4.00650.75 3.4684 2.8887 2.67101.00 1.9605 1.9739 2.0032

RCTdesign




Regulatory guidance





I Stopping boundaries:

0.4 0.6 0.8 1.0

−10

−5

05

1015

Sample size

Diff

eren

ce in

Mea

ns

CF.10CF.35

CF.50

RCTdesign




Regulatory guidance





I Sample Size:

0 1 2 3 4

0.6

0.7

0.8

0.9

1.0

Difference in Means

Sam

ple

Siz

e

Average Sample Size

CF.10CF.35CF.50

0 1 2 3 4

0.6

0.7

0.8

0.9

1.0

Difference in Means

Sam

ple

Siz

e

75th percentile

CF.10CF.35CF.50

RCTdesign




Regulatory guidance





I Power:

0 1 2 3 4

0.0

0.2

0.4

0.6

0.8

1.0

Theta

Pow

er (

Upp

er)

CF.10CF.35

CF.50

RCTdesign




Regulatory guidance




Conditional power (Sepsis example)Comparing inference at lower (efficacy) boundary:

I Design: DSMB2Bias-adjusted inference

IA (N) aj δ̂j 95% CI p-value1 (N = 425) -0.175 -0.166 (-0.228, -0.090) 0.00002 (N = 850) -0.087 -0.081 (-0.132, -0.027) 0.00203 (N = 1275) -0.058 -0.055 (-0.097, -0.008) 0.01074 (N = 1700) -0.044 -0.044 (-0.086, -0.001) 0.0225

I Design: Conditional Futility (α = 0.1):Bias-adjusted inference

IA (N) aj δ̂j 95% CI p-value1 (N = 425) -0.266 -0.258 (-0.319, -0.182) 0.00002 (N = 850) -0.124 -0.120 (-0.168, -0.064) 0.00003 (N = 1275) -0.075 -0.072 (-0.114, -0.026) 0.00124 (N = 1700) -0.043 -0.043 (-0.085, -0.001) 0.0225

RCTdesign




Regulatory guidance




Conditional power (Sepsis example)Comparing inference at upper (futility) boundary:

I Design: DSMB2Bias-adjusted inference

IA (N) dj δ̂j 95% CI p-value1 (N = 425) 0.087 0.079 ( 0.003, 0.141) 0.97892 (N = 850) 0.000 -0.006 (-0.060, 0.044) 0.40133 (N = 1275) -0.029 -0.032 (-0.079, 0.010) 0.06364 (N = 1700) -0.044 -0.044 (-0.086, -0.001) 0.0225

I Design: Conditional Futility (α = 0.1):Bias-adjusted inference

IA (N) dj δ̂j 95% CI p-value1 (N = 425) 0.180 0.172 ( 0.096, 0.234) 1.00002 (N = 850) 0.039 0.034 (-0.022, 0.083) 0.88253 (N = 1275) -0.010 -0.014 (-0.060, 0.028) 0.25064 (N = 1700) -0.043 -0.043 (-0.085, -0.001) 0.0225

RCTdesign




Regulatory guidance




Conditional power (Summary)

I When compared with an O’Brien-Fleming design, theconditional power family has the following characteristics:

I Nearly identical powerI Much lower stopping probability at early interim analysesI Much larger ASN (loss of efficiency)I Extreme conservatism in making any early decision

I Resulting questions:I Is it ethical to continue in the presence of overwhelming

evidence of harm (or benefit)?I Does conditional power obfuscate the essential

clinical/scientific issues in deciding whether to terminate atrial?

RCTdesign




Regulatory guidance




Conditional power (Summary)

I Note: There are “fixes" for the problems with conditionalpower:

I Calculate conditional power under a different value of δ;e.g.,:

I Calculate Pδ=δ+/2(δ̂J < G|δ̂j = dj ) instead ofPδ=0(δ̂J < G|δ̂j = dj ).

I Calculate Pδ=δ+/2(δ̂J > G|δ̂j = aj ) instead ofPδ=δ+ (δ̂J > G|δ̂j = aj ).

I Calculate conditional power under the MLE δ̂j :I Calculate Pδ=δ̂j

(δ̂J < G|δ̂j = dj ) instead of

Pδ=0(δ̂J < G|δ̂j = dj ).I Calculate Pδ=δ̂j

(δ̂J > G|δ̂j = aj ) instead of

Pδ=δ+ (δ̂J > G|δ̂j = aj ).I (Consider predictive power.)

RCTdesign




Regulatory guidance




Predictive power (overview)

I Conditional power relied on a specific choice for δ:I Efficacy: Pδ=0(δ̂J < G|δ̂j = dj).I Futility: Pδ=δ+(δ̂J > G|δ̂j = aj).

I Predictive power integrates over a posterior distribution forδ:

I Let λ0(δ) denote a prior distribution for δI Let λ(δ|δ̂j) denote the posterior distribution of δ at the j th

interim analysis.I Predictive probability:

P(δ̂J > G|δ̂j) =

Zu

Pδ=u(δ̂J > G|δ̂j)λ(u|δ̂j)du

I Decision criteria can be defined based on the magnitude ofthe predictive probability.

RCTdesign




Regulatory guidance




Concluding remarks

I There are foundational issues with both conditional andpredictive power. Neither frequentist nor Bayesianfoundations construct inference around the probability ofchanging our mind (see paper).

I Underlying principles illustrated with this discussion ofstochastic curtailment:

I All designs can be expressed as conditions on the observedestimate.

I Always consider statistical inference on the boundary whenevaluating stopping criteria.

RCTdesign




Regulatory guidance



Designs meriting special consideration:Non-inferiority designs

Non-inferiority designs

I Regulatory guidance (FDA)I Example 1: Daptomycin

I Fixed-sample designI Trial monitoringI Evaluation

I Example 2: Rivaroxaban in non-valvular atrial fibrillation

RCTdesign




Regulatory guidance



Designs meriting special consideration


Recall reasons for early termination (session 3)I Why would you want to stop a study early?

I Superiority study:I For superiority (reject H0 : θ ≤ 0)I For lack of superiority (reject HA : θ ≥ θ+)

I Approximate equivalence study:I For lack of inferiority (reject H0 : θ ≤ θ−)I For lack of superiority (reject HA : θ ≥ θ+)

I Non-inferiority study:I For lack of inferiority (reject H0 : θ ≤ θ−)I For inferiority (reject HA : θ ≥ 0)

I Equivalence (2-sided) study:I For superiority (reject θ ≤ 0)I For inferiority (reject θ ≥ 0)I For both non-inferiority and non-superiority (reject both

θ ≤ θ− and θ ≥ θ+)

RCTdesign




Regulatory guidance



FDA Regulatory Guidance

FDA Regulatory Guidance (http://www.fda.gov/downloads/Drugs/.../Guidances/UCM202140.pdf)

RCTdesign




Regulatory guidance



FDA Regulatory Guidance

III. General Consideration of NI studies:

A. Basic principles of a NI study1. Superiority vs NI2. Logic of NI trial3. Reasons for using NI design4. The non-inferiority margin5. Assay sensitivity and choosing NI margin6. Regulatory conclusions

B. Practical Considerations for use of NI designs1. Consider alternative designs2. Number of studies needed3. Statistical inferences4. Choice of active control5. Choice of NI method

RCTdesign




Regulatory guidance




A. Basic principles

A. Basic principles of a NI study1. Superiority vs NI

I Superiority study: Interpretable in their own right ... resultsspeak for themselves without additional assumption.

I Non-inferiority study: Requires knowing something that is notmeasured in the study; that is:

The active control had its expected effect in the NI study.Assay sensitivity (think power):

I Trial could have distinguished an effective from an ineffectivedrug.

I Establishing NI is meaningless without assay sensitivity.I Knowing if the trial has assay sensitivity relies on external

information.

RCTdesign




Regulatory guidance




2. Logic of the NI trialI Superiority study: establish significant benefit over placebo:

RCTdesign




Regulatory guidance



III. General Consideration of NI studies:2. Logic of the NI trial

I Non-inferiority study: benefit at least as large as controlsuperiority:

RCTdesign




Regulatory guidance




2. Logic of the NI trialI Non-inferiority study: Choice of non-inferiority margin (M1)

(1) Treatment effect based on historical experience with theactive control drug

(2) Assessment of the likelihood that the current effect of theactive control is similar to the past effect (the constancyassumption)

(3) Assessment of the quality of the NI trial, particularly lookingfor defects that could produce bias toward the null.[Note: for this reason the size of M1 cannot usually beassessed until the trial is finished.]

RCTdesign




Regulatory guidance




I “Constancy Assumption"I Historic evidence of sensitivity to drug effects (HESDE)

(ICH E-10)I HESDE means that past trials that used a specific active

treatment regularly showed this treatment to be superior toplacebo.

I The conclusion that HESDE applies only when the NI studyis sufficiently similar to the past studies:

I Characteristics of the patient population.I Concomitant treatments.I Definitions and ascertainment of study endpointsI Dose of active control, entry criteria, analytic approaches.

RCTdesign




Regulatory guidance




I Summary commentsI Beware of ‘bias toward the null’ in a non-inferiority trial:

I Inflated variance (⇒ poor assay sensitivity):Outcome ascertainmentAdequate blinding (central adjudication)Compliance

I Missing data!!I “Biocreep" or “Noninferiority creep":

I Repeated comparisons to latest newest treatment results insteady drift to inferiority relative to placebo.

I Avoid NI trials whenever possible.I “If it is ethical to conduct a placebo-controlled trial, then it is

unethical not to" (Lloyd Fisher).

RCTdesign




Regulatory guidance



Non-inferiority trials

I Two examples:I Daptomycin for Staph infection

I Trial results: Fowler, VG, et.al. NEJM (2006); 355(7):653-65I Editorial: Grayson, ML. NEJM (2006); 355(7):724

I Rivaroxaban for atrial fibrillationI Trial results: Patel, MR, et.al. NEJM (2011); 365: 883-91I Editorial: Fleming, TR, Emerson SS. NEJM (2011); 365(17):

1557

RCTdesign




Regulatory guidance




Example: Daptomycin NEJM (2006); 355(7):653-65

Structuring parameter space:I Background: Patients with bacteremia and endocarditis

caused by S. aureus need alternative antibiotic therapies.Current antibiotic treatments are successful in about 65%of the cases.

I Clinical question: Can daptomycin be used as anotherapproach for treating S. aureus bacteremia?

I Study interventions: open label assignment toI Standard antibiotic careI Daptomycin

I Primary outcome: Clinical success at 42 days (failure = noresponse to drug based on ongoing signs and symptoms)

I Functional: θ1, θ0: population success probability withdaptomycin and standard care

I Contrast: θ = θ1 − θ0

RCTdesign




Regulatory guidance





Trial information:I Non-inferiority definition: “Lower bound of the 95%

confidence intervals must be within the prespecifiednoninferiority margin of 20 percent and the upper boundscontaining 0 percent."

I Sample size: 90 patients per group gives 80% power forNI if θ0 = θ1 = 0.65.

I Sample size evaluation: If θ0 = θ1 = 0.65, then the 95% CIhas half-width:

1.96

√(0.65× 0.35)

290

= 0.14

RCTdesign




Regulatory guidance



Structuring parameter space: Non-inferiority designsI What should we decide with an infinite sample size (θ is

known)?

H

G

F

E

D

C

B

A

Pot

entia

l tria

l res

ults

(in

finite

sam

ple

size

)

||

|

NoDifference

θ∅

ClinicallyImportant

Benefitθ+

ClinicallyImportant

Harmθ−

Inferiority Superiority

ClinicalInferiority

θ < θ−

ClinicalSuperiority

θ > θ+

RCTdesign




Regulatory guidance



Structuring parameter space: Non-inferiority designs


I What should we decide with an infinite sample size(θ known)?

A, B, C, D ⇒ Superior (use daptomycin)E ⇒ Unimportant inferiority (use daptomycin?)

F, G, H ⇒ Inferior (do not use daptomycin)

I May be willing to accept some inferiority:I Diversity of antibiotics may avoid evolution of resistant

strainsI Unresponsive patients may respond to a different treatment

RCTdesign




Regulatory guidance





I Other considerationsI In the marginal situations the acceptability of daptomycin

would also consider:* The clinical “importance" of a difference, which is difficult to

quantify.* Nature of the primary endpoint (e.g., survival vs treatment

success).* Effects on secondary endpoints (toxicity, rescue therapies)?* Potential for long-term effects that could not be measured in

this trial.I Bottom line:

* There are grey-areas in parameter space.* These will affect the design (e.g., particularly in non-inferiority

trials).I Remark: A question that cannot be answered with an

infinite amount of data cannot be answered with a finiteamount of data.

RCTdesign




Regulatory guidance



Structuring parameter space: Non-inferiority designsI What should we decide with a finite sample size

(θ is estimated with uncertainty)?

( )

( )

( )

( )

( )

( )

( )

( )

H

G

F

E

D

C

B

A

Pot

entia

l tria

l res

ults

(co

nfid

ence

inte

rval

s)

||

|

NoDifference

θ∅

ClinicallyImportant

Benefitθ+

ClinicallyImportant

Harmθ−


ClinicalInferiority

θ < θ−

ClinicalSuperiority

θ > θ+

RCTdesign




Regulatory guidance





I What should we decide with an infinite sample size(θ known)?

A, B, C, D ⇒ Superior (use daptomycin)E, F ⇒ Unimportant inferiority (use daptomycin?)G, H ⇒ Significant inferiority (do not use daptomycin)

RCTdesign




Regulatory guidance




Superiority/non-superiority studies

I Application: Evaluating new therapies to determine if theyincrease benefits.

I Defining Hypotheses:I Inferiority: θ ≤ θ∅

(Decide for superiority if inferiority is rejected)I Clinical Superiority: θ ≥ θ+

(Decide against superiority if clinical superiority isrejected)

I Example: Suppose we hypothesize daptomycin is superiorto standard care and we want to discriminate betweenθ ≤ θ∅ and θ ≥ θ+:

Inferiority : θ ≤ θ∅ = 0.0Clinical superiority : θ ≥ θ+ = 0.28

RCTdesign




Regulatory guidance




Non-inferiority/inferiority studies

I Applications:I Evaluating new therapies to determine if they are

non-inferior to existing therapy.I Evaluating existing therapies to determine if they are

harmful.I Defining Hypotheses:

I Clinical Inferiority: θ ≤ θ−(Decide for non-inferiority if clinical inferiority is

rejected)I Superiority: θ ≥ θ∅

(Decide against non-inferiority if superiority is rejected)I Example: Suppose daptomycin is in routine use, but there

is some evidence it may be harmful, and we want todiscriminate between θ ≤ θ− and θ ≥ θ∅:

Clinical Inferiority : θ ≤ θ− = −0.28Superiority : θ ≥ θ∅ = 0.0

RCTdesign




Regulatory guidance




Equivalence studies

I Application:I Evaluating therapies to determine if they can be used

interchangeably.I (If not, then to select the best therapy.)

I Defining Hypotheses:I Clinical inferiority (θ ≤ θ−)I Clinical superiority (θ ≥ θ+)I Equality (θ = θ∅)I Decisions

Decide equivalence if reject both clinical inferiority andsuperiority.Decide treatment A better than B if reject inferiority (of A).Decide treatment B better than A if reject superiority (of A).

I Example: Daptomycin vs standard therapy:Clinical Inferiority (of daptomycin): θ ≤ −0.28Clinical Superiority (of daptomycin): θ ≥ 0.28Equality: θ = 0

RCTdesign




Regulatory guidance




Types of equivalence (iroically: not all equivalents are the same)

I Categories of equivalence studies include:I Bioequivalence: showing that two formulations of the same

drug are equivalent.

Reject θ ≥ θ∅ ⇒ Conclude inferior

Reject θ ≤ θ∅ ⇒ Conclude superior

Reject θ ≤ θ− and θ ≥ θ+ ⇒ Conclude equivalence

In some settings FDA defines bioequivalence as0.8 < θ1/θ0 < 1.25.

RCTdesign




Regulatory guidance




Types of equivalence (iroically: not all equivalents are the same)

I Categories of equivalence studies include:I Non-Inferiority:

Reject θ ≥ θ∅ ⇒ Conclude inferior

Reject θ ≤ θ− ⇒ Conclude non-inferior

I Approximate equivalence: Require that observed effect bebeneficial and important inferiority is rejected:

Observe θ̂ > θ∅ ⇒ Conclude non-inferior

Reject θ ≤ θ− ⇒ Conclude non-inferior

RCTdesign




Regulatory guidance




Non-inferiority trials of new therapeutics:public health implications

I Notes (see figure below):I Non-inferiority creep:

I Suppose treatment A replaces an existing treatment in a strictnon-inferiority design (as above).

The point estimate may indicate inferior (θ̂ < θ∅)I Now suppose treatment B replaces treatment A in a strict

non-inferiority design.The point estimate may indicate inferior (θ̂ < θ∅)

I If this process continues, there may be a steady creep thatresults in increasingly inferior treatments.

I Confidence interval A on the following figure illustrates how aninferior point estimate could produce a non-inferior decision.

RCTdesign




Regulatory guidance





I Notes (see figure below):I Approximate equivalence designs:

I Requiring a positive point estimate (confidence interval B)would avoid non-inferiority creep.

I Requiring a positive point estimate and requiring rejection ofθ ≤ θ− would allow a smaller sample size (confidenceinterval C).

I Note: Sample size may be relaxed somewhat, but not asmuch as the difference illustrated in confidence intervals Band C.

I Note: Requiring a positive point estimate means that there isonly 50% power for a treatment that is truly equivalent(θ = θ∅).

RCTdesign




Regulatory guidance





I Daptomycin example (see figure below)I Approximate equivalence design (confidence interval B)

I Non-inferiority bound = −14%.I Sample size: N = 180 (90 per group)

⇒ critical value = 0%.I Reported trial design (confidence interval D)

I Non-inferiority bound = −20%.I Sample size: N = 180 (90 per group)

⇒ critical value = −6%.I Actual trial results (confidence interval E)

I N = 235 (N = 120 daptomycin; N = 115 standard care)I Point estimate: 2.4%I CI: -10.2% to 15.1%

RCTdesign




Regulatory guidance



Structuring parameter space: Non-inferiority designsI Potential ways to specify non-inferiority decisions

( )

( )

( )

( )

( )

C

B

A

Pot

entia

l tria

l res

ults

E

D

Dap

tom

ycin

tria

l

||

|

NoDifference

θ∅

ClinicallyImportant

Benefitθ+

ClinicallyImportant

Harmθ−


ClinicalInferiority

θ < θ−

ClinicalSuperiority

θ > θ+

RCTdesign




Regulatory guidance




Bottom line

I Design of equivalence trials (including non-inferiority trials)requires careful consideration of the standards that definethe decision.

I Poorly run equivalence trials can introduce bias toward thenull hypothesis (an equivalence conclusion).(See editorial with daptomycin paper.)

RCTdesign




Regulatory guidance



Monitoring plan for non-inferiority trial


I Comparison of designs (illustrated using daptomycin):I Two-sided bioequivalence designI Shifted superiority designI Approximate equivalence designI Approximate equivalence with a single boundary

RCTdesign




Regulatory guidance




A not-so-hypothetical example

Stage 1: Investigator discovers bioequivalenceI Investigator hypothesizes that daptomycin might work for

sepsis:I “Knows" it cannot be superior, so wants to show it is

equivalent to standard careI Finds a website about bioequivalence and when he plugged

in his numbers it told him:* Use 90 patients per group and measure the difference in

success proportions.* Conclude equivalence if the 95% CI for the difference in

proportions is contained entirely between -0.28 and 0.28.

I A research assistant says he should really use anon-inferiority design.

I A clinical trialist says he should worry about non-inferioritycreep.

I Investigator is seeking help from the biostatisticians -everyone is confused ...

RCTdesign




Regulatory guidance





Stage 2: The research assistant says...I A non-inferiority website says the design has 80% power

for equivalence.I You end up having to explain/compare these approaches,

and begin with the fixed-sample design:I 2-sided bioequivalence design(dap.2side)

I 1-sided superiority design(dap.fixSup)

I 1-sided shifted superiority design (eventually you reproducetheir design)(dap.fixShift)

I 1-sided “approximate equivalence" design(dap.fixNI)

RCTdesign




Regulatory guidance





Stage 3: Enter the biostatisticiandap.2side <- seqDesign(prob.model=’proportions’,arms=2, null.hypothesis=c(0.65,0.65),alt.hypothesis="calculate", test.type=’two.sided’,variance="null", alpha=0.025, sample.size=180,power=0.975, nbr.analyses=1)

dap.fixSup <- seqDesign(prob.model=’proportions’,arms=2, null.hypothesis=c(0.65,0.65),alt.hypothesis="calculate", test.type=’greater’,variance="null", alpha=0.025, sample.size=180,power=0.975, nbr.analyses=1)

e <- 0.715

dap.fixShift <- update(dap.fixSup,epsilon=c(e,1-e))

dap.fixNI <- update(dap.fixSup,epsilon=c(0.5,0.5))

RCTdesign




Regulatory guidance




I Design critical values (fixed sample designs):

Comparison of decision criteria (180 patients):DesignName aJ bJ cJ dJ

(dap.2side) -0.14 -0.14 0.14 0.14(dap.fixSup) 0.14 NA NA 0.14(dap.fixShift) -0.06 NA NA 0.06(dap.fixNI) 0 NA NA 0

I Inference:95% CI have width ±0.14 centered at the critical value

RCTdesign




Regulatory guidance



Monitoring plan for non-inferiority trialI Power curves (fixed-sample designs:

−0.3 −0.2 −0.1 0.0 0.1 0.2 0.3 0.4

0.0

0.2

0.4

0.6

0.8

1.0

Difference in Proportions

Pow

er (

Upp

er)

dap.fixSupdap.fixShift

dap.fixNI

(dap.2side and dap.fixSup have same upper power curve)

RCTdesign




Regulatory guidance





Stage 4: Study section requests interim analysesI Add interim analyses:

dap.2sideIA <-update(dap.2side,sample.size=c(60,120,180))

dap.SupIA <-update(dap.fixSup,sample.size=c(60,120,180))

dap.ShiftIA <-update(dap.fixShift,sample.size=c(60,120,180))

dap.NI.IA <-update(dap.fixNI,sample.size=c(60,120,180))

RCTdesign




Regulatory guidance





Stage 5: Focus switches to inferiority decisionI Ethics demands rapid termination for inferiorityI Increasing sensitivity of lower (aj ) boundary

dap.NI.IAasym<-update(dap.fixNI,sample.size=c(60,120,180),

P=c(0.8,Inf,Inf,1))

I Now the steering committee decides it is only necessary tostop for inferioritydap.NI.IAinf<-update(dap.fixNI,sample.size=c(60,120,180),P=c(0.8,Inf,Inf,Inf))

RCTdesign




Regulatory guidance





Stage 6: Evaluate design propertiesI Stopping boundaries, power, ASN, stopping probabilities,

inference at boundariesseqPlotBoundary(dap.NI.IA,dap.NI.IAasym,dap.NI.IAinf,fixed=F,col=c(1,2,4))

seqPlotPower(dap.NI.IA,dap.NI.IAasym,dap.NI.IAinf,fixed=F,col=c(1,2,4))

seqPlotASN(dap.NI.IA,dap.NI.IAasym,dap.NI.IAinf,fixed=F,col=c(1,2,4),prob=NULL)

RCTdesign




Regulatory guidance



Monitoring plan for non-inferiority trialProperties: stopping boundaries

0 50 100 150

−0.

4−

0.2

0.0

0.2

0.4

0.6

Sample Size

Diff

eren

ce in

Pro

port

ions

● dap.NI.IA● dap.NI.IAasym

● dap.NI.IAinf

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

RCTdesign




Regulatory guidance



Monitoring plan for non-inferiority trialProperties: power

−0.15 −0.10 −0.05 0.00 0.05 0.10 0.15

0.0

0.2

0.4

0.6

0.8

1.0


Pow

er (

Upp

er)

dap.NI.IAdap.NI.IAasym

dap.NI.IAinf

RCTdesign




Regulatory guidance



Monitoring plan for non-inferiority trialProperties: ASN

−0.15 −0.10 −0.05 0.00 0.05 0.10 0.15

100

120

140

160

180


Sam

ple

Siz

e

Average Sample Size

dap.NI.IAdap.NI.IAasymdap.NI.IAinf

RCTdesign




Regulatory guidance



Example 2: Rivaroxaban is a new anticoagulation therapy

Rivaroxaban

I Anticoagulation therapy is used in a wide range ofsettings:

I Atrial fibrillation for prevention of thrombus and/or strokeI Thromboprophyllaxis in patients undergoing orthopedic

surgery (knee or hip replacement).I Cancer chemotherapy patients

I Standard anticoagulation therapy = warfarin (coumadin)I Warfarin has a narrow therapeutic range:

I Too much causes bleeding, too little does not prevent clots.I Patients on warfarin must monitor their clotting (INR)

I Rivaroxaban is a new oral anticoagulantI Easier to control amount of anticoagulationI Might be preferred over warfarin if it:

I Has equal ability to reduce blood clots/strokes.I Does not increase bleeding risk.I Clinical trials for a wide range of indications are ongoing.

RCTdesign




Regulatory guidance



Example 2: ROCKET-AF trial

Rivaroxaban for non-valvular atrial fibrillation

I Atrial Fibrillation:I AF is the most common cardiac arrhythmia; symptoms

include a “racing" heart.I Can cause incomplete emptying of the left atrium.I Pooled blood in the atrium can lead to clots and stroke.I It is common to treat atrial fibrillation with anticoagulation

therapy to prevent clots.I Standard anticoagulation therapy = warfarin (coumadin)I Can Rivaroxaban be used instead of warfarin for AF?

I ROCKET-AF: large clinical trial to evaluate noninferiority ofrivaroxaban when compared with warfarin.

I Key references:I Trial design: Am Heart J 2010; 159: 340-347.e1.I Trial results: Patel, MR, et.al. NEJM (2011); 365: 883-91I Perspective: Fleming, TR, Emerson SS. NEJM (2011);

365(17): 1557

RCTdesign




Regulatory guidance




ROCKET-AF design synopsis

I Patient population:I Patients with non-valvular AF at risk for stroke.

I Treatment groups:I Warfarin (standard care):

I Titrated to target INR of 2.5 (range 2.0-3.0).I Plus oral rivaroxaban placebo

I Rivaroxaban: 20mg od:I Warfarin placebo od titrated to sham INR of 2.5 (range

2.0-3.0).

All patients have screening period, double-blind treatmentperiod, posttreatment observation periodAll patients treated according to local standard of care.

I EndpointsI Primary efficacy endpoint: composite of all-cause stroke

and systemic embolism.I Primary safety endpoint: composite of major and clinically

relevant non-major bleeding.

RCTdesign




Regulatory guidance




Analysis and sample size

I Analysis plan:I Per protocol population (not intention to treat)

I PP instead of ITT is not uncommon in noninferiority trials.I Lack of compliance or failure to complete treatment can

introduce bias toward the null.I JK: both PP and ITT results should be reported.I JK: missing data must be avoided!

I Primary efficacy analysis seeks 363 events:I 363 events gives 95% power for hazard ratio of θ = 1.46 in a

one-side level α = 0.025 logrank test:

363 =

»1.96 + 1.645

log(1.46)

–2× 4

I Plan to enroll 14,000 patients to get 405 events:I Allow for 14% dropoutI Allow for “robust evaluation across subgroups"

RCTdesign




Regulatory guidance




Noninferiority margin

I Noninferiority margin:I Previous meta-analysis of 6 trials comparing warfarin to

placebo:I Relative risk (warfarin/placebo): 0.38 (95% CI: 0.28-0.52).I Relative risk (placebo/warfarin): 2.63 (95% CI: 1.92-3.57).

I Seek to preserve 50% of previous warfarin benefit:I Calculation of pervious benefit (subtract 1 from above RR):

1.63 (95%CI : 0.92, 2.57)

I Half of previous benefit (add 1 to half of above):

1.82 (95%CI : 1.46, 2.29)

I “The most conservative approach was chosen, selecting thelower limit, 1.46, of this confidence boundary."

RCTdesign




Regulatory guidance




Sample size evaluation

I Asymptotic standard error is se =√

4363 = 0.105:

I Superiority decision:I Critical value: e−1.96×0.105 = 0.814I CI at critical value: e(−3.92×0.105,0) = (0.661, 1.0)I Concluding non-superiority rules out HR (θ) less than 0.661.I Concluding superiority rules out HR (θ) larger than 1.0.

I Noninferiority decision:I CI when θ̂ = 1.0: e±1.96×0.105 = (0.814, 1.23)I CI when θ̂ = 1.123: elog(1.89)±1.96×0.105 = (0.914, 1.380)

I Inferiority decision:I CI when θ̂ = 1.228: elog(1.228)±1.96×0.105 = (1.00, 1.509)

RCTdesign




Regulatory guidance



Example 2: ROCKET-AF trial results

Trial results: Patel, MR, et.al. NEJM (2011); 365: 883-91

I Primary analysis event rates:I Rivaroxaban: (188 patients) 1.7% per yearI Warfarin: (241 patients) 2.2% per yearI Hazard ratio: 0.79 (95% CI: 0.66 to 0.96); p < 0.001 for

noninferiority.I Intention-to-treat event rates:

I Rivaroxaban: (269 patients) 2.1% per yearI Warfarin: (306 patients) 2.4% per yearI Hazard ratio: 0.88 (95% CI: 0.74 to 1.03); p = 0.12 for

superiority.I Bleeding risk (major and non-major clinically relevant):

I Rivaroxaban: (1475 patients) 14.9% per yearI Warfarin: (1449 patients) 14.5% per yearI Hazard ratio: 1.03 (95% CI: 0.96 to 1.11); p = 0.44.

RCTdesign




Regulatory guidance



Example 2: ROCKET-AF Regulatory considerations

Perspective: Fleming and Emerson. NEJM (2011); 365(17): 1557

I Design synopsis:I NI margin = 1.38: rivaroxaban be superior to placebo by at

least 50% of the margin by which warfarin is superior toplacebo.

I Per-protocol “ on-treatment" analysis pre-specified becauseof concerns about assay sensitivity (bias toward null).

I FDA issues in interpretation:I Constancy assumption must be satisfied: control treatment

with same magnitude of benefit as in reference trials.I Concerns over non-constancy:

I Higher risk patients enrolled in ROCKET-AFI More than 5% who discontinued due to withdrawal of consent.I INR in therapeutic range (between 2 an 3) only 55% of the

time; other trials have 62-73% in therapeutic range.I Per-randomization analysis should also be reported. On

treatment analysis is subject to bias:I Observations truncated at 2 days after treatment

discontinuation.

RCTdesign




Regulatory guidance



Summary Remarks: Non-inferiority Trials

Non-Inferiority Trials

I General principles in non-inferiority trialsI Constancy assumption

* The effect of the comparator relative to placebo remainsconstant across all trials

* May break down because of changes other aspects such as:- ancillary care- diagnostic methods and patient populations

I Non-inferiority is difficult to objectively define* The “non-inferiority bound":

- depends on the setting (population, treatment, endpoints)- may be driven by cost/feasibility as opposed to public health

I Sloppy science my bias toward non-inferiority* Assure compliance* Endpoints must be clinically important/relevant* Avoid surrogates

I Design principlesI Designs should consider potential inference (CI) rather than

powerI Monitoring designs should evaluate inference on the

boundaries

Documents

Sequential Monitoring of Clinical Trials - Session 7 - Special ...rctdesign.org/ShortCourses/uwAugust2013/session7.pdfNon-Inferiority Designs Fixed-sample non-Inferiority designs Regulatory