Alex DmitrienkoEli Lilly and Company

Brian WiensMyogen, Inc

Branching tests in clinical trials with multiple objectives

[Slide 2]

Gatekeeping proceduresSerial and parallel testingSerial and parallel testing

Branching proceduresBranching proceduresBranching proceduresMultiple tests for clinical trials with Multiple tests for clinical trials with Multiple tests for clinical trials with hierarchically ordered objectiveshierarchically ordered objectiveshierarchically ordered objectivesExtension of gatekeeping methodsExtension of gatekeeping methodsExtension of gatekeeping methods

Clinical trial examplesClinical trial examplesTrial with multiple endpoints and objectives Trial with multiple endpoints and objectives Trial with multiple endpoints and objectives Trial with multiple endpoints and objectives Trial with multiple endpoints and objectives Trial with multiple endpoints and objectives Dose-fi nding trial with multiple endpointsDose-fi nding trial with multiple endpointsDose-fi nding trial with multiple endpoints

Outline

[Slide 3]

Mickey Mouse problem

[Slide 4]

Multiple endpointsTwo co-primaries/one secondary

Primary endpoint 1

p≤0.025

Primary endpoint 2

p≤0.025

Secondary endpoint

p≤0.05

[Slide 5]

Multiple endpointsTwo co-primaries/one secondary

Primary endpoint 1

p≤0.025

Primary endpoint 2

p≤0.025

Secondary endpoint

p≤0.05

Family 1: Bonferroni test

[Slide 6]

Multiple endpointsTwo co-primaries/one secondary

Primary endpoint 1

p≤0.025

Primary endpoint 2

p≤0.025

Secondary endpoint

p≤0.05Family 2: Family 2: Test if at Test if at Test if at least one least one least one primary primary primary endpoint is endpoint is signifi cant signifi cant

[Slide 7]

Multiple endpointsTwo co-primaries/one secondary

Primary endpoint 1

p≤0.025

Primary endpoint 2

p≤0.025

Secondary endpoint

p≤0.05

Type I error rate is inflated 0.025+0.05>0.05

[Slide 8]

Gatekeeping proceduresMultiple testing procedures for sequential Multiple testing procedures for sequential Multiple testing procedures for sequential families of null hypothesesfamilies of null hypothesesfamilies of null hypothesesSerial gatekeeping methods, Westfall and Serial gatekeeping methods, Westfall and Serial gatekeeping methods, Westfall and Krishen (2001)Krishen (2001)Krishen (2001)Parallel gatekeeping methods, Dmitrienko, Parallel gatekeeping methods, Dmitrienko, Parallel gatekeeping methods, Dmitrienko, Off en and Westfall (2003)Off en and Westfall (2003)Off en and Westfall (2003)Parallel gatekeeping methods with logical Parallel gatekeeping methods with logical restrictions, Chen, Luo and Capizzi (2005)restrictions, Chen, Luo and Capizzi (2005)restrictions, Chen, Luo and Capizzi (2005)restrictions, Chen, Luo and Capizzi (2005)

General overviewDmitrienko et al (2005, Chapter 2)Dmitrienko et al (2005, Chapter 2)Dmitrienko et al (2005, Chapter 2)

Gatekeeping methods

[Slide 9]

Gatekeeping methodsSerial versus parallel strategies

Serial strategy (Rheum arthritis)

Endpoint 1: Signs and symptoms

Endpoint 2: Disease

progression

Endpoint 3: Physical function/

disability

Parallel strategy (Acute lung injury)

Endpoint 1: Lung

function

Endpoint 2: Mortality

rate

Endpoint 3: Quality of life

Endpoint 4: ICU-free

days

[Slide 10]

Branching methodsExtension of gatekeeping methods

Serial strategy (Rheum arthritis)

Endpoint 1: Signs and symptoms

Endpoint 2: Disease

progression

Endpoint 3: Physical function/

disability

Parallel strategy (Acute lung injury)

Endpoint 1: Lung

function

Endpoint 2: Mortality

rate

Endpoint 3: Quality of life

Endpoint 4: ICU-free

days

Branching methodsTrial designs are becoming increasingly more complexClinical researchers explore complex testing strategies

ExamplesTwo- or three-dimensional rather than simple sequential strategiesLogical restrictions

[Slide 11]

DesignExperimental drug versus active controlExperimental drug versus active controlExperimental drug versus active control

Four endpointsFour endpointsFour endpointsPrimary (P): Systolic blood pressurePrimary (P): Systolic blood pressurePrimary (P): Systolic blood pressureSecondary (S1 and S2): Diastolic blood Secondary (S1 and S2): Diastolic blood Secondary (S1 and S2): Diastolic blood pressure and proportion of patients with pressure and proportion of patients with pressure and proportion of patients with controlled systolic/diastolic blood pressurecontrolled systolic/diastolic blood pressureTertiary (T): Average blood pressure based Tertiary (T): Average blood pressure based Tertiary (T): Average blood pressure based Tertiary (T): Average blood pressure based Tertiary (T): Average blood pressure based on ambulatory blood pressure monitoringon ambulatory blood pressure monitoringon ambulatory blood pressure monitoring

Noninferiority vs superiorityNoninferiority vs superiorityNoninferiority vs superiority

Clinical trial examplesHypertension trial

[Slide 12]

Hypertension trialDecision tree

PNoninferiority

S1Noninferiority

PSuperiority

S2Noninferiority

S1Superiority

TNoninferiority

S2Superiority

TSuperiority

P=Primary, S1 and S2=Secondary, T=Tertiary endpointsP=Primary, S1 and S2=Secondary, T=Tertiary endpointsP=Primary, S1 and S2=Secondary, T=Tertiary endpointsP=Primary, S1 and S2=Secondary, T=Tertiary endpointsP=Primary, S1 and S2=Secondary, T=Tertiary endpointsP=Primary, S1 and S2=Secondary, T=Tertiary endpointsP=Primary, S1 and S2=Secondary, T=Tertiary endpointsP=Primary, S1 and S2=Secondary, T=Tertiary endpoints

[Slide 13]

DesignThree doses (L, M and H) versus placebo (P)Three doses (L, M and H) versus placebo (P)Three doses (L, M and H) versus placebo (P)Three doses (L, M and H) versus placebo (P)

Three endpointsThree endpointsThree endpointsPrimary (P): Hemoglobin A1cPrimary (P): Hemoglobin A1cPrimary (P): Hemoglobin A1cSecondary (S1 and S2): Fasting serum Secondary (S1 and S2): Fasting serum Secondary (S1 and S2): Fasting serum glucose and HDL cholesterolglucose and HDL cholesterolglucose and HDL cholesterol

Logical restrictionsLogical restrictions

Clinical trial examplesType II diabetes trial

[Slide 14]

Diabetes trialDecision tree

PL vs P

PM vs P

PH vs P

P=Primary, S1 and S2=Secondary endpointsP=Primary, S1 and S2=Secondary endpointsP=Primary, S1 and S2=Secondary endpointsP=Primary, S1 and S2=Secondary endpointsP=Primary, S1 and S2=Secondary endpointsP=Primary, S1 and S2=Secondary endpointsP=Primary, S1 and S2=Secondary endpointsP=Primary, S1 and S2=Secondary endpointsP=Primary, S1 and S2=Secondary endpoints

S1L vs P

S1M vs P

S1H vs P

S2L vs P

S2M vs P

S2H vs P

[Slide 15]

Closed testing principleMarcus, Peritz and Gabriel (1976)Marcus, Peritz and Gabriel (1976)Marcus, Peritz and Gabriel (1976)Defi ne a branching procedure based on Defi ne a branching procedure based on Defi ne a branching procedure based on Bonferroni testBonferroni testCompute multiplicity-adjusted p-valuesCompute multiplicity-adjusted p-valuesCompute multiplicity-adjusted p-values

Branching framework

[Slide 16]

Gatekeeping setsGatekeepers specifi c to each null Gatekeepers specifi c to each null Gatekeepers specifi c to each null hypothesisParallel gatekeeping and serial Parallel gatekeeping and serial Parallel gatekeeping and serial gatekeeping sets for each null hypothesisgatekeeping sets for each null hypothesisgatekeeping sets for each null hypothesis

Gatekeeping sets

[Slide 17]

Serial gatekeeping set

PL vs P

PM vs P

PH vs P

S1L vs P

S1M vs P

S1H vs P

S2L vs P

S2M vs P

S2H vs P

S1M vs P

S1H vs P

S2M vs P

S2H vs P

Null hypothesis HSerial gatekeeping set: All null hypotheses must be rejected in this set to test H

[Slide 18]

Parallel gatekeeping set

PNoninferiority

S1Noninferiority

PSuperiority

S2Noninferiority

S1Superiority

TNoninferiority

S2Superiority

TSuperiority

[Slide 18]

TSuperiority

Parallel gatekeeping set for H: At least one null hypothesis must be rejected in this set to test H

[Slide 19]

Hypertension trialDecision tree

H11P, Noninf

H21S1, Noninf

H23P, Super

H22 S2, Noninf

H31S1, Super

H33T, Noninf

H32S2, Super

H41T, Super

[Slide 20]

Null hypothesis Parallel setParallel set

H11 (P, Noninf ) NA

H21 (S1, Noninf )H21 (S1, Noninf )H21 (S1, Noninf ) H11

H22 (S2, Noninf )H22 (S2, Noninf )H22 (S2, Noninf ) H11

H23 (P, Super)H23 (P, Super) H11

H31 (S1, Super)H31 (S1, Super) H21

H32 (S2, Super) H22

H33 (T, Noninf ) H21, H22H21, H22H21, H22

H41 (T, Super) H33H33

Hypertension trialParallel gatekeeping sets

Serial gatekeeping sets are emptySerial gatekeeping sets are emptySerial gatekeeping sets are emptySerial gatekeeping sets are emptySerial gatekeeping sets are emptySerial gatekeeping sets are empty

[Slide 21]

Hypertension trialMultiplicity-adjusted p-values

P, Noninf0.001 0.001

S1, Noninf0.008 0.024

P, Super0.003 0.009

S2, Noninf0.026 0.078

S1, Super0.208 0.624

T, Noninf0.010 0.045

S1, Super0.302 0.906

T, Super0.578 0.906

Raw p-valuesRaw p-values Multiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-values

[Slide 22]

Diabetes trialDecision tree

Endpoint P H11 H12 H13

H21 H22 H23

H31 H32 H33

Endpoint S1

Endpoint S2

L vs P M vs P H vs P

[Slide 23]

Null hypothesis Serial setSerial setH11 (P, L vs P) NANAH12 (P, M vs P)H12 (P, M vs P) NAH13 (P, H vs P)H13 (P, H vs P)H13 (P, H vs P) NA

H21 (S1, L vs P)H21 (S1, L vs P)H21 (S1, L vs P) H11H22 (S1, M vs P)H22 (S1, M vs P) H12H23 (S1, H vs P)H23 (S1, H vs P) H13H31 (S2, L vs P) H11, H21H32 (S2, M vs P) H12, H22H12, H22H12, H22H33 (S2, H vs P) H13, H23H13, H23

Diabetes trialSerial gatekeeping sets

Parallel gatekeeping sets are emptyParallel gatekeeping sets are emptyParallel gatekeeping sets are emptyParallel gatekeeping sets are emptyParallel gatekeeping sets are emptyParallel gatekeeping sets are empty

[Slide 24]

Diabetes trialBranching strategy

Logical restrictions

Endpoint P 0.0180.054

0.0110.033

0.0050.015

0.0130.054

0.0070.033

0.0090.041

0.0510.054

0.0120.033

0.0100.041

Endpoint S1

Endpoint S2

L vs P M vs P H vs P

Raw p-values Multiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-values

[Slide 25]

No logical restrictions

Diabetes trialParallel gatekeeping strategy

No logical restrictions

Endpoint P 0.0180.054

0.0110.033

0.0050.015

0.0130.054

0.0070.033

0.0090.041

0.0510.054

0.0120.054

0.0100.054

Endpoint S1

Endpoint S2

L vs P M vs P H vs P

Raw p-values Multiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-valuesMultiplicity-adjusted p-values

[Slide 26]

Basic branching frameworkBased on Bonferroni testBased on Bonferroni test

Account for correlationAccount for correlationAccount for correlationCorrelation among multiple endpointsCorrelation among multiple endpointsCorrelation among multiple endpointsCorrelation among multiple dose-control Correlation among multiple dose-control Correlation among multiple dose-control comparisonscomparisonscomparisonsAccount for correlation via resampling Account for correlation via resampling (Westfall and Young, 1993)(Westfall and Young, 1993)

Extensions

[Slide 27]

Branching proceduresEffi cient way to account for hierarchically Effi cient way to account for hierarchically Effi cient way to account for hierarchically ordered multiple objectives in clinical trialsordered multiple objectives in clinical trialsordered multiple objectives in clinical trialsExtend serial and parallel gatekeeping Extend serial and parallel gatekeeping Extend serial and parallel gatekeeping methodsmethodsSimple software implementation (SAS Simple software implementation (SAS Simple software implementation (SAS macro)macro)macro)

Closed testing principleClosed testing principleControl the familywise error rate in the Control the familywise error rate in the Control the familywise error rate in the strong sense

Summary

[Slide 28]

Chen, Luo, Capizzi. The application of enhanced parallel gatekeeping strategies. Statistics in Medicine. parallel gatekeeping strategies. Statistics in Medicine. parallel gatekeeping strategies. Statistics in Medicine. 2005; 24:1385-1397.

Dmitrienko, Off en, Westfall. Gatekeeping strategies for Dmitrienko, Off en, Westfall. Gatekeeping strategies for Dmitrienko, Off en, Westfall. Gatekeeping strategies for clinical trials that do not require all primary eff ects to clinical trials that do not require all primary eff ects to clinical trials that do not require all primary eff ects to be signifi cant. Statistics in Medicine. 2003; 22:2387-be signifi cant. Statistics in Medicine. 2003; 22:2387-be signifi cant. Statistics in Medicine. 2003; 22:2387-2400.2400.

Dmitrienko, Molenberghs, Chuang-Stein, Off en. Dmitrienko, Molenberghs, Chuang-Stein, Off en. Dmitrienko, Molenberghs, Chuang-Stein, Off en. Analysis of Clinical Trials Using SAS: A Practical Guide. Analysis of Clinical Trials Using SAS: A Practical Guide. SAS Press: Cary, NC, 2005.SAS Press: Cary, NC, 2005.

Marcus, Peritz, Gabriel. On closed testing procedures Marcus, Peritz, Gabriel. On closed testing procedures Marcus, Peritz, Gabriel. On closed testing procedures Marcus, Peritz, Gabriel. On closed testing procedures with special reference to ordered analysis of variance. with special reference to ordered analysis of variance. with special reference to ordered analysis of variance. Biometrika. 1976; 63:655-660.

References

[Slide 29]

Westfall, Krishen. Optimally weighted, fi xed sequence Westfall, Krishen. Optimally weighted, fi xed sequence and gatekeeper multiple testing procedures. Journal of and gatekeeper multiple testing procedures. Journal of and gatekeeper multiple testing procedures. Journal of Statistical Planning and Inference. 2001; 99:25-41.Statistical Planning and Inference. 2001; 99:25-41.Statistical Planning and Inference. 2001; 99:25-41.

Westfall, Young. Resampling-Based Multiple Testing: Westfall, Young. Resampling-Based Multiple Testing: Westfall, Young. Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment. New Examples and Methods for P-Value Adjustment. New Examples and Methods for P-Value Adjustment. New York: Wiley, 1993.York: Wiley, 1993.York: Wiley, 1993.

References