Subgroup Analyses:  Can Subgroup Analyses:  Can We ‘Smooth' out the Rough We ‘Smooth' out the Rough

Edges?Edges?

Daniel Sargent, PhDDaniel Sargent, PhD

Mayo ClinicMayo Clinic

Sept 28, 2006Sept 28, 2006

OutlineOutline

MotivationMotivation Subgroups ARE medicine (especially its Subgroups ARE medicine (especially its

future)future) ExamplesExamples

Good and bad conductGood and bad conduct StrategiesStrategies

Hierarchical modelsHierarchical models Smoothing approachesSmoothing approaches

ConclusionConclusion

Subgroups analysis: My Subgroups analysis: My Definition & My BiasDefinition & My Bias

Definition: An effort to draw inference on Definition: An effort to draw inference on an effect of an intervention in a set of an effect of an intervention in a set of patients smaller than the entire patients smaller than the entire experimental cohortexperimental cohort

Bias: Such inferences will be more robust Bias: Such inferences will be more robust when based on a model using all patients when based on a model using all patients than an analysis restricted to just the than an analysis restricted to just the cohort of interestcohort of interest

Subgroups are medicineSubgroups are medicine

If all patients were the same, wouldn’t need If all patients were the same, wouldn’t need physiciansphysicians

Human Genome Project massively Human Genome Project massively expanding knowledge baseexpanding knowledge base

Technology, biology, chemistry, etc. Technology, biology, chemistry, etc. allowing manufacture of highly specific, allowing manufacture of highly specific, targeted compoundstargeted compounds

Patients seek ‘tailored’ treatment Patients seek ‘tailored’ treatment recommendationsrecommendations

Example: Colon Cancer: Model-Example: Colon Cancer: Model-Derived Estimates of 5 year DFS (%) Derived Estimates of 5 year DFS (%) with with SurgerySurgery plus plus Adjuvant TherapyAdjuvant Therapy

Nodal Status

T stage Low Grade High Grade

S S 0 nodes T3 73 65 T4 60 51 T1-T2 62 53 1-4 nodes T3 49 38 T4 33 23 T1-T2 39 28 > 5 nodes T3 24 15 T4 11 5

+AT +AT 77 70

66 57 75 68 65 56 52 40 57 46 43 32 27 17

+AT +AT 77 70

66 57 75 68 65 56 52 40 57 46 43 32 27 17

Gill, JCO 2004; http://www.mayoclinic.com/calcs

Example: Breast CancerExample: Breast Cancer

Most common cancer in women in the US The HER-2 gene is overexpressed in 25-30% of

breast cancers; associated with worse prognosis.

Trastuzumab, a humanized monoclonal antibody targets the HER-2 receptor; previous trials have demonstrated activity in the treatment of HER-2 overexpressing late stage breast cancer.

Performed a clinical trial testing trastuzumab in subset of HER-2 positive women with early stage breast cancer

0 1 2 3 4 5

50

60

70

80

90

10

0

0 1 2 3 4 5

50

60

70

80

90

10

0

Disease-Free Survival Survival

Years Years

AC→T+H →H134 events

AC→T+H →H62 events

AC→T261 events

HR=0.48, 2P=3x10-12

HR=0.67,2P=0.015

94%91%

92%

87%

87%

75%

85%

67%

AC→T92 events

Romond et al, NEJM 2005

Avoiding subgroup analysis: Avoiding subgroup analysis: Targeted Phase II/III TrialsTargeted Phase II/III Trials

Patient Selection for targeted therapiesPatient Selection for targeted therapies Test the recommended dose on patients Test the recommended dose on patients

who are most likely to respond based on who are most likely to respond based on their molecular expression levelstheir molecular expression levels

May result in a large savings of patients May result in a large savings of patients (Simon & Maitournam, Clinical Cancer Research 2004)(Simon & Maitournam, Clinical Cancer Research 2004)

Trials in targeted populationsTrials in targeted populations

Gains in Gains in efficiency efficiency depend on depend on marker marker prevalence and prevalence and relative efficacy relative efficacy in marker + and in marker + and marker – patientsmarker – patients

Details: Session Details: Session #13 tomorrow#13 tomorrow

PrevalencePrevalence Relative Relative EfficacyEfficacy

Efficiency Efficiency GainGain

25%25% 0%0% 16x16x

25%25% 50%50% 2.5x2.5x

50%50% 0%0% 4x4x

50%50% 50%50% 1.8x1.8x

75%75% 0%0% 1.8x1.8x

75%75% 50%50% 1.3x1.3x(Simon & Maitournam, CCR 2004)

Case Study: Stage II colon cancerCase Study: Stage II colon cancer

Colon cancer: Prognosis defined by stageColon cancer: Prognosis defined by stage Prior trials generally enrolled patients with both Prior trials generally enrolled patients with both

stage II and III diseasestage II and III disease Previous randomized trials uniformly Previous randomized trials uniformly

demonstrate benefit of chemotherapy in demonstrate benefit of chemotherapy in stage III patients (node positive)stage III patients (node positive)

Previous trials & pooled analyses mixed Previous trials & pooled analyses mixed regarding benefit in stage II patientsregarding benefit in stage II patients

No single trial powered for modest effect No single trial powered for modest effect seen in stage II ( ↑ 2-3% in 5 year survival)seen in stage II ( ↑ 2-3% in 5 year survival)

Meta-analysis Stage II Adjuvant TherapyMeta-analysis Stage II Adjuvant Therapy

Benson et al. J Clin Oncol. 2004

N=2,732RR=0.88P=0.08

American Society of Clinical American Society of Clinical Oncology Guidelines 2004Oncology Guidelines 2004

Direct evidence from randomized trials does not Direct evidence from randomized trials does not support routine use of chemotherapy for patients support routine use of chemotherapy for patients with stage II colon cancer. with stage II colon cancer.

Those who accept the relative benefit in stage III Those who accept the relative benefit in stage III disease as adequate indirect evidence of benefit disease as adequate indirect evidence of benefit for stage II disease are justified in considering for stage II disease are justified in considering chemotherapy, particularly for patients with high-chemotherapy, particularly for patients with high-risk stage II disease. risk stage II disease.

Ultimate clinical decision should be based on Ultimate clinical decision should be based on discussions with the patientdiscussions with the patient..

Benson et al. J Clin Oncol. 2004

Primary end-point: disease-free survival (DFS)Primary end-point: disease-free survival (DFS)

R

LV5FU2

FOLFOX4: LV5FU2 + oxaliplatin 85 mg/m²

N=2246

Stage II: 40%

Stage III: 60%

New therapy: FOLFOXNew therapy: FOLFOX

de Gramont et al., ASCO 2005

6.6%

Disease-free Survival (ITT)Disease-free Survival (ITT)1.0

0.9

0.8

0.7

0.6

0.5

0.3

0.4

0.2

0.1

0.0 0 666 12 18 24 30 36 42 48 54 60

Months

Events

FOLFOX4 279/1123 (24.8%)

LV5FU2 345/1123 (30.7%)

HR [95% CI]: 0.77 [0.65 – 0.90]

DF

S p

rob

abil

ity

p<0.001

de Gramont et al., ASCO 2005

Disease-free Survival (ITT) Disease-free Survival (ITT) Stage II and Stage III PatientsStage II and Stage III Patients

1.0

0.9

0.8

0.7

0.6

0.5

0.3

0.4

0.2

0.1

0.0 0

FOLFOX4 – 451 Stage IILV5FU2 – 448 Stage IIFOLFOX4 – 672 Stage IIILV5FU2 – 675 Stage III

HR [95% CI]:0.82 [0.60 – 1.13] Stage II0.75 [0.62 – 0.89] Stage III

Months

DF

S p

rob

abil

ity

666 12 18 24 30 36 42 48 54 60Data cut-off: January 16, 2005

8.6%

3.5%

de Gramont et al., ASCO 2005

DFS (months)

DFS in high-risk* stage II patientsDFS in high-risk* stage II patientsDFS in high-risk* stage II patientsDFS in high-risk* stage II patients1.01.0

0.90.9

0.80.8

0.70.7

0.60.6

Pro

bab

ility

*T4 and/or bowel obstruction and/or tumor perforation and/or poorly differentiated tumor and/or venous invasion and/or <10 examined LNsData cut-off: January 16, 2005

0 6 12 18 24 30 36 42 48

5.4%

HR 0.76FOLFOX4 – 286 HRStage IILV5FU2 – 290 HR Stage II

de Gramont et al., ASCO 2005

FDA ActionFDA Action

Approval of FOLFOX therapy only in stage Approval of FOLFOX therapy only in stage III patients, even though trial designed for III patients, even though trial designed for stage II and III patientsstage II and III patients

Possible rationalePossible rationale Standard chemotherapy vs control not shown Standard chemotherapy vs control not shown

beneficial in stage II patientsbeneficial in stage II patients This trial not significant for experimental vs This trial not significant for experimental vs

standard chemotherapystandard chemotherapy

Stage II trial: QUASARStage II trial: QUASAR

Chemotherapy(n = 1622)*

Observation(n = 1617)

No clear indicationfor chemotherapy

(n = 3239)

RANDOMIZE

Colon or rectal cancer

• Stage I-III• Complete resection

with no evidence of residual disease

Gray et al. ASCO 2004. Abstract 3501. At: http://www.asco.org/ac/1,1003,_12-002511-00_18-0026-00_19-0010698,00.asp. Accessed November 2004.

% o

f P

atie

nts

QUASAR: Overall SurvivalQUASAR: Overall Survival

P = .025-year OS, Observation = 77.4% vs Chemotherapy = 80.3%Relative risk = 0.83 (95% CI, 0.71-0.97)

YearsGray et al. ASCO 2004. Abstract 3501. At: http://www.asco.org/ac/1,1003,_12-002511-00_18-0026-00_19-0010698,00.asp.Accessed November 2004.

0 1 2 3 4 5 6 7 8 9 100

20

40

60

80

100Observation (n=1622)

Chemotherapy (n=1617)

Implication: Stage II patientsImplication: Stage II patients

Compared to control, 5-FU provides 2-3% ↑ in OS, Compared to control, 5-FU provides 2-3% ↑ in OS, statistically significant in a single trialstatistically significant in a single trial Debate over clinical relevanceDebate over clinical relevance

In a large trial, FOLFOX provides 3-4% ↑ in DFS In a large trial, FOLFOX provides 3-4% ↑ in DFS compared to 5-FU, not statistically significant for compared to 5-FU, not statistically significant for stage II alonestage II alone No hint of interaction between rx and stage, p = 0.77No hint of interaction between rx and stage, p = 0.77 On its own, debatable benefit compared to 5-FUOn its own, debatable benefit compared to 5-FU

Cross trial comparison: FOLFOX may result in 5-7% Cross trial comparison: FOLFOX may result in 5-7% improvement vs control, but not approvedimprovement vs control, but not approved No debate about clinical relevance No debate about clinical relevance

Grothey & Sargent, JCO 2005

Stage II Colon Cancer: Stage II Colon Cancer: Lessons LearnedLessons Learned

Decisions based on subgroups may seem Decisions based on subgroups may seem rational at the time, but lead to unintended rational at the time, but lead to unintended consequencesconsequences Results may make further trials impossible Results may make further trials impossible

(FOLFOX vs control) (FOLFOX vs control) Need better approaches to analyze Need better approaches to analyze

subgroups using modeling (or meta-subgroups using modeling (or meta-analyses), not individual trial resultsanalyses), not individual trial results

Potential solution for prospectively Potential solution for prospectively defined subgroups: Hierarchical defined subgroups: Hierarchical

modelsmodels Goal: Test a treatment in a number of Goal: Test a treatment in a number of

populationspopulations Hypothesis: Effect may depend vary Hypothesis: Effect may depend vary

between populationsbetween populations Example: Targeted cancer therapyExample: Targeted cancer therapy

Mechanism of action based therapyMechanism of action based therapy Multiple tumor types express ‘target’, to Multiple tumor types express ‘target’, to

varying degreesvarying degrees

Basic statistical formulationBasic statistical formulation

Suppose N subgroups, with mean Suppose N subgroups, with mean response response ii, i=1,...N, i=1,...N

Assume Assume i i ~ N(,2)

If Bayesian, put a prior on 2 Depending on estimate of 2, allows

heterogeneity between subgroups Easily extends to non-normal models

Hierarchical Model: ExampleHierarchical Model: Example

Phase II clinical trial of a new agent Phase II clinical trial of a new agent specifically targeted at patients with a specifically targeted at patients with a methylated MGMT promotermethylated MGMT promoter

Prevalence from 10% to 60% across Prevalence from 10% to 60% across various cancer typesvarious cancer types High prevalence seen in Head and Neck, High prevalence seen in Head and Neck,

Esophageal, Colorectal, and Non Small-Cell Esophageal, Colorectal, and Non Small-Cell Lung CancerLung Cancer

Goal: Determine if overall efficacy > 10%, Goal: Determine if overall efficacy > 10%, but efficacy may depend on tumor typebut efficacy may depend on tumor type

Logistic regression ExampleLogistic regression Example

Hierarchical logistic model for tumor responseHierarchical logistic model for tumor response Stopping rules for each tumor siteStopping rules for each tumor site

P ( Response rateP ( Response rateii > 10%) < 10% > 10%) < 10% OROR

P (Response rateP (Response rateii > 10%) < 25% & > 10%) < 25% &

P (Response rateP (Response rateOverallOverall > 10%) < 10% > 10%) < 10%

Simulation for operating characteristicsSimulation for operating characteristics BenefitsBenefits

Single trial (opposed to 4)Single trial (opposed to 4) Use all data formally but flexibly Use all data formally but flexibly

Survival ExampleSurvival Example

Survival following chemotherapy for colon Survival following chemotherapy for colon cancercancer

Pooled analysis of 5 trials, suggestion of a Pooled analysis of 5 trials, suggestion of a study-specific treatment effect (a different type of study-specific treatment effect (a different type of subgroup)subgroup)

Fit a random effect Cox modelFit a random effect Cox model (t; x) = (t; x) = 0i0i(t) exp (x(t) exp (xii)) i i ~ N(,2) Can either model 0 parametrically, or use

Cox model

Model ResultsModel Results

StudyStudy Single fixed Single fixed Treatment Treatment EffectEffect

Study Specific Study Specific Fixed Treatment Fixed Treatment EffectEffect

Study Specific Study Specific Random Random Treatment EffectTreatment Effect

OverallOverall -0.22 (0.14)-0.22 (0.14) -0.21 (0.20)-0.21 (0.20)

11 -0.25 (0.34)-0.25 (0.34) -0.21 (0.20)-0.21 (0.20)

22 0.24 (0.35)0.24 (0.35) -0.11 (0.22)-0.11 (0.22)

33 -0.28 (0.33)-0.28 (0.33) -0.22 (0.20)-0.22 (0.20)

44 -0.25 (0.20)-0.25 (0.20) -0.22 (0.17)-0.22 (0.17)

55 -1.10 (0.68)-1.10 (0.68) -0.29 (0.28)-0.29 (0.28)

Prior mean for precision (1/2) = 50, posterior mean 106, Little evidence of heterogeneity

Sargent et al, 2000

Another approach: Modeling Another approach: Modeling Interactions using ShrinkageInteractions using Shrinkage

Subgroup analyses are fundamentally looking at Subgroup analyses are fundamentally looking at interactionsinteractions

In multi-factor experiment, the number of In multi-factor experiment, the number of interactions can explodeinteractions can explode

Well known that shrinkage (or model averaging) Well known that shrinkage (or model averaging) provides much better performance than all or provides much better performance than all or nothing approach (stepwise)nothing approach (stepwise)

Idea: Include interactions in model, but shrink Idea: Include interactions in model, but shrink them away if they are not strongly supported by them away if they are not strongly supported by the datathe data

Another approach: Modeling Another approach: Modeling Interactions using shrinkageInteractions using shrinkage

Dental ExperimentDental Experiment Dentures are often made with a soft liner between the gums and Dentures are often made with a soft liner between the gums and

the hard denture basethe hard denture base Polishing the liner can cause a gap between the liner and the Polishing the liner can cause a gap between the liner and the

basebase Such gaps harbor pathogens like Such gaps harbor pathogens like CandidaCandida

The experimentThe experiment Main interest: new vs. standard soft liner materialMain interest: new vs. standard soft liner material Factor M:Factor M: 2 materials2 materials Factor P:Factor P: 4 polishing methods4 polishing methods Factor F:Factor F: 8 finishing methods8 finishing methods

Fully crossed design, no replicationFully crossed design, no replication Outcome measure: gap btwn liner & base, in logOutcome measure: gap btwn liner & base, in log1010 mm mm

Pesun, Hodges & Lai (2002) Pesun, Hodges & Lai (2002) J. Prosthetic DentistryJ. Prosthetic Dentistry

Smoothing interactions: Smoothing interactions: Smoothed ANOVASmoothed ANOVA

Fit full ANOVA model (include all interactions) Fit full ANOVA model (include all interactions) yy = = XX + + yy is 64 x 1, contains log is 64 x 1, contains log1010 gap gap ee is 64 x 1, normal mean 0, precision is 64 x 1, normal mean 0, precision 00II6464

XX is 64 x 64 is 64 x 64 is 64 x 1; we will smooth/shrink its elementsis 64 x 1; we will smooth/shrink its elements

12 main effects, 52 interactions12 main effects, 52 interactions Model interactionsModel interactions

• kk ~ N (0,1/ ~ N (0,1/ kk) , k=13, …, 64) , k=13, …, 64• Large Large k k implies implies kk shrunk toward 0 shrunk toward 0

Smoothed ANOVA: The Smoothed ANOVA: The model/prior for the model/prior for the kk

How to model the interactionsHow to model the interactions Each interaction smoothed by its own Each interaction smoothed by its own kk

Each effect's Each effect's kk are all the same, are all the same, effecteffect All two-way interactions are smoothed by a All two-way interactions are smoothed by a

single single Mix the above optionsMix the above options

Use priors on Use priors on kk to specify desired to specify desired

operating characteristics for interactionsoperating characteristics for interactions

Use Degrees of Freedom to set Use Degrees of Freedom to set priors for the priors for the kk

Hodges & Sargent (2001 Hodges & Sargent (2001 BiometrikaBiometrika) extended ) extended methods for computing DF in standard ANOVA methods for computing DF in standard ANOVA to linear hierarchical modelsto linear hierarchical models

Hodges et al (Hodges et al (TechnometricsTechnometrics, 2006) present , 2006) present methodology to use DF to set priorsmethodology to use DF to set priors Example: I want the 51 2-way interactions to share 5 Example: I want the 51 2-way interactions to share 5

degrees of freedomdegrees of freedom

See references for technical detailsSee references for technical details Ongoing work: extending to non-linear (Cox) Ongoing work: extending to non-linear (Cox)

modelsmodels

Summary: Smoothed ANOVASummary: Smoothed ANOVA

Subgroup analyses are fundamentally Subgroup analyses are fundamentally looking at interactionslooking at interactions

A priori have low probability of a significant A priori have low probability of a significant interaction, but don’t want to exclude the interaction, but don’t want to exclude the possibilitypossibility

Idea: Include interactions in model, but Idea: Include interactions in model, but shrink themshrink them

SummarySummary

Subgroup analysis is essential to clinical Subgroup analysis is essential to clinical researchresearch

People usually perform such analyses with People usually perform such analyses with best of intentionsbest of intentions

Up-front thought can allow us to Up-front thought can allow us to Carefully define population under studyCarefully define population under study Pre-specify sub-populations to be examinedPre-specify sub-populations to be examined

Hierarchical/Shrinkage models offer Hierarchical/Shrinkage models offer attractive possibilities for addressing attractive possibilities for addressing subgroups, if defined prospectivelysubgroups, if defined prospectively

Thank YouThank You

AcknowledgementsAcknowledgements Smoothed ANOVA: Jim HodgesSmoothed ANOVA: Jim Hodges Colon Cancer: Axel Grothey, Aimery Colon Cancer: Axel Grothey, Aimery

deGramont, Sharlene GilldeGramont, Sharlene Gill