Joint Analysis of Clinical Trial and Observational Study Data: Lessons from the Women’s Health

InitiativeRoss L. Prentice

Fred Hutchinson Cancer Research Centerand University of Washington

Observational studies - allow study of many exposure-disease associations, but specificity, reliability, and comprehensiveness?

Intermediate outcome RCTs - useful, but have suffered from a rather typical limited set of outcomes

Full-scale RCTs - reliable for a limited set of hypotheses, but cost, logistics, adequacy of follow-up durations?

Some possible ways forward

Joint analyses of RCT and observational data to Joint analyses of RCT and observational data to extend CT results, and to sharpen design and extend CT results, and to sharpen design and analysis methodsanalysis methodsBiomarkers for difficult to measure exposures Biomarkers for difficult to measure exposures (e.g. in diet and physical activity areas)(e.g. in diet and physical activity areas)HighHigh--dimensional outcomes (e.g. plasma dimensional outcomes (e.g. plasma proteomic profiles) in intermediate outcome trials proteomic profiles) in intermediate outcome trials

Design of WHI

DM HRT

CaD

os

48,835 27,347

36,282

93,676

CT=68,132

WHI =161,808

WHI Hormone Program Design

Hysterectomy

Conjugated equine estrogen (CEE) 0.625 mg/d

Placebo

CEE 0.625 mg/d + medroxyprogesterone acetate (MDA) 2.5 mg/d

N= 16,608

N= 10,739YES

NO

Placebo

Clinical Outcomes in the WHI Postmenopausal Hormone Therapy Trials (JAMA 2002, 2004)

Hazard Ratio 95% CI Hazard Ratio 95% CI

Coronary heart disease 1.29 1.02 - 1.63 0.91 0.75 - 1.12Stroke 1.41 1.07 - 1.85 1.39 1.10 - 1.77Venous thromboembolism 2.11 1.58 - 2.82 1.33 0.99 - 1.79Invasive breast cancer 1.26 1.00 - 1.59 0.77 0.59 - 1.01Colorectal cancer 0.63 0.43 - 0.92 1.08 0.75 - 1.55Endometrial cancer 0.83 0.47 - 1.47 Hip fracture 0.66 0.45 - 0.98 0.61 0.41 - 0.91Death due to other causes 0.92 0.74 - 1.14 1.08 0.88 - 1.32Global index 1.15 1.03 - 1.28 1.01 0.91 - 1.12

Number of women 8506 8102 5310 5429Follow-up time, mean (SD), mo 62.2 (16.1) 61.2 (15.0) 81.6 (19.3) 81.9 (19.7)

E+P Trial E-Alone TrialOutcomes

Postmenopausal Hormone Therapy (E+P) and Cardiovascular Disease

Women’s Health Initiative study of estrogen plus progestin among postmenopausal women in the age range 50-79 at baseline*

CT OS Age-adj Age-adj

Placebo E+P HR Control E+P HR

Number of women

8102 8506

35,551 17,503Number of events:

CHD 147 188 1.21 615 158 0.71Stroke

107

151

1.33

490

123

0.77 VT

76 167

2.10

336

153

1.06

*Prentice RL, Langer R, Stefanick ML, Howard BV, Pettinger M, Anderson G, Barad D, Curb D, Kotchen J, Kuller L, Limacher M, Wactawski-Wende J. American Journal of Epidemiology 162:404-414; 2005.

Cox Model

h(t; Z(t)) = hos

(t)exp(x(t)/β)

HR (95% CI) HR (95% CI) HR (95% CI)Factor CHD Stroke VT

E+P in CT 1.27 (1.00, 1.61) 1.21 (0.93, 1.59) 2.13 (1.59, 2.85)E+P in OS 0.87 (0.72, 1.05) 0.86 (0.70, 1.07) 1.31 (1.07, 1.61)E+P in OS/E+P in CT 0.70 (0.52, 0.95) 0.72 (0.52, 1.02) 0.62 (0.43, 0.88)

CVD Hazard Ratios for E+P Use,in Joint Analyses of Data from CT and OS Cohorts,

Controlling for Potential Confounding

Factors

Years From E+P

InitiationHR in CT HR in OS

HR in CT Under Constant OS to CT

Ratio

HR Common to CT and OS

<

2 1.68 (1.15, 2.45; 80) 1.12 (0.46, 2.74; 5) 1.58 (1.12, 2.24) 1.56 (1.12, 2.19)

(2, 5) 1.25 (0.87, 1.79; 80) 1.05 (0.70, 1.58; 27) 1.19 (0.87, 1.63) 1.16 (0.89, 1.51)

> 5 0.66 (0.36, 1.21; 28) 0.83 (0.67, 1.01; 126) 0.86 (0.59, 1.26) 0.81 (0.67, 0.99)

E+P in OS/

E+P in CT 0.93 (0.64, 1.36)

E+P Hazard Ratio in the CT and OS as a Function of Time from Initiation of E+P Use

Coronary Heart Disease

ObservationalStudy

1.16CHD HR

0.81

Clinical Trial

2 5 10

Years from E+P Initiation during WHI Follow-up

1.56

Difference in Distribution in Years from E+P Initiationbetween WHI Cohorts

0.61 (0.46, 0.81)

0.93 (0.64, 1.36)

0.71 (0.52, 0.95)

0.58 (0.42, 0.80)

0.52 (0.37, 0.73)

0.72 (0.52, 1.02)

0.76 (0.49, 1.18)

0.62 (0.43, 0.88)

0.84 (0.54, 1.28)

0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5

Coronary Heart Disease

Stroke

Venous Thromboembolism

HR in OS / HR in CT

Ratio of OS to CT Hazard Ratios for E+P Use

E+P Hazard Ratios (95% CIs) as a Function of Years from E+P Initiation, and Average HRs over Various Times from E+P Initiation, Assuming Common HR Functions in the CT

and OS

Years from Coronary Heart Disease Venous ThromboembolismE+P Initiation HR (95% CI) HR (95% CI)

< 2

1.56 (1.12, 2.19)

2.87 (1.89, 4.35)2 –

5

1.16 (0.89, 1.51)

1.70 (1.28, 2.26)> 5

0.81 (0.67, 0.99)

1.26 (1.02, 1.56)

Average HR (95% CI) Average HR (95% CI)

2

1.56 (1.12, 2.19)

2.87 (1.89, 4.35)4

1.36 (1.09, 1.70)

2.28 (1.72, 3.03)6

1.27 (1.04, 1.54)

2.07 (1.62, 2.63)8

1.13 (0.96, 1.33)

1.83 (1.50, 2.23)10

1.07 (0.92, 1.24)

1.71 (1.43, 2.05)

Postmenopausal Estrogen-alone andCardiovascular Disease

(Prentice RL, Langer R, Stefanick ML, et al. AJE 163:589-599,2006)

CT OSAge-adj Age-Adj

Placebo E-alone HR Control E-alone HR

Number of women 5,429 5,310 16,411 21,920

Number of events:

CHD

201 217 0.96 548 421 0.68

Stroke

127 168 1.37 408 431 0.95

VT

86 111 1.33 274 265 0.78

Years from Hormone CT OS E-alone E+PTreatment Initiation HR (95% CI) HR (95% CI) HR (95% CI) HR (95% CI)

Coronary Heart Disease< 2 1.07 (0.68, 1.68) 1.20 (0.49, 2.94) 1.11 (0.73, 1.69) 1.58 (1.12, 2.24)2 - 5 1.13 (0.79, 1.61) 1.09 (0.75, 1.60) 1.17 (0.88, 1.56) 1.19 (0.87, 1.63)> 5 0.80 (0.57, 1.12) 0.73 (0.61, 0.84) 0.81 (0.62, 1.06) 0.86 (0.59, 1.26)

Hormone Therapy: HR in OS/HR in CT 0.89 (0.67, 1.19) 0.93 (0.64, 1.36)

Stroke< 2 1.69 (0.97, 2.94) 0.37 (0.05, 2.65) 1.48 (0.89, 2.44) 1.41 (0.90, 2.22)2 - 5 1.14 (0.76, 1.72) 0.89 (0.50, 1.44) 1.18 (0.83, 1.67) 1.14 (0.82, 1.59)> 5 1.41 (0.94, 2.11) 1.01 (0.87, 1.18) 1.48 (1.06, 2.06) 1.12 (0.73, 1.72)

Hormone Therapy: HR in OS/HR in CT 0.68 (0.48, 0.97) 0.76 (0.49, 1.18)

Venous Thromboembolism< 2 2.36 (1.08, 5.16) 1.48 (0.46, 4.75) 2.18 (1.15, 4.13) 3.02 (1.94, 4.69)2 - 5 1.31 (0.77, 2.20) 0.91 (0.51, 1.61) 1.22 (0.80, 1.85) 1.85 (1.30, 2.65)> 5 1.16 (0.71, 1.89) 0.85 (0.70, 1.03) 1.06 (0.72, 1.56) 1.47 (0.96, 2.24)

Hormone Therapy: HR in OS/HR in CT 0.82 (0.54, 1.23) 0.84 (0.55, 1.28)

E-alone Combined CT and OS

Hormone treatment hazard ratios (95% CIs) in the estrogen (E-alone) clinical trial (CT) and corresponding observational study (OS) sample; and in the

estrogen and estrogen plus progestin (E+P) clinical trials and corresponding observational study samples

Years from Hormone CT OS Adjusted CT OS AdjustedTreatment Initiation HR (95% CI) HR (95% CI) HR (95% CI) HR (95% CI)

< 2 0.55 (0.14, 2.18) 1.00 (0.38, 2.60) 1.54 (0.66, 3.55) 1.72 (0.54, 5.54)2 - 5 0.82 (0.28, 2.37) 0.58 (0.24, 1.40) 1.17 (0.52, 2.60) 1.38 (0.54, 3.48)> 5 0.67 (0.29, 1.54) 0.41 (0.17, 1.01) 0.15 (0.02, 1.27) 1.14 (0.46, 2.84)

HR in OS/HR in CT 1.28 (0.59, 2.77) 0.92 (0.43, 1.96)

Average HR‡ 0.70 (0.36, 1.34) 0.65 (0.37, 1.13) 1.19 (0.67, 2.11) 1.47 (0.75, 2.90)

10-Year Average HR 0.69 (0.38, 1.25) 0.58 (0.34, 0.98)

E-alone E+P

Coronary heart disease hormone treatment hazard ratios (95% CIs) among women 50-59 years of age at baseline

from the OS with adjustment using CT and OS data on the alternative preparation

Invasive Breast Cancer Incidence Rates in the Clinical Trial Hormone Trials (HT) and the Observational Study (OS) Subcohort*

*From Prentice RL, Chlebowski R, Stefanick M, Manson J, Langer R, Pettinger M, Hendrix S, Hubbell A, Kooperberg C, Kuller L, Lane D, McTiernan A, O’Sullivan MJ, Anderson G (2007). Submitted.†Age-adjusted to the 5-year age distribution in the CT cohort.

Placebo CEE Ratio Control E-alone Ratio

Number of Women 5,429 5,310 14,458 21,663Mean age 63.6 63.6 65.3 63.0Mean years of follow-up 7.1 7.1 7.0 7.2Number of Events 133 104 380 626Age-Adjusted† Annualized Incidence (%) 0.35 0.28 0.80 0.36 0.41 1.13

Placebo CEE Ratio Control E-alone Ratio

Number of Women 8,102 8,506 32,755 17,382Mean age 63.3 63.2 64.7 61.1Mean years of follow-up 5.6 5.6 5.5 5.6Number of Events 150 199 610 583Age-Adjusted† Annualized Incidence (%) 0.33 0.42 1.25 0.33 0.65 1.94

CT OS

CT OSE+P

E-alone

Invasive Breast Cancer Hazard Ratios

for HT Use Adjusted for Potential Confounding Factors, in Combined

Analyses of Data from the CT and OS

*Adjusted for age (linear), ethnicity, bmi (categorical and linear), education, smoking history, alcohol consumption, prior HT use, general health, physical activity, Gail risk score

Factor

HT in CT 0.71 (0.53, 0.97) 1.20 (0.95, 1.52)HT in OS 1.28 (1.05, 1.56) 1.71 (1.38, 2.13)

HT in OS/HT in CT 1.79 (1.25, 2.58) 1.42 (1.03, 1.96)

E-alone E+PHR* (95% CI) HR* (95% CI)

Breast Cancer Hazard Ratio Estimates according to Prior Postmenopausal Hormone Therapy Status and Years from

Hormone Therapy Initiation

CEE CEE/MPA Prior HT No Prior HT Prior HT No Prior HT HR (95% CI) HR (95% CI) HR (95% CI) HR (95% CI) Years from HT Initiation < 2 1.23 (0.57, 2.68) 0.53 (0.25, 1.12) 1.12 (0.58, 2.15) 0.64 (0.40, 1.04) 2 - 5 0.72 (0.42, 1.24) 0.74 (0.48, 1.15) 2.17 (1.31, 3.58) 1.23 (0.42, 1.65) > 5 0.84 (0.52, 1.34) 0.73 (0.48, 1.11) 2.44 (1.34, 4.43) 1.91 (1.34, 2.71) HT in OS/HT in CT 1.42 (0.93, 2.16) 1.28 (0.90, 1.83)

Summary

●

Different overall hazard ratios for HT suggested from WHI Clinical Trial and Observational Study.

●

Detailed accommodation of potential confounding factors explains

some of this difference, but significant discrepancy remains.

●

Accommodation of time from initiation of HT provides an explanation for much of the residual difference for CHD and VT, but some further explanation may be needed for stroke.

Differences in breast cancer HRs between the CT and OS

appear to be substantially explained by an interaction of HR with gap time from menopause to HT initiation

●

WHI CT and OS may combine to provide reliable estimates of risks

and benefits over longer durations, or in important subsets, than is

possible from CT alone.

●

Careful design and analysis methods needed to obtain accurate information from observational studies; intervention trials are needed if public health implications are sufficiently great.

Exposure Biomarkers

(Diet and Physical Activity)

Age-Adjusted Breast Cancer Incidence among Women of ages 55-69 in 1980 versus per capita for consumption in

1975

Dietary Fat and Postmenopausal Breast Cancer

Fat Intake Quintile

Case-control StudiesHowe et al (1990, JCNI) 1 1.20 1.24 1.24 1.46 (p<0.0001)

Cohort StudiesHunter et al (1996, NEJM) 1 1.01 1.12 1.07 1.05 (p = 0.21)

Any reason to continue research on this topic?

Ability to adequately characterize and adjust for measurement error?

Underreporting of Energy and Protein(Heitmann and Lissner, 1995, BMJ)

Dietary Change Goals: Intervention Group

Photos courtesy of USDA Agricultural Research Service

20% energy from fat

5 or more fruit and vegetable servings daily

6 or more grain servings daily

Mean (SD) of Nutrient Consumption by Randomization Group

*Difference significant at p<0.001 from a two sample t-test

Year 1 Intervention Control

Year 1 Difference

Year 3 Difference

Year 6 Difference

Fat (% of calories) 24.3 (7.5) 35.1 (6.9) -10.7* (7.0) -9.5* (7.4) -8.1* (7.8) Total Fat (g) 40.8 (21.4) 63.0 (31.0) -22.4* (31.1) -20.1* (32.0) -18.4* (33.5) Saturated Fat (%) 8.1 (2.8) 11.8 (2.9) -3.7* (2.9) -3.3* (3.1) -2.9* (3.3) Polyunsaturated Fat (%) 5.2 (1.8) 7.2 (2.1) -2.0* (2.1) -1.7* (2.2) -1.4* (2.3) Monounsaturated Fat (%) 8.9 (3.1) 13.3 (2.9) -4.4* (3.0) -3.9* (3.2) -3.3* (3.4) Energy (kcal) 1500.5 (544.2) 1593.8 (644.0) -95.8* (616.2) -92.5* (632.1) -119.9* (662.9)

Mean (SD) of Nutrient Consumption and Body Weight by Randomization Group

Year 1 Intervention Control

Year 1 Difference

Year 3 Difference

Year 6 Difference

Fruits and Veg. (sv/day) 5.1 (2.3) 3.9 (2.0) 1.2* (1.9) 1.3* (2.0) 1.1* (2.1) Grains (sv/day) 5.1 (2.7) 4.2 (2.3) 0.9* (2.5) 0.7* (2.6) 0.4* (2.6) Fiber (g/day) 18.1 (7.5) 14.9 (6.5) 3.2* (6.1) 3.1* (6.4) 2.4* (6.6) Folate (mcg) 398.5 (215.0) 346.1 (195.1) 52.3* (192.3) 62.1* (208.2) 45.6* (201.1) Alcohol Intake (g/day) 4.3 (8.9) 4.3 (9.2) 0.0 (6.7) 0.1 (7.1) -0.1 (7.4) Weight (kg) 74.4 (16.7) 76.3 (16.7) -2.2* (8.4) -1.3* (9.1) -0.8* (9.4)

*Difference significant at p<0.001 from a two sample t-test

Risk of Breast Cancer and Other Major Clinical Outcomes by Randomization Assignment (JAMA, 2006)

Intervention Comparison Group Group

# of Cases

(Annualized %) # of Cases

(Annualized %) Hazard Ratio

(95% CI) Unweighted

P-value Weighted P-value

Breast cancer incidence 655 (0.42%) 1072 (0.45%) 0.91 (0.83, 1.01) 0.07 0.09 Breast cancer mortality 27 (0.02%) 53 (0.02%) 0.77 (0.48, 1.22) 0.26 0.27 Total cancer incidence 1946 (1.23%) 3040 (1.28%) 0.96 (0.91, 1.02) 0.15 0.10 Total cancer mortality 436 (0.28%) 690 (0.29%) 0.95 (0.84, 1.07) 0.41 0.22 Total mortality 950 (0.60%) 1454 (0.61%) 0.98 (0.91, 1.07) 0.70 0.29 Global index 2051 (1.30%) 3207 (1.35%) 0.96 (0.91, 1.02) 0.16 0.16

Intervention versus Comparison Group Breast Cancer Hazard Ratios by Baseline Dietary Factors

No. of Participants Intervention Comparison

Baseline Variable (# of

cases = 655) (# of

cases = 1072) HR (95% CI) P-value

for interaction

% energy from fat (kcal) 0.04 < 27.9 144 222 0.97 (0.79, 1.20) 27.9 - < 32.3 186 259 1.08 (0.89, 1.30) 32.3-< 36.8 160 283 0.85 (0.70, 1.03) > 36.8 151 291 0.78 (0.64, 0.95)

Fat intake (g) 0.42 < 46.2 128 221 0.87 (0.70, 1.08) 46.2-< 59.8 176 261 1.01 (0.84, 1.22) 59.8-<76.0 194 300 0.97 (0.81, 1.16) > 76.0 143 273 0.79 (0.64, 0.96)

Energy intake (kcal) 0.89 < 1391.8 139 226 0.92 (0.75, 1.14) 1391.8-<1663.6 164 290 0.85 (0.70, 1.03) 1663.6-<1958.7 179 271 0.99 (0.82, 1.20) > 1958.7 159 268 0.89 (0.73, 1.08)

Nature and magnitude of random and systematic bias likely varies among assessment instruments.

Systematic bias may relate to many factors (e.g., age, ethnicity, body mass, behavioral factors).

Bingham et al (2003, Lancet) report a positive association between breast cancer and total and fat when consumption was assessed using a 7-day food diary, but the association was modest and non-significant when consumption was assessed with a FFQ. Very similar results from 4-day food record and FFQ analyses among DM comparison group women (Freedman et al 2006, IJE).

Objective measures (biomarkers) are needed to make progress in this important research area. Biomarker assessments in substudies (such as DLW measures of total energy expenditure) can be used to calibrate self-report assessments.

Nutrient Biomarker Substudy in the WHI DM Trial and Nutrition and Physical Activity Assessment

Study in WHI Observational Study

544 women completed two544 women completed two--week DLW protocol with urine and week DLW protocol with urine and blood collection and with FFQ and other questionnaire data blood collection and with FFQ and other questionnaire data collection (50% intervention, 50% control). A 20% reliability collection (50% intervention, 50% control). A 20% reliability subsample repeated protocol separated, by about 6 months subsample repeated protocol separated, by about 6 months from original data collection.from original data collection.

Biomarker study among 450 women in the WHI Observational Biomarker study among 450 women in the WHI Observational Study for calibrating baseline FFQ, 4DFR, and PA questions, Study for calibrating baseline FFQ, 4DFR, and PA questions, and for evaluating measurement properties of prominent and for evaluating measurement properties of prominent dietary and physical activity assessment approaches dietary and physical activity assessment approaches (frequencies, records, and recalls) and their combination.(frequencies, records, and recalls) and their combination.

Regression of FFQ kcal –

TEE kcal on subject characteristics

Variable β

coefficient (SE) p-value

DM-Intervention -104.8

(65)

0.11Overweight

-94.1 (71) 0.19

Obese

-302.7

(87)

< 0.001Age

11.4 (5.2)

0.03

Black

-190.9

(134)

0.15Hispanic -132.0

(127)

0.30

Other Ethnicity

183.7

(108)

0.09_______________________________________________

Measurement Models for Nutritional Epidemiology (Carroll, Freedman, Kaaks, Kipnis, Prentice…)

Recovery Biomarkers:Xbiomarker

= Z + eWself-report

= a0

+ a1

Z + a2

V + a3

ZV + r + εCan estimate odds ratios (Sugar et al, 2006), or hazard ratios (Pam Shaw 2006 dissertation), corresponding to Z from cohort data on W and subcohort data on X.

Concentration Biomarkers:

Xbiomarker

= b0

+ b1

Z + s + eInability to disassociate actual intake ‘Z’

from person-specific bias ‘s’

is a major limitation.

Needed research:

•

Development of additional recovery-type biomarkers•

Methodologic work (e.g., feeding study designs) to facilitate use of concentration biomarkers

Intermediate Outcome Trials Having High-Dimensional Responses

Evaluate impact of potential preventive intervention on high-dimensional response (e.g., plasma proteome)

Develop knowledge base to relate high-dimensional response to risk of a broad range of clinical outcomes

Predict intervention effects on clinical outcomes of interest, from high-dimensional response, to help determine whether a full-scale intervention trial is merited

Ongoing Proteomic Studies in WHI

Comparison of plasma proteomes between baseline and 1-year for 50 E+P users and 50 E-alone users in HT trials (pools of size 10), using an Intact Protein Analysis System

Comparison of plasma proteomes between 800 cases of CHD, stroke, and breast cancer and matched controls (pools of size 100) to identify protein concentration and disease risk associations.

Novel candidate proteins to be selected for individual testing among cases and controls in HT trials

Aspects of the Population Science / Chronic Disease Prevention Research Agenda

For reasons of cost and logistics only a few well-selected chronic disease primary prevention trials can be conducted at any point in time.

Reliability of state-of-the-art observational studies sometimes unclear, even if well-conducted.

Circumstances where both observational study data and clinical trial data are available provide an important opportunity to identify reasons for any discrepancy in results, and to strengthen corresponding design and analysis procedures.

Aspects of the Population Science / Chronic Disease Prevention Research Agenda (continued)

Disease prevention research involves many interesting topics where further methodologic and statistical development is needed.

A much enhanced preventive intervention development enterprise is needed, perhaps facilitated by genomic and proteomic approaches.