Endpoints

(also Outcomes, Major Response

Variables)

How do you determine whether the treatment is effective?

Randomization (√)

Blinding (√)

Well-defined target population (√)

Appropriate control (√)

Excellent follow-up (√)

Adequate sample size (√)

Appropriate interim monitoring (√)

Endpoints (?)

Outline

• General endpoint considerations

• Surrogate endpoints

• Composite endpoints and recurrent events

• Safety outcomes (adverse events)

Major Problems that Limit Interpretation of Randomized Trials

• Inappropriate controls

• Endpoints which are not clinically relevant

• Inadequately powered studies

• Poor follow-up

• Improper interim analyses

Protocol and Trial Report - 1

• Endpoints should be pre-specified:– Written down in the protocol before trial begins– Defines the trial objectives (in part)– Basis for sample size

• Example: Herpes Zoster Vaccine Protocol– Hypotheses and Objectives: “The primary objective of

this study is to determine whether immunization with live-attenuated varicella-zoster vaccine (OKA/Merck Strain) can reduce the incidence and/or severity of herpes zoster (HZ) and its complications, primarily postherpetic neuralgia (PHN) in persons 60 years of age and older. This will be accomplished by comparing a measure of the burden of illness due to HZ and PHN in vaccine and placebo recipients.

Protocol and Trial Report - 2

• Example: Herpes Zoster Vaccine Trial Report (N Engl J Med 2005)– “The Shingles and Prevention Study (Department of

Veterans Affairs [VA] Cooperative Study No. 403) was conducted to determine whether vaccination with a live attenuated VZV vaccine would decrease the incidence, severity, or both of herpes zoster and post-herpetic neuralgia in adults 60 years of age or older.”

– “The primary endpoint was the burden of illness due to herpes zoster, a severity-by-duration measure of the total pain and discomfort associated with herpes zoster in the population of study subjects.”

Primary Endpoint

– Usually one outcome is specified as most important (primary endpoint)

– Key variable in design (follows from objective)

– Basis for sample size

– A focus of interim monitoring and QA

– Response variable given major attention in trial report

– Usually, but not always, e.g., Cox-2 trials on GI bleeding, relates to efficacy. In some studies the primary endpoint encompasses efficacy and safety, e.g., mortality in CHF study, lipid study, HIV treatment study

– “A clinical endpoint that provides evidence sufficient to fully characterize the effect of a treatment in a manner that would support a regulatory claim for treatment” (O’Neill RT, Cont Clinical Trials, 1997;18:550-556)

Secondary Endpoints

• There are usually several efficacy endpoints and these are commonly referred to as secondary endpoints or secondary efficacy endpoints.

• Safety endpoints must also be specified and are usually consider secondary– Discontinuation of study treatment– Side effects/adverse events– Serious adverse events– O’Neill (FDA) defines a secondary endpoint as one that

“provides additional characterization of treatment effect but that is not sufficient to characterize fully the benefit or to support a claim for a treatment effect”.

Characteristics Desired for Endpoint

• Relevant; easy to interpret

• Easy to diagnose

• Can be ascertained and classified in an unbiased manner

• Sensitive to treatment differences

• Measurable within a reasonable period of time

General Considerations - 1

More commonly occurring endpoints (high incidence) will result in smaller sample sizes than less frequent events (low incidence) as long as expected relative difference between treatment groups is similar • CHD + non-fatal MI vs. CHD death for lipid-lowering trial• Progression-free survival vs survival for cancer trial

Continuous response variables usually result in smaller sample sizes than binary or time to event• BP change vs. % with normal BP• HIV RNA change vs. % < 50 copies/mL• Weight change vs. % who lose > 5% of baseline

weight

General Considerations - 2

More serious events should be considered along with less serious ones • Count CHD deaths along with non-fatal MIs

Related to this, some events may have to be included to avoid misinterpretation due to informative censoring (this could result in a loss of power)

• Non-arrhythmic deaths along with arrhythmic deaths (DEFINITE, NEJM 2004)

• Death and missing data along with change in exercise duration (PICO, Heart 1996)

• Progression to AIDS or death from any cause (SMART, NEJM 2006)

Endpoint Examples - 1

MRFIT (JAMA 1982 and Amer J Cardiol 1986)

Primary: CHD DeathSecondary: CHD Death or non-fatal MI

CVD mortalityAll-cause mortality

NuCombo HIV Study (N Engl J Med 1996)

Primary: Progression to AIDS or death from any cause

Secondary: Death

AMIS (JAMA 1982)

Primary: All deathsSecondary: CHD death or non-fatal MI

Endpoint Examples - 2

• New BP-lowering drug – Systolic BP change

• CHF device – change in NYHA class; 6-minute walk

• New antiretroviral drug – HIV RNA suppression at 24 and 48 weeks

Special considerations: longitudinal measurements; informative missing data; left censoring

Choice of Endpoint in TOMHS[Antihypertensive Drugs]

BP, Side Effects, Quality of Life

Echocardiographic and Electrocardiographic Changes (Asymptomatic CVD)

Non-fatal MI, Stroke, Angina, Peripheral Artery Disease (Symptomatic CVD)

CVD Death• CHD• Stroke

Total Mortality

Choice of Endpoint in Antiretroviral Trials

CD4+ count; viral load

Genotypic/phenotypic resistance;Loss of drug options

Clinical disease progression (AIDS)

Serious AIDS and non-AIDS events

Survival

Choice of Endpoint in HIV Vaccine Trials

HIV Infection

Durable control of viremia post-infection (viralload set point)

CD4+ decline/ART

AIDS or death

Endpoints Used to Approve Cancer Drugs and Biologics

• Survival• Symptom endpoints (patient reported

outcomes)• Disease-free survival (e.g., time to tumor

recurrence or death)• Objective response rate (e.g., proportion of

patients with tumor size reduction of a pre-defined amount for a minimum time period)

Guidance for Industry. Clinical Trial Endpoints for theApproval of Cancer Drugs and Biologics. May 2007.

Requirements for FDA Approval Vary

• Antiretroviral drugs for HIV – viral load (regimen failure)

• Antihypertensive drug – BP

• CHF drug – morbidity and mortality

• Device for CHF – functional status

• Osteoporosis drugs – bone density

For Other Areas There is Uncertainty

• Treatments for community acquired bacterial pneumonia (CABP)– Should focus be on clinical endpoints that

capture how a patient feels or should outcomes incorporate clinical signs (e.g., fever) and laboratory tests (e.g., WBC count)?

– FDA position in November 2011: “improvement in at least 2 symptoms attributable to CABP… at a minimum cough, sputum production, chest pain, and shortness of breath at an early time point (i.e., day 3 or 5 after enrolment).

Anti-Infective Drugs Advisory Committee Briefing DocumentNovember 2011

Choice of Endpoint: General Hierarchical Categorization

• Clinical outcome (morbidity and mortality)

• Surrogate for clinical outcome (may be hard to establish)

• Intermediate outcome that is likely to predict clinical benefit (non-validated surrogate)

• Biomarker which measures biologic activity

Types of Endpoints from an Analytic Point of View

• Binary

• Ordered categorical

• Continuous (single point in time, repeated measures, slope)

• Counts

• Time to event and rates

Other Endpoint Considerations

1. Training of evaluators (e.g., BP measurement)

2. Ongoing quality assurance (e.g., laboratory QC)

3. Endpoint classification committee

4. Blinding of endpoint determination (as noted previously this can be done even in open- label studies)

5. Methods for reducing missing data

– Informed consent

– Training

– Quality assurance procedures

– Collect identifying information at entry

– National Death Index

Outline

• General endpoint considerations

• Surrogate endpoints

• Composite endpoints

• Safety outcomes (adverse events)

Surrogate Endpoint(Definition)

• Failure of treatment (relative to control) to influence the surrogate implies failure to influence “true” endpoint (Prentice, Stat Med, 1989)

• Surrogate must be able to capture “full” dependence of “true” endpoint rate on randomization group, i.e., no pathways whereby randomization assignments affect “true” endpoint that bypass the surrogate response

• Occurs with greater frequency than “true” endpoint

• Occurs sooner after treatment than “true” endpoint

Other Definitions of Surrogate Endpoints

• “a laboratory measurement or clinical sign used as a substitute for a clinically meaningful endpoint that measures directly how a patient feels functions or survives” (Temple).

• “ A surrogate endpoint is expected to expected to predict clinical benefit (or harm or lack of benefit or harm) based on epidemiologic, therapeutic, pathophysiologic, or other scientific evidence” (Biomarker Definitions Working Group; also, IOM report).

• “an outcome measure that substitutes for a clinical event of true importance” (Grimes and Schulz).

Institute of Medicine Report: Evaluation of Biomarkers and Surrogate Endpoints in

Chronic Disease• Biomarker evaluation should consist of 3 steps:

– Analytical validation (e.g., performance of an assay)

– Qualification (assessment of association of biomarker with disease and the effect of interventions on biomarkers)

– Utilization (contextual analysis of available evidence on the specific use proposed). A biomarker may be used as a surrogate for some disease states and not in others, e.g., HIV viral load in setting where suppression is complete versus partial

IOM 2010, National Academies Press

Relationship Between Treatment (A or B), a Surrogate Marker (S), and the Clinical

Outcome (T)

A or B

S

T

OtherMechanisms of Action Surrogate Marker

Treatment

Clinical Outcomes

Treatment has many mechanisms of action

Fleming and DeMets, Ann Int Med, 1996

Surrogate Endpoint:Not in Causal Pathway of Disease Process

Disease Surrogate True Clinical Endpoint Endpoint

Causal Pathway

Fleming and DeMets, Ann Int Med, 1996

A correlate does not a surrogate make!

Fleming TR and DeMets DL, Ann Int Med, 1996.

0

1 0

2 0

3 0

4 0

5 0

6 0

7 0

8 0

9 0

< 1 2 0 1 2 0 -1 2 9

1 3 0 -1 3 9

1 4 0 -1 5 9

1 6 0 -1 7 9

1 8 0 -2 0 9

2 1 0 +

Ag

e-a

dju

ste

d C

HD

Mo

rtal

ity

(pe

r 1

0,0

00

pe

rso

n-y

ea

rs)

Systolic Blood Pressure (mmHg)

Data from the MRFIT Study

Blood Pressure is Considered An Acceptable Surrogate Endpoint by the FDA

• Substantial epidemiological and clinical trial data.

• Demonstration that diverse BP-lowering agents provide benefit.

• Considered to be the principal causal pathway.

ALLHAT Results on Doxazosina: Cumulative 4-Year Rate (%)

Doxazosin Chlorthalidone

No. Patients 9,067 15,268

Fatal/Non-Fatal CHD 6.3%6.3%

Fatal/Non-Fatal Stroke 4.2%3.6%

Congestive Heart Failure 8.1%4.5%

All Cause Mortality 9.6%9.1%

a JAMA; 283:1967-1975, 2000

Surrogate True ClinicalEndpoint Endpoint

Intervention

Disease

True Clinical Endpoint

Surrogate Endpoint

Disease

Intervention

Multiple Pathways of the Disease Process

Surrogate True ClinicalEndpoint Endpoint

Intervention

Disease

Best Situation for Assessing Surrogacy

CD4+ AIDS or Count Death

IL-2

Disease

• IL-2: known to increase CD4+ cell count

• IL-2: known to be associated with toxicities

• Unknown whether IL-2 is increasing functional CD4+ cells

Interleukin -2 Trials for HIV

Deaths and Serious AIDS Event Rates by Latest CD4+ Count Following Initiation of ART

< 200 1327 29 4.29 907 74 8.16

200-350 2624 18 0.84 2284 38 1.66

350-499 3532 21 0.42 3228 18 0.56

500+ 8425 22 0.30 7964 21 0.26

CD4+ Level PY Events Rate PY Events Rate

All-Cause Mortality Serious AIDS

CASCADE Collaboration http://www.cascade-collaboration.org

ESPRIT Study Design

Patients taking ART with CD4+ counts ≥ 300/μL

N = 2071 N = 2040

IL-2

ART plus:

• 3 cycles of IL-2 (7.5 MIU twice daily for 5 days, 8 wks apart)

• additional cycles to maintain goal (2x baseline or ≥ 1000 CD4+ cells)

Control

ART without IL-2

Plan: 320 primary events

Closure date 15 Nov 2008

323 primary events observed

Median follow-up = 7 years

N Engl J Med 2009 361:1548-1559.

Median CD4+ During Follow-up

0

100

200

300

400

500

600

700

800

0 1 2 3 4 5 6 7Year

CD

4+ IL-2

Control

Avg Difference:160 cells, p<.001

Time spent IL-2 Control

< 300 cells 6% 9%

> 600 cells 57% 36%

IL-2: 2071 1846 1829 1797 1757 1721 1410 878

Control: 2040 1928 1861 1803 1739 1648 1350 824

No. pts

Primary EndpointOpportunistic Disease or Death

IL-2 Control

No. Rate* No. Rate* HR (95% CI) p-value

158 1.13 165 1.21 0.93 (0.75, 1.16) 0.52

Predicted HR based on CD4+ difference = 0.74

* rate per 100 person years

Surrogate True Clinical Endpoint EndpointDisease

Intervention

Interventions having Mechanisms of Action Independent of the Disease

Process

Concorde Study Results(Lancet 341:889-90, 1993)

Deaths 95 76

AIDS or Death 175 171

ARC, AIDS or Death 263 284

ImmediateZDV

DeferredZDV

CD4 difference over 3 yearsof follow-up (immediate - deferred) =

30 cells (p < 0.0001)

0

5

10

15

20

25

30

35

40

< 25 25 -49

50 -99

10 0 -19 9

20 0 -49 9

12-

mo

nth

Cu

mu

lati

ve

Mo

rta

lity

(%)

Baseline CD4+ Lymphocyte Count(cells/mm3)

CID 1996; 22:513-520

CD4+ Count and Mortality: Pre-HAART Era

Overview of Trialsof ZDV vs. Placebo (Immediate vs.

Deferred)

Total 4431 3291

No. deaths 734 617

No. AIDS/deaths 1026 882

ImmediateZDV

DeferredZDV (placebo)

Risk ratio = 1.04

Risk ratio = 0.96

Lancet 353:2014-2025, 1999.

Operational Criteriafor Valid Surrogates

• The surrogate (S) must predict the clinical event (T)

• Treatment must effect surrogate

• The surrogate (S) must fully capture the effect of treatment on the clinical event (T)

Prentice R, Stat Med, 1989.

Evaluation of Surrogacy

Determine relative risk (treatment versus control) of long-term clinical outcome

Show that relative risk when adjusted for marker in each treatment group is one

Not an optimal approach. Ideally, results from several studies with surrogate and clinical

outcomes would be compared.

Statistical Analysis for Single Study

• Fit logistic regression models:

1) log odds (long-term clinical outcome) =

a + b (trt)

2) log odds (long-term clinical outcome) =

a + ba (trt) + c (marker change)

• If marker change fully explains treatment effect, then ba would be zero

• (b-ba)/b measures proportion of effect on long-term clinical outcome explained by effect on marker

Validation of Surrogate Endpoints

Statistical – want more than one study · Meta-analyses of clinical trials data

Clinical · Comprehensive understanding of the causal pathways and intended and unintended mechanisms of action

Overview of 16 Antiretroviral Trials State-of-the-Art Conference 1993

Sig. Diff.

Sig. Diff.

No Diff.

7 6

1 2

No Diff.

8 8

13

3

Clinical DiseaseProgression

CD4+Change

Colon Cancer Example

• Traditional endpoint is overall survival (OS)

• Hypothesis: Disease free survival (DFS), assessed after 3

yrs, is an appropriate endpoint to replace overall survival

(OS) in adjuvant colon trials

Allow more rapid completion, reporting of trials

Allow promising agents to benefit patients more quickly

• Approach: Compare difference between treatment and

control (hazard ratio) within each trial for DFS and OS

Sargent DJ et al, J Clin Oncol, 2005;23:8664-8670.

Overall and Disease-Free Survival for Adjuvant Treatment for Colon Cancer

Sargent DJ et al, J Clin Oncol, 2005;23:8664-8670.

“The validity of a surrogate endpoint should be

judged by the probability that the trial results

based on the surrogate endpoint alone are

‘concordant’ with the trial results that would be

obtained if the true endpoint were observed and

used for the analysis”

Begg and Leung, J R Statis Soc A 2000; 163:15-28

Examples

• Anti-arrhythmic treatment for sudden death

• HSV-2 suppressive treatment to prevent HIV infection

• Estrogen/progestin treatment for coronary heart disease

There are many other examples of failed surrogates (e.g., HF treatment, drugs to

raise HDL cholesterol, weight-loss drugs)

Choice of Endpoint in Cardiac Arrhythmia Suppression Trial (CAST)

[Antiarrhythmic Drugs]

Ventricular Premature Beats

Arrhythmias

Sudden Death

Total Mortality

CAST Study DesignDouble-blind, placebo controlled

Eligible patients

Open-label titration

A. Encainide

B. Flecainide

C. Moricizine

Greater 80% suppression of VPDs

Yes No, excluded

Flecainide Placebofor

Flecainide

Moricizine Placebofor

Moricizine

Encainide Placebofor

Encainide

N Engl J Med 1989; 321:406-412.

CAST Study Sample Size Assumptions

Type I error () = .025 (1-sided)

Power (1-) = 0.85

Projected mortality in placebo group = 11% over 3 years

Expected reduction in mortality due to treatment = 30%

(No difference among active treatments)

Required sample size = 4400 patients

CAST Study Results

No. patients730 725

Deaths from arrhythmia 33 9or cardiac arrest

Other cardiac death 14 6

Other deaths or 9 7unclassified cardiac arrest

TOTAL 56 22

Encainide/Flecainide Placebo

Acyclovir and HIV Acquisition

• Background– Herpes simplex virus type 2 (HSV-2 infection

is the most common cause of genital ulcers

– Observational studies indicate that HSV-2 infection is associated with 2-3 fold increased risk for HIV infection

– Acyclovir is effective in suppressing HSV-2

• Two trials were conducted to determine if acyclovir treatment reduced HIV acquisition

HSV-2/HIV Trial #1

HIV-negative HSV-2, sero-positive men and women at risk

for HIV based on sex history

3,277 randomized

1637 Acyclovir 1640 Placebo

HR (acyclovir/placebo) for genital ulcers = 0.53 (95% CI: 0.46-0.62)

HR for HIV = 1.16 (95% CI: 0.83-1.62)

Lancet 2008, 371: 2109-2119

HSV-2/HIV Trial #2

HSV-2/HIV dually infected persons in heterosexual relation

with HIV-negative partner

3,408 couples randomized

1707 acyclovir 1701 placebo

HR (acyclovir/placebo) for genital ulcers = 0.27 (95% CI: 0.20-0.36)

HR for HIV = 0.92 (95% CI: 0.60-1.41) among partners

N Engl J Med 2010; 362: 427-439

Estrogen + Progestin inHealthy Postmenopausal Women: Lipidsafter 1 year and Events after 5.2 Years

LDL -12.7%

HDL +7.3%

CHD +29%

Stroke +41%

All CVD +22%

Estrogen + Progestinvs.

Placebo

JAMA 288:321-333, 2002

Review of Trials Funded by NHLBIGordon D et al N Engl J Med 2013; 369:1926-1934.

• 244 trials completed in 2000 or 2011.– 45 (18%) with clinical endpoints– 199 with surrogates or health-related behaviors

• 156 published (23% <12 mos. and 57% < 30 mos.)• Faster time to publication (predictors in

multivariable analysis)– Clinical endpoints (median 10 vs 30 mos. for surrogate

endpoint studies) – Cost ($5 million)– Positive result

• Clinical endpoint trials accounted for 82% of citations

Can be a big payoff for what usually is many years of work!

Summary

• There is no such thing as a risk-free intervention (unintended effects of treatment are common)

• Failure of surrogates to predict clinical outcome differences are common

• Trials with clinical endpoints have a greater impact on care than trials with surrogate endpoints

• It is time-consuming (and expensive) to rigorously establish surrogacy (need trials for which both the surrogate and clinical outcome are measured)

• Some judgment about causal pathways is always needed

• Potential surrogates are important in early phase studies