Challenges in the conduct and interpretation of Phase II (pilcit) randomized trials SaIIm Yusuf, MBBS, DPhU, FRCP Humi&on, Onturlq Cunadu

In general, Phase II (pilot) clinical trials seek to answer at least three questions that differ from those addressed in Phase III trials. First, is the intervention that is being studied promising enough to invest further substantial efforts and resources in larger and more definitive trials? Second, is the intervention safe enough that the benefits of its use likely outweigh the risks? Third, what dose of the drug (intervention) will be appropriate for patients, so that the benefits of its use can be m aximized with little or at least no excess of adverse effects? In addition to the above three questions, Phase II clinical trials may seek to address additional issues such as the feasibility of a specific approach to recruitment, event rates, and operational aspects of the study (for example, definitions of specific events or procedural and logistical issues). However, these latter sets of issues are relatively straightforward and can be satisfactorily dealt with by experienced clini- cal trlalists. This essay focuses on the first three issues and illustrates with specific examples the challenges associated with them. A few approaches to these chal- lenges are proposed-with the ready acknowledgment that they may not be universally applicable.

Is the intervention sufficiently promising to be evaluated in further large trials?

Phase II trials are usually a stepping stone to larger and more definitive trials. They are usually small or moderate in size and in some cases study only a few “tens” of patients. More recently, such trials have occa- sionally included a few hundred or, extremely rarely, a thousand or two thousand patients. Phase II trials of a few hundred patients (400 patients) are relatively common in the cardiovascular field; larger Phase II trials are extremely rare.

Three categories of outcomes may be measured in Phase II trials. The first is a surrogate outcome that reflects the mechanism of action of the study drug.

From the Deportment of Medicine, McMaster University, Hamilton General Hospital.

Reprint requests: Dr Sofim Yusuf, Department of Medicine, &Master University, Hamilton General Hospital, McMoster Clinic, Room 252, 237 Barfon St E, Ham& ton, Ontario, 1812X2 Canada. Email: yusuf@ccc.mcmoster.co Am Heart J 2000; 139:s 136s 142. Copyright 0 2000 by Mosby, Inc. 0002-8703/2000/$12.00 + 0 4/O/1041 89 doi: IO. 1067/mhj.2000. IO4 189

These include such outcomes as coronary artery patency, frequency of arrhythmias, blood pressure (BP) level, and cholesterol levels. The effects on these out- comes can be measured with reasonable accuracy in trials of a few hundred patients. In addition, measuring the effects is a necessary prerequisite in most cases to ensuring that an intervention does in fact possess the putative “mechanistic effect” by which it is purported to produce its beneficial effect.

Although, as noted, the effects on these outcomes can be measured with reasonable accuracy, the value of the measurements may be questioned at times; and for some interventions, no useful surrogate exists. For example, an antiarrhythmic agent can be shown to reduce the frequency of ventricular arrhythmias on a Holter monitor, but this effect need not translate into a reduction in arrhythmic events such as sudden death. Conversely, there may be no reliable surrogate measure for some useful interventions; for example, P-blockers used to prevent vascular events after myocardial infarc- tion (Ml)1 or in heart failure.* Sometimes the effect of a treatment on a particular surrogate outcome may be adverse or neutral, but the treatment may still lead to clinical benefit. For example, P-blockers may have no effect on exercise tolerance in patients with heart failure; yet recent evidence supports the value of j3-blockers in reducing overall death in patients with congestive heart failure (CHF).3 Sometimes surrogates that are useful in evaluating a specific agent or a narrow therapeutic class may not necessarily be reliable mark- ers for comparing agents in related classes. For exam- ple, BP lowering with diuretics reduces stroke, MI, and/or death. However, it is unclear whether BP-lower- ing agents of a different class (eg, a calcium channel blocker or an angiotensinconverting enzyme [ACE] inhibitor) provide approximately the same degree of benefit for similar reductions in BP. Therefore when sur- rogate outcomes are used in Phase II trials, their results should be interpreted with caution.

A second potential category of outcomes for Phase II trials is that of major clinical outcomes such as death, MI, strokes, or hospitalizations, any of which might be measured as the primary outcome of interest in the larger Phase III trials. The problem with this approach is readily obvious-the event rates for these outcomes are usually relatively low (eg, 5%, lo%, or 15%) over the period of observation. In Phase II trials of several hun- dred patients (or even several thousand patients), the

Amerrcan Heart Journal Volume 139, Number 4 Yusuf 5137

Interim results over the course of the ISIS-2 trial, in which aspirin was compared with placebo. By the time approx-

imately 400 deaths had accrued, there appeared to be little benefit from therapy with aspirin. Yet by the end of

the study, such therapy was associated with a clear reduction in mortality rate (by approximately 20%) that was

highly significant (personal communication from S. Parish and R. Collins).

play of chance is likely to influence the effects on these outcomes to a large extent, Therefore at times even the direction of treatment effect may not be a reliable indi- cator. For example, in the pilot study before the Sec- ond International Study of Infarct Survival (ISISZ) (which used a 2 x 2 x 2 factorial design evaluating streptokinase vs control, aspirin vs control, and heparin vs control in 600 patients), heparin appeared to be a very promising therapy, whereas the results with strep tokinase or aspirin were found to be less promising.4 By contrast, the large trials of ISIS-2 (16,000 patients)5 and ISIS-3 (40,000 patients)6 indicated a clear reduction in overall death with streptokinase and aspirin, whereas little benefit was observed with heparin with respect to mortality rate reduction.

This problem of measuring major clinical outcomes in Phase II trials is further emphasized by experiences during interim monitoring ln large trials. In ISIS-2, after the random assignment of approximately 3000 patients, there was, as Figure 1 shows, no difference seen in mortality rates between the group receiving aspirin and the group receiving placebo (personal communication, R. Collins and S. Parish). However, when the study was completed, a clear and highly significant reduction in the relative risk of death, by approximately 20%, was observed with aspirin. In the GISSI-1 trial,’ approxi- mately halfway through the study, no difference in mor- tality rates was seen (Figure 2), but by the end of the study a clear reduction in mortality rate was evident

(personal communication, A. Maggioni). Conversely, early in the Study of Left Ventricular Dysfunction (SOLVD) prevention trial, when approximately 20% of the events had occurred, there appeared to be a large and highly signilicant reduction in mortality rate (P = .OOl> (unpublished data); yet at the end of the study there was only a moderate risk reduction in mortality rate, which was not statistically sign&ants (Pigure 3).

Occasionally, small trials may demonstrate an unex- pected favorable or unfavorable treatment effect that is “statistically significant.” This finding may lead lnvesti- gators to have unrealistically large expectations of a treatment benefit; alternatively, an apparent adverse effect in a pilot study may be spuriously perceived as being disappointing. For example, in a small trial of approximately 450 patients, vesnarinone was compared with placebo in patients with heart failure; a high dose of vesnarinone (160 mg/d) produced a large excess in overall death (15 vs 3 deaths), whereas a medium dose (80 mg/d) reduced mortality rate by 50% (P = .003).9 This contrasting pattern led the Food and Drug Admln- istration to withhold approval of this compound and recommend that further larger trials be conducted. This decision was vindicated when a trial of approximately 3000 patients, the vesnarinone survival trial (VEST), indicated a significant 27% excess in mortality rate (P = .OI) with vesnarinone used at 80 mg/d.to

Conversely, in small trials, even beneIicial treatments may occasionally indicate an apparent harmful effect.

5138 Yusuf American Heart Journal

April 2CCO

GISSI-1

3130 8311 10732 (378) ftiy (Qw ww

Interim results over the course of the GISSI-1 trial. Strepto- kinase was clearly demonstrated to be superior to control in

. reducing mortality rates; yet halfway into the study (approxi- mately 5600 patients), little evidence of benefit was seen (personal communication with A. Maggioni).

For example, in a trial of benazeprll for reducing the progression of diabetic nephropathy, this drug was found to produce a nominally significant increase in overall mortality rate.‘* However, when all similar trials of ACE inhibitors are e xamined collectively, a significant reduction in overall mortality rate is observed. There- fore the apparent adverse effect on mortality rate in the benazeprll trial is almost certainly caused by the play of chance. A similar experience occurred with intravenous /3-blockers in acute MI. In the pilot study of 477 patients, intravenous ~blockers were associated with a large reduction in overall mortality rate that was statistically significant (P = .04).l* However, when the large ISIS-l study was conducted, these agents appeared to produce a significant excess in mortality rate when approxi- mately 2500 patients were randomly assigned. However, by the end of the study (16,000 patients), the therapy was associated with approximately a 15% risk reduction in mortality rate13 (Figure 4).

Thus, although the effects of a treatment on major outcomes are clinically relevant, the evidence from vari- ous trials indicates that these effects cannot be reliably evaluated when observed in moderate-sized trials of several hundred, or even several thousand, patients.

A thini category of events that may be considered in Phase II trials is a cluster of clinically relevant outcomes that are biologically related and that may all be affected by the same mechanism of action of the study drug. For example, ln the evaluation of a drug expected to prevent deaths by preventlng the progression of heart faihue, one might e xamine the impact of the treatment on a range of outcomes including death, hospitalizations for heart failure, or other measures of symptomatic worsen-

ing of heart failure, such as the need for additional heart failure therapy (eg, increasing dose of a diuretic). Although this approach certainly has some advantages compared with using a surrogate outcome or major clini- cal outcomes, it also has several potential problems. First, it may not be feasible in some conditions such as that of post-MI (eg, secondary prevention) or for antiar- rhythmic agents. In other instances, the long-term effects of a drug may differ from its short-term effects. For exam- ple, cholesterol-lowering agents may have little impact on any clinical outcome (eg, death, MI, stroke, angina, revascularization) over the initial 6 to 12 months of treat- ment, whereas several years of therapy with these agents may lead to large treatment benefits. Conversely, the adverse effects of some treatments may manifest early (eg, &blockers may initially women heart failure), but the benefits may take longer to emerge (eg, Pblockers reduce overall mortality rates after several months of treatment in patients with heart failure). Because most Phase II trials are of relatively short duration, such con- trasting effects may produce misleading conclusions even if the composite clinical outcomes selected were sufBciently numerous to detect plausible differences.

Is the treatment safe?

A second key aim of Phase II trials is to identify whether a therapy has an acceptable level of safety such that its development is worth pursuing. Three categories of adverse events occur in trials. First, there may be minor but relatively common adverse effects (eg, mild skin rashes or other mild symptoms such as bruising) that may have little clinical impact as long as a benefit on a major clinical outcome (eg, death, MI, or strokes) is found. Although Phase II trials may detect these effects fairly reliably if they are of reasonable size, they do not necessarily affect the decision to proceed with the development of a therapy. A second category of adverse events is an increase in an expected cate- gory, such as bleeding with antithrombotic agents. If these events are minor, they may not be of great clini- cal relevance. If they are major, cause severe disability (eg, an intracranial bleed), or require surgical interven- tions, then they are clinically important. However, although Phase II trials are likely to be large enough to detect an excess in events such as minor bleeding, they are unlikely to be large enough to detect even a 2-fold increase in the rate of major bleeding. A third category of adverse events comprises rare but serious events (eg, idiosyncratic outcomes such as a potentially fatal aller- gic reaction, kidney failure, liver failure, or neutrope- nia). Often, given that the rates of these categories of adverse events are low, conventional Phase II trials are unlikely to detect such differences in treatment effects.

In summary, Phase II trials may detect whether a particular therapy increases the risk of common, and

American Heart Journal Volume 139 Number 4

Figure 3

SOLVD Data and Safety Monitoring Board Report, May, 1901 Prevention Trial Lan-Dameta Spending Function Results, Z Value vs. Tima

6

5

4-

s a3 - >

N

2-

1 -

0 0.2 0.4 0.6 Information Time

0.6 1

Interim results over the course of the SOLVD prevention trial. Early in the trial there appeared to be a large difference in mortality rates (P < .003), which became more modest by the end of the study. Based on data received at coordinating center through April 12, 1991. Assumes expected number of placebo deathsa322.

generally less important, adverse events; these trials also may detect large increases in major adverse events, which are generally less frequent than less important events.

Selection of the most appropriate dose

Often, because Phase III trials tend to be large trials, they can only assess the impact of a single dose of the therapy of interest compared with the control. Thus, during the conduct of Phase II trials, investigators usually test several doses of an agent to identify the most appro- priate dose (or mode of delivery) to be selected for the large trial. In cases in which reliable surrogate outcomes exist (or a measure of putative mechanism is presumed or known) and surrogate outcomes can be measured eas ily with little variability and few measurement errors (eg, BP lowering or cholesterol lowering), Phase II trials can potentially identify the dose of a study drug that pm vides the greatest efficacy. However, Phase II trials can rarely provide reliable evidence of the most appropriate dose of a drug that improves coronary patency or affects ventricular function unless these improvements are large by usual standards. Furthermore, the dose with the great- est impact on coronary patency or ventricular function may produce an unacceptable increase in the risk of major adverse events (eg, intracranial bleeding), but investigators may not be able to discover such a risk

without performing larger studies. This indeed has been the experience with many thrombolytic agents. For example, in the case of alteplase, the risk of intracra- nial bleeding was unacceptably high with a dose of 150 mg, but the impact of this dose on patency was appar- ently superior to those achieved by lower doses.14

To select the most appropriate dose of a therapy that reduces major clinical events in even moderately large Phase II trials is virtually impossible. Indeed, such selec- tion requires trials generally even larger than the usual Phase III trials, which explore whether a particular treatment produces different effects as compared with placebo. Perhaps dose effects of drugs may be best explored by using large trials with both composite clini- cal outcomes and easily measured surrogates (see below). The alternatives are to accept a dose based on its effects on surrogate outcomes or to select the high- est dose with an acceptable safety profile.

Summary of challenges in Phase II trials

The discussion above suggests that Phase II trials are generally more difficult to design and interpret than are Phase III trials. The moderate size and short dura- tion of Phase II trials generally limit what can be learned from them. Unless the surrogate outcomes used are clearly reliable, or until Phase II trials can be made much larger than they are at present, those who

s140 Yusuf American Heart Journal

April 2CXXl

.

Interim results over the course of the ISIS1 study. Early in the study (when approximately 120 deaths had accrued) there appeared to be a nominally significant (P = .05) excess in mortality rotes with intravenous P-block- ers. However, by the end of the study, there was a significant reduction in mortolity rate that was supported by the results of other trials (personal communication with S. Porish and R. Collins).

interpret results from Phase II trials should be extremely cautious. Some suggestions for the design of Phase II trials follow.

Suggestions for the design of Phase II trials

Use sequential, doseescalation designs or crossover designs when feasible. This suggestion is practical only when the underlying disease or condition is stable and one is not dealing with an acute outcome. It is more practical with surrogate outcomes or clinical outcomes that are reversible (eg, BP, anginal status, or heart fail- ure symptoms).

Consider multi-arm and factorial designs and explore trends. For example, a trial could test both treatment dose and duration of therapy with a factorial design. Thus two doses and two durations of treatment might be tested. Patients could be randomly assigned to dose 1 and dose 2 of the study drug and each dose could be further randomized to a shorter duration or longer duration. The relative impact of each dose could then be compared by using both durations of a dose versus control, and the impact of duration might be evaluated by combining the regimen with a specific duration ver- sus another duration or control. Further, one might examine dose-response relations (eg, cumulative doses) ln a structured fashion and examine for trends. Such an approach was used to evaluate BheothBx in the Collab

orative Organization for BheothRx Evaluation (CORE) study, in which the effects of 3 doses of this agent and 2 durations of each dose were tested against control, with both surrogate outcomes and clinical outcomes versus control in approximately 3000 patients.15 This study demonstrated that lower doses did not result in adverse events but neither did they have a favorable impact on efficacy outcomes. By contrast, higher doses of the drug reduced infarct size but increased the risk of renal dysfunction. Consequently, the development of BheothBx was discontinued.

Use a tiered approach that assesses both surrogate outcomes and clinical outcomes. If the surrogate is expensive or logistically difficult to measure, it could be measured in a subset of patients, whereas in the larger study, only simpler data on clinical outcomes would be measured. For instance, in the CORE study, infarct size and left ventricular function were measured in approximately 1000 patients, but clinical outcomes were measured in an additional 2000 patients.15

Combine Phase II and Phase III trials with the use of internal pilots. The trial could start with more than one arm for measuring a number of surrogate and clinical outcomes. After an interim analysis, one could drop one or more of the arms and proceed with the one that was the most promising. Although this design offers several attractive features, determining the exact crite- ria by which to choose the most promising arm is quite a complex procedure. Such a determination may be

Ametlcan Heart Journal Volume 139, Number 4

reliably made only when the study has included a sub stantial proportion of the overall sample size (eg, half or more of the sample) and issues about statistical adjustments of the probability value have been addressed. Therefore, although there may be potential efficiencies in such a design, it nevertheless requires the random assignment of fairly large numbers of patients into each arm in the Phase II part of the study, before the study moves into the Phase III part.

Enrich the study with patients at high risk. In some cases (eg, CHF), one may initiate the trial program by targeting the impact of therapy on major clinical out- comes (eg, death or heart failure hospitalizations) in patients at very high risk such as those with class IV CHF. Once this trial is completed, if it is favorable, future trials could examine the impact among individu- als with less severe disease. In a sense, the initial trials in this design would not be Phase II trials but highly tar- geted, moderate-sized Phase III trials. Such an approach has been used in the area of hypertension in which ini- tial trials were conducted among patients with severe hypertension (diastolic BP > 115 mm Hg).lG A clear reduction in major vascular events was demonstrated with the random assignment of approximately 200 patients followed for less than 2 years. Subsequent tri- als of several hundred to several thousand patients were conducted in patients at moderate risk (diastolic BP >105 mm Hg) and then in those with mild hyperten- sion (diastolic BP >95 mm Hg).i7 A similar approach was used in the evaluation of ACE inhibitors in heart failure, with the initial trials being conducted in those with New York Heart Association class IV heart failure (CONSENSUS)18 and later trials in those with class II and III CHF (SOLVD Treatment trial)ia or class I (SOLVD Prevention trial).8 This approach provides the advantage of a graded number of individuals entering the trials, but it has at least two disadvantages. First, although fewer patients may be needed initially, if these patients are difficult to recruit, the screening efforts may be substantial. Second, the stepwise approach may lead to an increase in the time required to learn about the effects of the therapy on the broader, and usually larger, group of patients.

Conclusions

Because Phase II trials impose certain limitations in study size and duration, they have lower study power as compared with Phase III trials to provide reliable answers about clinical outcomes. Yet the demands to be met in Phase II trials are greater than those in Phase III trials. For example, Phase II studies are expected to answer second-order questions (such as which dose is most appropriate) when they are rarely of sufficient size to explore even the main effects of treatment. Therefore a combination of efforts is needed-includ-

Yusuf 5141

ing judicious use of surrogate measures and a cluster of related clinical outcomes in fairly large Phase II trials. Unexpected favorable or unfavorable results for major clinical outcomes, relatively infrequent in Phase II tri- als, are generally unreliable and should be interpreted with great caution unless consistent results are observed on a range of related outcomes or on similar outcomes across a number of related trials. Even with this approach, the size of any treatment effect may be substantially overestimated. Therefore large treatment effects on clinical outcomes in Phase II trials should be viewed with skepticism. Considerable judgment is required when a promising intervention has the expected favorable effect on a surrogate outcome or on a composite of clinical outcomes, but the major clinical outcome (eg, death) unexpectedly shows an adverse effect. Even if this latter effect is statistically significant, it may well be a type I error unless extraneous data sup port this position.

Improving the design and conduct of Phase II trials, making these trials larger than traditional Phase II trials, and interpreting the trial results with caution, should ail lead in turn to the design and conduct of Phase III trials that are more definitive than current Phase III trials. These studies in concert should lead to a more reliable and more rapid evaluation of new treatments.

References 1.

2.

3.

4.

5.

6.

7.

8.

Yusuf S, Peto R, Lewis J, et al. Beta blockade during and after

myocardiol infarction: cm overview of the randomized trials. Prog

Cardiovasc Dis 1985;27:335-71.

Averum A, Tsuyuki RT, Yusuf S, et al. Beta blocker therapy for con-

gestive heart failure: a systematic overview and critical appraisal of

the published data. Can J Cardiol. 1998; 14: 1045-53.

Packer M, Bristow MR, Cohn JN, et al, for the US Carvedilol Heart

Failure Study Group. The effect of carvedilol on morbidity and mor-

tality in patients with chronic heart failure. N Engl J Med 1996;

334: 1349-5s.

ISIS (International Studies of Infarct Survival) Pilot Study Investiga-

tors. Randomized factorial trial of highdose intravenous strepto-

kinase, of oral aspirin and of intravenous heparin in acute myocar-

dial infarction. Eur Heart J 1987;8:634-42.

ISIS2 (Second International Study of Infarct Survival) Collaborative

Group. Randomised trial of intravenous streptokinase, oral aspirin,

both, or neither among 17,187 cases of suspected acute myocar-

dial infarction: ISIS2. Lancet 1988;1:34960.

ISIS3 (Third International Study of Infarct Survival) Collaborative

Group. lSlS~3: A randomised comparison of streptokinase vs tissue

plasminogen activator vs anistreplase and of aspirin plus heparin VI

aspirin alone among 4 1,299 cases of suspected acute myocardial

infarction. Lancet 1992;339:1-18.

Gruppo Italian0 per lo Studio della Streptachinasi nell’lnfarto Mio-

cardico (GISSI). Effectiveness of intravenous thrombolytic treatment

in acute myocardial infarction. Lancet 1986; 1:397-402.

SOLVD Investigators. Effect of enalapril on mortality and the

development of heart failure in asymptomatic patients with reduced

left ventricular ejection fractions. N Engl J Med 1992;327:

685-9 1.

5142 Yusuf American Heart Journal

April 2000

9. Feldman AM, Bristow MR, Pormley WW, et al. Effects pf vesnari-

none on morbidity and mortality in patients with heart failure: Ves

nortnone Study Group. N Engl J Med 1993;329: 14955.

10. Cohn JN, Goldstein SO, Greenberg BH, et al, for the Vesnarinone

Trial Investigators. A dose-dependent increase in mortality with

Vesnorinone among patients with severe heort failure. N Engl J

Med 1998;339: 1 B 1 O-6.

1 1. Moschio G, Alberti D, Janin G, et al. Effect of the ongiotensinton-

verttng-enzyme inhibitor benorepril on the progression of chronic

renal insufficiency: the Angiotensin-Converting Enzyme Inhibition in

Progressive Renal Insufficiency Study Group. N Engl J Med

1996;334:939-45.

12. Yusuf S, Sleight P, Rossi P, et al. Reduction in infarct size, orrhyth-

mios and chest pain by early intravenous beto blockade in sus-

pected acute myocardial infarction. Circulation 1983;67[suppl I]:l-

3241.

13. ISIS1 (First International Study of Infarct Survival) Collaborative

Group. Randomised trial of intravenous otenolol omong 16,027

cases of suspected acute myocordial infarction: ISIS-l. Loncet .

1986;1:57-66.

14. Braunwald E, Knotterud GL, Possomani ER, et al. Announcement of

protocol change in Thrombolysis in Myocardial Infarction trial. J Am Coil Cordiol 1987;9:467.

15. Colloborotive Orgonizotion for RheothRx Evoluation (CORE) Inves-

tigators. Effects of RheothRx on mortality, morbidity, left ventricular

function and infarct sire in patients with acute myocordial infarction.

Circulation 1997;96: 192.201.

16. Veterons Administrotion Cooperative Study Group on Antihyperten-

sive Agents. Effects of treatment on morbidity in hypertension: results

in patients with diastolic blood pressure averaging 1 15 through

129 mm Hg. JAMA 1967;202: 1028.34.

17. MocMahon S, Peto R, Cutler J, et al. Blood pressure, stroke, and

coronary heort disease, I: prolonged differences in blood pressure:

prospective observational studies corrected for the regression dilu- tion bias. Loncet 1990;335:765-74.

18. CONSENSUS Trial Study Group. Effects of enalopril on mortality

in severe congestive heort failure: results of the Cooperative North

Scondinovion Enolopril Survival Study (CONSENSUS). N Engl J Med 1987;3 16:1429-35.

19. SOLVD Investigators. Effect of enalopril on survival in patients with

reduced left ventricular ejection fractions and congestive heart foil-

ure. N Engl J Med 1991;325:293-302.