Int. 1. Radiation Onco/ogy Biol. Phys.. Vol. 7, pp. 1741-1742 Printed in the U.S.A. All rights reserved.

0360-30l6/8l/l21741~2~02.00/0 Copyright 0 1981 Pergamon R*is Ltd.

0 Correspondence

Letters to tbe Editor will be published if they are suitable and if space permits. Tbe Letter skould be typewritten (double spaced) and must not exceed II/4 pages including references; submit in triplicate. The Letter my be edited and sbortened in our Editorial Dtlice. A letter regarding a recent Jourd article should be received within six weeks of the article’s publication date.

CLINICAL TRIALS: STATISTICAL SIGNIFICANCE. AND CLINICAL IMPORTANCE

TO the Editor: In two recent articles in this journal, Levitt and Potish,’ and Potish er al.’ commented on the relationship between clinical importance and statistical significance as they pertain to the design and interpretation of clinical trials. The questions they raise are timely and appropriate. We hope that our comments on these papers will initiate a broader discussion.

The main difference between the views expressed by the authors and our own is that the concepts of clinical importance and statistical significance in a clinical trial are complementary rather than disparate. We agree that it is important not to confuse clinical importance with statistical significance; we disagree, however, in that we believe that no definite clinical importance can be ascribed to the result of a clinical trial before it has been shown that this result is statistically significant. Of course, the decision to accept a new treatment is not based on statistical significance alone. We must also consider the plausibility of the hypothesis on which it is based and other supporting evidence.

Levitt and Potish in reference’ discussed the clinical trials of Host and Brennhovd, and Fisher et al., showing an increased survival in patients treated by irradiation following mastectomy. Since statistical significance was not achieved in these particular studies, the results can only apply to the group of patients included therein and provide only a limited clue as to what might be expected in a larger patient population. We therefore take exception to the conclusion that “although statistical significance was not achieved in these studies, clinical significance (i.e. improved survival) was shown” (Ref. 1, p. 795). We concur with the authors when they criticize Stjernsward’s premature interpretation of a study in which the first 200 patients showed a 10% (statistically insignificant) difference in five-year survival in favor of patients not receiving radiotherapy, while later data from the same study show only a 2% difference. They appropriately warn against “misinterpretation, premature usage, and inappropriate utilization of information.” This warning, however, should apply equally to all studies.

In the second paper’ an attempt is made to clarify the distinction between statistical significance and clinical importance. The main point seems to be that large differences between treatments found in small trials may go unnoticed and not be used because statistical significance was not achieved. This subject is discussed in an extensive review on the design and interpretation of clinical trials by a group from several British and American centers.‘.> These authors give well-substantiated warnings against attributing too much importance to the results of relatively small trials.

Potish er ol.’ point out that the same statistical significance (P) in a small as in a large study means that a much larger and therefore probably more clinically important difference was demonstrated in a small study. This may be true if different treatments are compared in these studies; however, if the same two treatments are compared in different size studies and the same statistical significance is achieved in the large as in the small study, there are compelling reasons to attach more clinical importance to the larger study.**’ As pointed out in this reference, the value of P can be interpreted as a probability of imbalanced assignment of patients to two treatments. Such an imbal- anced assignment may magnify differences between treatments. This is particularly important when accepting results of small studies where proper randomization may not have been achieved.

We would also like to comment on the example given to illustrate the difference between clinical importance and statistical significance. The authors state that a large trial requiring 3000 patients would be necessary to have a 95% chance of achieving statistical significance in determining that a treatment (e.g. post-mastectomy irradiation) would result in a 5% increase in survival for breast cancer. This would mean 1500 lives saved in the United States every year, “clearly a clinically important advance.” One should not forget, however, that premature implementation of a treatment mode, which did not give a statistically significant improvement in a trial, may have a substantial probability of decreasing survival by 1500 patients.

Finally, we would like to comment on clinical importance per se. Throughout these papers,‘.’ and in our discussion, clinical importance has never been adequately defined and is assumed to be a generally self-evident concept. While we are unable to offer a precise definition of this term we believe this will eventually be necessary in the design and evaluation of clinical trials.

GEORGE SCHWARZ, M.D. ANDRZEJ SZECHTER, PH.D. WILLIAM P. KOWALSKY. M.S. Department of Radiation Medicine St. Vincent’s Hospital and Medical Center of New York New York, N.Y. 10011

Levitt, S.H., Potish, R.A.: The role of radiation therapy in the treatment of breast cancer: The use and abuse of clinical trials, statistics and unproven hypotheses. Int. J. Radiot. Oncol. Biol. Phys. 6: 791-798, 1980. Peto, R., Pike, M.C., Armitage, R., Breslow, N.E., Cox, D.R., Howard, S.V., Mantel, N., McPherson, K., Peto, J., Smith, P.G.: Design and analysis of randomized clinical trials requiring prolonged observation of each patient. 1. Introduction and design. Br. J. Comer 34: 585-612, 1976. Peto, R., Pike, M.C., Armitage, R., Breslow. N.E., Cox, D.R., Howard, S.V., Mantel, N., McPherson, K., Peto, J., Smith, P.G.: Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. Analysis and examples. Br. J. Comer 34: l-39, 1977. Potish, R., Boen, J., Levitt, S.: The distinction between statistical significance and clinical importance. Inf. J. Rudiuf. Oncol. Biol. Phys. 6: 941-944, 1980.

REBUTI-AL

To the Editor: We are writing in reply to the letter by Schwarz, Szechter, and Kowalsky in which they take issue with some of the points raised by the 1979 Presidential Address of the 21st Annual ASTR meeting.

Their letter differs with the Presidential Address mainly on the relationship of statistical significance to clinical importance. In partic- ular, we focus on the statement in their fifth sentence, “. . . no definite clinical importance can be ascribed to the result of a clinical trial before it has been shown that this result is statistically significant.” We infer from that statement, as well as from others in their letter, that they believe that a clinical trial cannot demonstrate that one treatment is clinically superior to the other unless the trial outcomes are statistically significantly different. We disagree.

In particular, clinical importance does not depend on the outcome of a clinical trial, but depends instead on the medical and ethical values of the treatment deciders (the treatment deciders are physicians and their patients). One treatment is clinically better than another if its advantages over the other outweigh its disadvantages. For example, if Treatment A gives a longer remaining life than Treatment B, but with

1741