122

PAIN 02165

Comments on Ward et al., PAZN, 44 (1991) 151-155

We have read with interest the recent article by Ward et al.

(1991). We believe that the authors’ primary conclusion that “...

caffeine has a direct effect not mediated via affects on mood or on

caffeine withdrawal” is unwarranted, overstated, and certainly pre-

mature. At best, the study can and should only be considered as a

broad-ranging, loosely controlled, hypothesis-generating study that

should have been clearly identified as such in both the abstract and

introduction. A critical review of the paper identifies many problems

in the areas of study design and analysis that preclude definitive

conclusions. The most limiting of these problems are the following.

I. A cross-over design is generally recommended only for chronic

and relatively stable disease states that are predictable in their

course and duration (Wallenstein et al. 1990). From our experience,

we have observed that the etiology, frequency duration, and initial

pain of headaches all contribute to make headache episodes unique

between, and even within, patients. Using a h-way, cross-over design

in conjunction with such an acute, unpredictable condition as

headache along with indirect, statistical control of 2 important con-

founding factors such as mood and prior caffeine consumption

makes the study excessively complicated and the outcome difficult to

interpret.

2. The main conclusions of the study are based almost entirely on

the outcome at a singular point in time (i.e., 2 h after dosing).

Limiting an analgesic study to only 2 h does not permit key analgesic

parameters such as peak effect, duration of effect, or total effect

(e.g.. SPID and TOTPAR) to be estimated. These parameters must

be determined and compared in any evaluation of relative analgesic

Meon PlDs (Hour 2 1 by pertod

Peraod Treatment 1-1~ Placebo s--z-+ Acet only

?--T-x *+C65 -4-a-4 *+c130 ++i C65 6-C-6 Cl30

Fig. 1. From Ward et al., Pain, 44 (1991) 151-1.55, Table 111

efficacy. Also, with respect to the duration of the study, it is stated in

the treatment section of the paper that the subjects were instructed

to take study medication only if their headache occurred at least 6 h

before bedtime. This suggests to us that the study was at least

planned (and probably conducted) as a 6-h trial that would have

permitted the estimation of these key parameters.

3. Finally, among other questionable analytical methods, the

approach used to explore the significant sequence effects found in

the study is particularly inappropriate. By inspection of the results,

the authors concluded that there was “. no consistent sequence

effect biasing a particular treatment” and that separate analyses

performed comparing treatments within sequences continued to show

significant analgesic effects for the caffeine treatments. When the

possibility that significant carryover effects are present, a more

conventional and recommended approach is to consider the effects

of each treatment across time or study period (Jones and Kenward

1989). When the data from Table III are plotted by period (see Fig.

I), an essentially uninterpretable pattern of drug by period interac-

tions becomes apparent. The treatment effects are clearly unstable;

there are no periods where the order of the treatments are the same.

When faced with this outcome. one recommendation is to analyze

only the first period data, since it is these data that are completely

unaffected by any carryover effects. Even without formal analysis, it

can be seen that the first period data clearly do not support the

conclusions of the paper. The most obvious results are: (1) the study

appears insensitive since mean PIDs for acetaminophen (APAP)

alone (10.23), APAP+ mg of caffeine (14.07), and APAP+ 130 mg

of caffeine (17.24) are essentially equal to or less than placebo

(16.78); and (2) caffeine has no analgesic effect since (a) the results

for 130 mg of caffeine (14.52) are less than placebo (16.78) and (b)

the expected dose response relationship for caffeine alone is not

evident (C6.5 = 19.50; Cl30 = 14.52).

We believe that conclusions based on limited data from a com-

plex study such as this must be taken (and presented) with greater

caution than the authors convey.

References

Jones, B. and Kenward, M.G., Design and Analysis of Cross-over

Trials, Chapman and Hall. London, 1989, pp. 39-46.

Wallenstein. S.. Patel, H.I.. Fava. G.M. et al.. Two treatment

crossover designs. In: K.E. Peace (Ed.). Statistical Issues in Drug

Research and Development, Marcel Dekker. New York, 1990,

pp. 171-224.

Ward, N., Whitney, C., Avery, D. et al.. The analgesic effects of

caffeine in headache, Pain. 44 (1991) 151- 155.

James B. Nick

Katherine V. Mann

Anthony R. Temple