394 EQUINE VETERINARY JOURNALEquine vet. J. (2005) 37 (5) 394-395

Editor’s comment

This series includes articles and commentary addressing issuesrelating to the design and execution of clinical research studiesand the implementation of their conclusion in clinical practice.Randomised controlled trials (RCTs) are considered the goldstandard for estimation of treatment effects and are becomingincreasingly commonplace in veterinary medicine. The followingarticle, reproduced from Evidence Based Medicine (Vol 5,March/April 2000, pp 36-37), discusses the importance of caseallocation and blinding in reducing bias in randomised controlledtrials and highlights the need for transparent reporting.

Background

Random allocation to intervention groups remains the onlymethod of ensuring that the groups being compared are on anequivalent footing at study outset, thus eliminating selection andconfounding biases. This technique has allowed RCTs to play akey role in advancing medical science.

The success of randomisation depends on 2 interrelatedprocesses (Schulz 1995a, 1996). The first entails generating asequence by which the participants in a trial are allocated tointervention groups. To ensure the unpredictability of thatallocation sequence, investigators should generate it by a randomprocess. The second process, allocation concealment, shieldsthose involved in a trial from knowing upcoming assignments.Without this protection, investigators and patients have beenknown to change who gets the next assignment, making thecomparison groups less equivalent (Pocock 1982; Chalmers et al.1987; Schulz et al. 1994, 1995a).

For example, suppose that an investigator creates an adequateallocation sequence using a random number table. However, theinvestigator then affixes the list of that sequence to a bulletinboard, with no allocation concealment. Those responsible foradmitting participants could ascertain the upcoming treatmentallocations and then route participants with better prognoses to theexperimental group and those with poorer prognoses to the controlgroup, or vice versa. Bias would result. Inadequate allocationconcealment also exists, for example, when assignment to groupsdepends on whether a participant’s hospital number is odd or evenor on translucent envelopes that allow discernment of assignmentswhen held up to a light bulb.

Allocation concealment should not be confused with blinding.Allocation concealment concentrates on preventing selection andconfounding biases, safeguards the assignment sequence beforeand until allocation, and can always be successfully implemented(Schulz 1995a, 1996). Blinding concentrates on preventing studypersonnel and participants from determining the group to whichparticipants have been assigned (which leads to ascertainmentbias), safeguards the sequence after allocation, and cannot alwaysbe implemented (Pocock 1982; Chalmers et al. 1987; Schulz et al.1994, 1995a; Schulz 1995a,b, 1996).

Reporting of methods

Investigators must not only minimise bias but must alsocommunicate those efforts to the reader. Readers should not haveto assume or guess the methods used. Yet assessments of thereporting quality of published trials have consistently found majorflaws (Mosteller et al. 1980; DerSimonian et al. 1982; Tyson et al.1983; Altman and Dore 1990; Schulz et al. 1994; Sonis and Joines1994; Williams and Davis 1994; Moher et al. 1996). Only 9% oftrials in the specialist journals and 15% in the general journalsreported both an adequate method of generating random sequencesand an adequate method of allocation concealment (Altman andDore 1990; Schulz et al. 1994, 1995b). Of trials reported as doubleblind, only 45% described similarity of the treatment and controlregimens, and only 26% provided information on the protection ofthe allocation schedule (Schulz et al. 1996). Most reports simplyprovide no information on methods.

With so little relevant information, many of us resort toinappropriate markers of trial quality. Two noteworthy examplesare described here. First, many designate a trial as high quality ifit is ‘double blind’, as if double blinding is the sine qua non of anRCT. Although double blinding can reflect good methods, it is notthe sole criterion of quality. As the author shall discuss later,adequate allocation concealment actually appears to be the moreimportant indicator. Moreover, many trials cannot be doubleblinded. Those trials must be judged on other merits and not on aninapplicable standard based on double blinding.

Second, some assume that a good-quality trial contains groupsof equal size, while a poor-quality trial contains groups of unequalsize. That standard applies only when the investigators use arestricted randomisation generation scheme that aims for equality.A simple randomisation method will seldom yield equal sample

Clinical Evidence NotebookAssessing allocation concealment and blinding inrandomised controlled trials: why bother?K. F. SCHULZ

Center for Disease Control and Prevention, Atlanta, Georgia, USA.

Keywords: horse; clinical evidence; randomsied controlled trials; allocation; blinding

K. F. Schulz 395

sizes. In fact, equal numbers in treatment groups may mean thatsome process other than randomisation was used, for example,allocation of every second patient to the intervention group or theuse of odd and even birth dates or chart numbers as a way toassign participants to study groups.

Although RCT reporting remains weak, it is improving.Methodologists, editors and clinicians addressed the prevailingflaws in reporting by publishing the Consolidated Standards ofReporting Trials (CONSORT) statement (Begg et al. 1996).Currently, more than 70 journals have adopted the standards,including such high-profile general medical journals as Journal ofthe American Medical Association, the Lancet, British MedicalJournal and Annals of Internal Medicine. Yet, even withimprovement, readers of RCTs should be wary of the informationprovided in many current trial reports.

Empirical evidence of bias

Recent studies have shown that poor-quality RCTs and poorlyreported RCTs yield biased results. For example, in a study of 250controlled trials from 33 meta-analyses in pregnancy andchildbirth, investigators found that alleged RCTs with inadequateand unclear allocation concealment yielded larger estimates oftreatment effects (41% and 33%, respectively, on average) thantrials in which authors reported adequate concealment (Schulz etal. 1995a). Investigators found similar results for trials indigestive diseases, circulatory diseases, mental health and stroke(Moher et al. 1998). Those trials that used inadequate or unclearallocation concealment yielded 37% larger estimates of effect, onaverage, than those that used adequate concealment.

These exaggerated estimates of treatment effects revealmeaningful levels of bias. If a study is designed to detect adecrease in mortality of 25% or 50% from a particular treatment,biases of 30% to 40% would overwhelm estimates of thetreatment effect. The elimination of bias is crucial in trialsdesigned to detect moderate effects.

Double blinding also appears to reduce bias. Trials that werenot double blinded yielded larger estimates of treatment effectsthan did trials in which authors reported double blinding (oddsratios exaggerated, on average, by 17%) (Schulz et al. 1995a).Another recent analysis has also noted the importance of doubleblinding (Khan et al. 1996). However, although double blindingappears to prevent bias, its effect appears weaker than that ofallocation concealment. Indeed, Moher and colleagues found littleeffect from double blinding (Moher et al. 1998).

Conclusions

As users of RCT results, we must understand the potential forhumans to interject bias. By including assessments of allocationconcealment and double blinding, abstracts in this journal will

help readers to discern those trials that have made superior effortsto minimise bias. Judging the quality of allocation concealmentand blinding reflects current empirical research and reflects thecommitment of the editors of this journal to apply the principles ofevidence-based medicine to the practice of reporting.

References

Altman, D.G. and Dore, C.J. (1990) Randomisation and baseline comparisons inclinical trials. Lancet 335, 149-153.

Begg, C., Cho, M., Eastwood, S., Horton, R., Moher, D., Olkin, I., Pitkin, R., Rennie,D., Schulz., K.F., Simel, D. and Stroup, D.F. (1996) Improving the quality ofreporting of randomized controlled trials. The CONSORT statement. J. Am. med.Ass. 276, 637-639.

Chalmers, T.C., Levin, H., Sacks, H.S., Reitman, D., Berrier, J. and Nagalingam, R.(1987) Meta-analysis of clinical trials as a scientific discipline. I: Control of biasand comparison with large co-operative trials. Stat. Med. 6, 315-328.

DerSimonian, R., Charette, L.J., McPeek, B. and Mosteller, F. (1982) Reporting onmethods in clinical trials. N. Engl. J. Med. 306, 1332-1337.

Khan, K.S., Daya, S., Collins, J.A. and Walter, S.D. (1996) Empirical evidence ofbias in infertility research: overestimation of treatment effect in crossover trialsusing pregnancy as the outcome measure. Fertil. Steril. 65, 939-945.

Moher, D., Pham, B., Jones, A., Cook, D.J., Jadad, A.R., Moher, M., Tugwell, P. andKlassen, T.P. (1998) Does quality of reports of randomised trials affect estimatesof intervention efficacy reported in meta-analyses? Lancet 352, 609-613.

Moher, D., Fortin, P., Jadad, A.R., Jüni, P., Klassen, T., Le Lorier, J., Liberati, A.,Linde, K. and Penna, A. (1996) Completeness of reporting of trials published inlanguages other than English: implications for the conduct and reporting ofsystematic reviews. Lancet 347, 363-366.

Mosteller, F., Gilbert, J.P. and McPeek, B. (1980) Reporting standards and researchstrategies for controlled trials: agenda for the editor. Control Clin. Trials 1, 37-58.

Pocock, S.J. (1982) Statistical aspects of clinical trial design. Statistician 31, 1-18.

Schulz, K.F. (1995a) Subverting randomization in controlled trials. J. Am. med. Ass.274, 1456-1458.

Schulz, K.F. (1995b) Unbiased research and the human spirit: the challenges ofrandomized controlled trials. Can. med. Ass. J. 153, 783-786.

Schulz, K.F. (1996) Randomized trials, human nature, and reporting guidelines.Lancet 348, 596-598.

Schulz, K.F., Chalmers, I., Grimes, D.A. and Altman, D.G. (1994) Assessing thequality of randomization from reports of controlled trials published in obstetricsand gynecology journals. J. Am. med. Ass. 272, 125-128.

Schulz, K.F., Chalmers, I., Grimes, D.A. and Altman, D.G. (1995a) Empiricalevidence of bias. Dimensions of methodological quality associated withestimates of treatment effects in controlled trials. J. Am. med. Ass. 273, 408-412.

Schulz, K.F., Chalmers, I., Altman, D.G., Grimes, D.A. and Doré, C.J. (1995b) Themethodologic quality of randomization as assessed from reports of trials in specialistand general medical journals. Online J. Curr. Clin. Trials 26, 4 (Doc No 197).

Schulz, K.F., Grimes, D.A., Altman, D.G. and Hayes, R.J. (1996) Blinding andexclusions after allocation in randomised controlled trials: survey of publishedparallel group trials in obstetrics and gynaecology. Br. med. J. 312, 742-744.

Sonis, J. and Joines, J. (1994) The quality of clinical trials published in The Journalof Family Practice, 1974-1991. J. Fam. Pract. 39, 225-235.

Tyson, J.E., Furzan, J.A., Reisch, J.S. and Mize, S.G. (1983) An evaluation of thequality of therapeutic studies in perinatal medicine. J. Pediatr. 102, 10-13.

Williams, D.H. and Davis, C.E. (1994) Reporting of assignment methods in clinicaltrials. Control Clin. Trials 15, 294-298.