Upload
danghanh
View
226
Download
1
Embed Size (px)
Citation preview
Current Issues in the Design of Cluster
Randomization Trials
Allan Donner, PhD, FRSC
Department of Epidemiology and Biostatistics
The University of Western Ontario
London, Canada
Robarts Clinical Trials,
Robarts Research Institute
London, Canada
What Are Cluster Randomization Trials
Cluster randomization trials are experiments in
which intact social units or clusters of individuals
rather than independent individuals are randomly
allocated to intervention groups.
Examples:
Medical practices selected as the randomization
unit in trials evaluating the efficacy of disease
screening programs
Communities selected as the randomization unit
in trials evaluating the effectiveness of new
vaccines in developing countries
Hospitals selected as the randomization unit in
trials evaluating educational guidelines directed at
physicians and/or administrators
Reasons for Adopting Cluster Randomization
Administrative convenience
To obtain cooperation of investigators
Ethical considerations
To enhance subject compliance
To avoid treatment group contamination
Intervention naturally applied at the cluster level
Role in vaccine trials
Randomization of geographic areas can be used to
capture indirect (herd ) effects of vaccination.
Incidence of disease among non-vaccinated individuals
in the intervention group is compared to incidence of
disease in the control group.
Unit of Randomization vs. Unit of Analysis
A key property of cluster randomization trials is
that inferences are frequently intended to apply
at the individual level while randomization is at
the cluster or group level. Thus the unit of
randomization may be different from the unit of
analysis.
In this case, the lack of independence among
individuals in the same cluster, i.e., intracluster
correlation, creates special methodological
challenges in both design and analysis.
Quantifying the Effect of Clustering
Consider a trial in which k clusters of size m are
randomly assigned to each of an experimental and
control group denoted by i =1 and 2 respectively. Let
denote the sample mean of the response variable in
the group
Then assuming is normally distributed with common
variance
where is the coefficient of intracluster correlation.
Equivalently,
If clusters of size are randomized to
each of two treatment groups,
then the effective sample size per group is
given by
Application of standard sample size approaches
leads to an underpowered study (Type II error)
Application of standard statistical methods
generally tends to bias p-values downwards, i.e.,
could lead to spurious statistical significance
(Type I error)
Examples:
COMPLETELY RANDOMIZED DESIGN
Study Purpose:
To evaluate the effectiveness of vitamin A
supplements on childhood mortality.
450 villages in Indonesia were randomly assigned
to ether participate in a vitamin A supplementation
scheme, or serve as a control. One year mortality
rates were compared in the two groups.
Sommer et al. (1986)
MATCHED PAIR DESIGN
Study Purpose:
The COMMIT community intervention trial(1995) was
designed to promote smoking cessation using a
variety of community resources. The primary
outcome measure was the 5-year smoking cessation
rate.
Unit of Randomization: Community
Number of Matched Pairs: 11
Matching factors: size, pop. density
STRATIFIED DESIGN
Study Purpose:
The purpose of the WHO antenatal trial was to
compare the impact of two programmes of antenatal
care on the health of mothers and newborns.
Unit of Randomization: Antenatal care clinic.
Stratification Variables: Primary stratification was
by country: Thailand,
Cuba, Argentina, Saudi
Arabia.
Number of
Clusters per Stratum: Ranged from 12 to 17.
Villar et al. (2001)
CLUSTER-CROSSOVER DESIGN
All participating clusters receive both intervention and
control in a sequence determined at random
Reduces total number of clusters required with a
considerable increase in study duration
Turner(2007)
STEPPED WEDGE DESIGN
All clusters eventually cross over but only from the
control to intervention at a time point determined at
random
Allows clusters to be enrolled gradually over time
Hussey and Hughes(2007)
Key question:
Are the trial inferences aimed at
(i) the individual subject? or
(ii) at a naturally defined cluster of individuals?
This decision guides both the choice of design and
the approach to the analysis
Selecting the Unit of Inference
Example 1:
The intervention consists of a blood pressure
screening program intended to lower the risk of
cardiovascular mortality
Bass et al (1986)
Example 2:
The intervention consists of educational
guidelines for the management of hyperlipidaemia
intended to increase the proportion of eligible patients
prescribed lipid-lowering drugs
Diwan et al (1995)
Consider a trial randomizing hospitals designed
to assess the effect of obtaining a second clinical
opinion on the decision to proceed with a
caesarian section operation (Altabe et al 2008).
The target of the intervention was the hospital rate
of caesarian section.
In this case, the hospital was the natural unit of
inference and standard methods of sample size
estimation and analysis applied at the cluster
level.
Key point:
If the unit of inference is at the cluster level then an
analysis at the cluster level is appropriate, and no
consideration need be given to the intracluster
correlation coefficient.
From the perspective of sample size estimation and
analysis the challenges are no different from those
that arise in individually randomized trials.
Simplified data collection and lower cost
Can be applied to any outcome variable
Exact statistical inferences can always be constructed
Can be adapted to adjust for baseline imbalances
Informed consent issues may be eased
Cluster level Analyses:Advantages
A Common Misconception
Investigators have occasionally claimed that cluster
level analyses will only provide valid statistical
inferences when the intracluster correlation
coefficient
However this is too stringent a claim.
For balanced designs, cluster level analyses
provide inferences which are equivalent to those
obtained using a mixed linear regression model
for any value of
Threats to trial validity?
Selection bias in the recruitment of patients over time
Experimental contamination
i)Risk of selection bias
Consider a trial using a simple randomization scheme
practices in which physicians are asked to identify as
well as to treat selected patients (e.g. Kinmouth et al
[1998]).
Can cluster randomization introduce bias through the
way patients are differentially recruited across
treatment groups?
Source of bias:
If the physician’s practice has already been randomized, recruitment for patient participation cannot be done blindly with respect to intervention group.
If physicians in the experimental group are more diligent in seeking out patients than in the control group, or tend to identify patients who are less ill, bias may result.
Unbiased estimates of the effect of intervention can be assured only if analyses are
(i) based on data from all cluster members
or
(ii) based on a random sub-sample of cluster
members or
Possible Solutions:
Identify eligible patients in each practice prior to randomization.
If eligible patients are identified after randomization, recruitment should be done by an individual independent of the trial.
Torgerson (2001) Farrin et al (2005)
ii) Is fear of contamination a legitimate
reason adopting cluster randomization?
Suppose under individual randomization a
proportion of R of the control group patients experience
the same success rate P1 as seen among experimental
patients.
Then the difference that can be detected is
reduced to
and the required sample size must be inflated by the
factor IF = 1/ (1-R)2
e.g. If R =0.30, then IF =2.04
But under unrestricted cluster randomization, the
inflation factor 1+(m-1) might be more
Farrin et al (2005)
However…..
Uncertainty concerning true level of contamination
Presence of contamination under individual
randomization will lead to an underestimate of the
effect of treatment
Logistical difficulties a factor-giving different
treatments at random to different patients in the same
office.
Can create pairs of clusters between which travel is
difficult.
Issues Involving Informed Consent
Two distinct levels of informed consent must be
distinguished in cluster randomization trials:
(i) informed consent for randomization (usually
provided by a ‘decision-maker’)
(ii) informed consent for participants given that
randomization has occurred.
By analogy to current ethical requirements for
clinical trials, it would be unethical not to obtain
informed consent from every cluster member prior to
random assignment.
Is such a strict analogy required for
community randomized trials?
“Ethical advice indicated that, since we were only
providing information to clinicians, there was no
reason to seek patient consent”
Wyatt et al (1998)
“Eliciting consent from the many thousands of patients
involved would create severe budgetary and logistic
obstacles.
The central human rights committee at the Hines
Coordinating Center and those at the participating
sites have deemed this type of quality improvement
research exempt from direct patients consent”.
US Dept. of Veterans Affairs
The Effect of Level of Intervention
Edwards et al. (1999) distinguished cluster
randomization trials by the level of intervention:
for “individual-cluster” trials intervention is provided
directly to individual study subjects (e.g., STD
treatment for prevention of HIV).
for “cluster-cluster” trials intervention is provided at
the cluster level (e.g., effect of local medical opinion
leaders on patient treatment).
The need for consent by individual study subjects is
deemed of particular concern for “individual cluster”
trials.
The need for consent by individual study subjects is
deemed of particular concern for “individual cluster”
trials.
However these distinctions may only be relevant
when randomizing larger clusters (e.g., work sites,
classrooms, communities).
Is Informed Consent Required for Cluster-
Cluster Trials?
“Informed consent to randomization may not be
required if it is not possible to approach
subjects at the time of randomization.
However potential subjects approached after
randomization must be provided with a detailed
description of the interventions in the trial arm to
which their clusters have been randomized”
McRae et al(2009)
Cluster Randomization Trials and the Zelen Randomized Consent Design
The requirements of informed consent are often perceived to be a barrier to patient accrual. Randomized consent designs were introduced by Zelen (1990) to lower this barrier, thus hastening trial completion.
In the single consent design patients are randomly assigned to one of two groups , but with only patients in the intervention group asked to provide informed consent. If not, they may be offered the control therapy.
Analyses are conducted based on the intervention to
which patients were initially assigned, i.e., analysis by
intention to treat.
Randomized consent designs have proved quite
controversial because of concern for the ethical
implications of randomizing subjects prior to obtaining
their consent.
But this strategy is typical of many cluster
randomized trials!
Factors Influencing Loss of Precision
Cluster Randomization Trials
1. Interventions often applied on a group basis
2. Some studies permit the immigration of new
subjects after baseline.
3. Entire clusters, rather than just individuals, may be
lost to follow-up.
4. Difficulties associated with prevention trials
Why are so many CRT’s unpowered?
“Generally the size of effects has been meager in
relation to the effort expended”
“We often do not have the resources to detect medium
effects”
Susser (1995)
Strategies for Increasing Precision
Restrict inclusion criteria
e.g. recruit practices of same size, with physicians having similar years of experience in a controlled geographic area.
Conduct a baseline survey of the prevalence of the trial outcome
- serves as powerful risk factor for outcome
- provides estimate of
- allows trial personnel to gain experience
- can help to identify potential stratification factors
Increasing cluster size provides diminishing
returns in statistical power if
e.g. If increasing beyond 100 provides little
statistical gain.
Donner and Klar (2004)
Consider sample size re-estimation as part of an
interim analysis.
Lake et al (2001)
Recognize likely size of in primary care
research (0.01 to 0.05 for outcome variables; 0.05
to 0.15 for process variables).
Campbell et al (2001)
Gulliford et al (2004)
IMPACT ON POWER OF INCREASING THE NUMBER
OF CLUSTERS VS. INCREASING CLUSTER SIZE:
Let
As
As
Statistical Challenges in the
Meta-Analysis of Cluster
Randomization Trials
Meta-analyses combining cluster randomization trials raise methodologic issues beyond those raised by meta-analyses which include only individually randomized trials:
- Increased Trial Heterogeneity
- Difficulties in Estimating Design Effects from Individual Trials
- Special Issues Involving Publication Bias
- Choice of Statistical Method
- Assessment of Trial Quality
Study Heterogeneity
Sources of heterogeneity in meta-analyses involving cluster randomization trials:
(i) Heterogeneity of study design e.g., completely randomized, matched-pair, stratified
(ii) Heterogeneity in unit of randomization e.g., worksites
schools
households
individuals
(iii) Variation in eligibility criteria at both the cluster and individual level
(iv) Wide variation in methodological quality
Example of Design Heterogeneity in a
Meta-Analysis:
Brunner et al. (1997) reported on a meta-analysis of dietary interventions aimed at reducing cardiovascular risk factors.
17 randomized trials with a total of 6893 participants were included in the meta-analysis.
16 of the trials were individually randomized, one was cluster randomized.
The one cluster randomized trial (allocating worksites) included approximately 42% of the 6893 participants.
In this case, the effective sample size for the meta-analysis is clearly much less than 6893.
Difficulties in Estimating Design Effects
from Individual Trials
For trials randomizing clusters of size m to two
intervention groups, ‘design effect’ is given by
Estimation of the design effect is crucial for optimal
weighting of the trials to be combined.
However many trial reports fail to provide estimates of the design effect (Laopaiboon [2003], Thomas et al. [2003]). Therefore meta-analysts have resorted to various ad-hoc strategies to obtain the required information:
(i) increasing the variances of estimated intervention effects by an arbitrary amount (e.g., Fawzi et al. [1993]).
(ii) using external data to obtain imputed values of the design effect (e.g., Rooney and Murray [1996]).
(iii) using regression methods to estimate design effects from information on design effects reported in other trials (e.g., Beaton et al. [1993]).
(iv) excluding cluster randomization trials from a meta-analysis (e.g., Xin et al. [2003]).
Assessing Quality of Individual Trials
Issues here particularly relevant to cluster randomization trials include:
1. Justification for randomizing clusters rather than individuals.
2. Unambiguous definition of the randomization unit.
3. Eligibility criteria defined at both the individual and cluster level.
4. Accounting for clustering in both the estimation of trial size and in the analysis.
5. Reporting of observed intracluster correlation and/or design effect.
Special Issues Involving Publication Bias
Complication in cluster randomization trials is that many
such studies ignore the clustering in the analysis. These
studies may then be reported as statistically significant
when in fact a correctly performed analysis would not show
significance.
Conversely, if the analysis is correctly performed but power
considerations were ignored in the design, a finding of non-
significance may be misleading.
Combining Trials in Which One Cluster Only
Has Been Assigned to Each Group
Community intervention trials have been reported in which
exactly one cluster has been assigned to the intervention
group and one to the control group.
These invariably result in interpretational difficulties due to
the confounding of two sources of variation:
(i) variation in response due to the effect of intervention.
(ii) natural variation that exists between two clusters even in the
absence of an intervention effect.
However such trials could potentially be included in a meta-
analysis provided the randomization unit and subject
population for such studies were comparable to those of the
other trials included. Under these conditions, the
intracluster correlation could also be assumed to be
comparable to, and hence estimable from, the other trials.
SOME FUTURE CHALLENGES:
What are the best methods of quantifying trial
heterogeneity?
Development of efficient meta-regression techniques
Methods for combining individually randomized trials
with cluster randomized trials
OTHER ISSUES:
1. Is it fair to combine cluster randomization trials that have used very different units of randomization? Or very different designs?
2. Since estimates of intracluster correlation and design effects are often unreported, what are the best strategies for dealing with this problem?
3. What are the best methods to adjust for clustering effects in meta-analyses for different choices of endpoints and study designs?
3. Are fixed effects or random effects models for cluster trials most appropriate?
4. Should trials which include only one cluster per intervention group be included in a meta-analysis?
5. Can checklists be developed that assign quality scores to individual trials? If so, how many points should be deducted when a study ignores clustering in the design or analysis?
Why adopt a non-randomized design ?
Intervention has already been implemented on a
major scale, raising ethical issues.
Often less expensive, easier logistics
Easier to explain to public officials ,gain public
acceptance.
The availability of a small number of clusters is not a
reason to avoid a randomized design!
Can match or stratify in the design
Possibility of imbalance remains a problem
Estimation of the ICC: unresolved issues
i)Confidence interval construction for community
intervention trials having a large number of small
clusters and a binary outcome.
ii)Definition of the ICC for time-to-event outcomes with
censored data.
- Includes a checklist of items that should be included in a trial report
- Extends CONSORT statement for reporting of individually randomized trials (Begg et al, JAMA, 1996)
- Key Points • Rationale for adopting cluster design
• Incorporation of clustering into sample size estimation and analysis
• Chart showing flow of clusters through the trial, from assignment to analysis
EXTENSION OF CONSORT STATEMENT TO
CLUSTER RANDOMIZATION TRIALS
(CAMPBELL ET AL, BMJ, 2004)