Dog-legs, Ladders and a CONSORT statement - Alan Girling and Karla Hemming

CLAHRC West Midlands

Beyond Stepped-Wedge

Dog-legs, Ladders and a

CONSORT statement

Alan Girling, Karla Hemming

University of Birmingham

06/07/2016 CLAHRC West Midlands


Stepped Wedge Designs

The State of the Art


Background: Cluster Studies over Time

• In contrast to an RCT, the intervention is delivered to a

Cluster, or to all members of a Cluster

• Measurements taken over time

– On new individuals (Cross-sectional designs)

– Repeatedly, on the same individuals (‘Cohort’ design)

– Repeatedly, on a shifting sample of individuals (Open

Cohort)

• At any one time, all members of a cluster experience the

same experimental condition


‘Traditional’ Cluster Designs

Parallel Study

Clu

ste

rs

Time

Treated

Controls


‘Traditional’ Cluster Designs

Parallel Study Cross-Over Before & After

Clu

ste

rs

Time

Before & After with

Control Clusters


Stepped Designs

Classic (‘Complete’)

Stepped Wedge SW with Transition

Periods (‘Incomplete’)

‘Extended’ SW

Clu

ste

rs

Time

‘Hybrid’

An Essential

Feature here is that

Reverse Cross-

Over is not allowed


Stepped Designs in Practice

• Why Stepped-Wedge?

– Service-Delivery Interventions

– Logistical constraints

– Opportunistic evaluations

• Stepped-Wedge designs in Healthcare

– Prevalence is Low but Increasing

– Systematic Reviews (Brown 2006, Mdege 2011, Beard 2015,

Martin 2016, Barker 6th June 2016)

– Landmark studies (EPOCH,…)

• SW designs have been widely criticised in some quarters

– Implementation Issues

– Limitations of the experimental setting (not specific to SW)

– Sometimes SW is the only feasible possibility for an evaluation


(Statistical) Methodology for SW

• In a stepped-design the proportion of measurements under the treated

condition increases with time.

(Time is a potential ‘Confounder’)

• ‘Standard’ Statistical Model (Hussey & Hughes, 2007)

– Additive treatment and time effects

– Observations not independent - Correlation structure reflects variation

between and within clusters through the ICC

(exactly as in standard model for parallel cluster studies)

– Model used for analysis and (especially) for sample size calculations

– Straightforward extensions available to some other correlation

structures, encompassing cross-sectional & cohort studies, time-varying

cluster effects, nested clusters…

• ‘Vertical Analysis’ has been proposed as an alternative

– Loss of information (but efficiency has not been studied??)


Issues for SW Methodology

• Many issues for SW are mirrored in other designs (hitherto ignored or

glossed over) and are related to the setting rather than the design per se.

• “Inclusion of time-effects is opaque, and unnecessary in ‘traditional’ designs”

– Omission of time-effects in a Parallel or Cross-Over design can inflate

standard errors and lead to loss of power

• “Additivity of time & treatment effects is unwarranted”.

– Additivity (on some scale) implicitly assumed whenever measurements

are taken at different times (including in Parallel Studies)

• “The methodology for SW designs is sensitive to mis-specification of the

correlation structure”

– This applies also to ALL traditional designs.

(the community is only beginning to come to terms with this.)

• SW suffers from natural “aversion to complexity”

– “Everything should be made as simple as possible, but not simpler”


Recent Papers

• Girling A.J. & Hemming K. (2016) Statistical efficiency and optimal design for stepped cluster

studies under linear mixed effects models. Statistics in Medicine DOI: 10.1002/sim.6850

• Hemming K & Taljaard M (2016) Sample size calculations for stepped wedge and cluster

randomised trials: a unified approach. J. Clin. Epidemiol. Jan;69:137-46. doi:

10.1016/j.jclinepi.2015.08.015. Epub 2015 Sep 5.

• James Martin, Monica Taljaard, Alan Girling, and Karla Hemming (2015) A systematic review

finds major deficiencies in sample size methodology and reporting for stepped wedge cluster

randomised trials. BMJ Open 2016;6:e010166 doi:10.1136/bmjopen-2015-010166

• Hemming K., Haines T.P., Chilton P.J , Girling A.J., Lilford R.J. (2015) The stepped wedge cluster

randomised trial: rationale, design, analysis and reporting. BMJ. BMJ 2015; 350 doi:

http://dx.doi.org/10.1136/bmj.h391

• Karla Hemming, Richard Lilford and Alan J. Girling (2015) Stepped-wedge cluster randomised

controlled trials: a generic framework including parallel and multiple-level designs. Statistics in

Medicine 34(2), 181-196. DOI: 10.1002/sim.6325

• K. Hemming and A.J. Girling (2014) A menu-driven facility for power and detectable-difference

calculations in stepped-wedge cluster-randomized trials The STATA Journal 14(2): 363-380

+ Conference/Workshop contributions

… SCT2015, Trials meeting, RSS, York, SCT2016, ISCB2016, …

SW & WM CLARHC Karla Hemming, Richard Lilford, Alan Girling + Collaborators


Future Projects (1)

Extension of CONSORT statement to

SW cluster randomised controlled trials

Karla Hemming1, Alan Girling1, Monica Taljaard2

, Andrew

Forbes3, Richard Lilford4 and Jeremy Grimshaw2

1 University of Birmingham, Birmingham, UK 2 Ottawa Hospital Research Institute, Ottawa, Canada 3 Monash University, Melbourne, Australia 4 University of Warwick, Coventry, UK

• The CONSORT (CONsolidated Standards of Reporting Trials)

guidelines are intended to improve the reporting of RCTs

• Consist of:

– Statement

– Checklist

– Explanation and Elaboration document

• Initially developed for the RCT, extensions exist for different trial

designs:

– Cluster trial etc. .

The CONSORT Statement

Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010 Statement: updated

guidelines for reporting parallel group randomised trials. Ann Int Med 2010;152. Epub 24 March.

http://annals.org/article.aspx?articleid=745807





Protocol registered with EQUATOR

a searchable database of 317 reporting guidelines

Includes existing CONSORT guidelines (also STROBE & PRISMA)

Outline process:

Systematic review of quality of reporting SW-CRTs published to date

(Martin et al, 2016)

WIP: Systematic review of

Methodology

current recommended reporting items for SW-CRTS

Progress to date

Map reporting items identified from the review process to CONSORT

statement for cluster trials

Identify advisory team – with expertise in cluster trials, reporting

statements, statistics, clinical trials, ethics, policy development …

Consensus meeting Liverpool May 2017

– Produce

Statement

Checklist

Explanation and Elaboration document

Next Steps

Expert and skilled advisory team – who have ample experience in

developing reporting guidelines

Following good practice guidelines for development

Following appropriate methodology


Future Projects (2)

Beyond Stepped Wedge

“Dog-Legs & Ladder-Designs”


Beyond Stepped-Wedge:

Staggered Implementation with incomplete observations

• Most (if not all?) methodological work has focussed on complete

rectangular designs

• Some studies have used incomplete designs – for logistical reasons,

e.g. to allow for transition between experimental conditions (non-

instantaneous implementation)

• What about (statistical) efficiency?

– Are complete designs better?


“Traditional” Cluster Designs:

Treatment effect computed by comparing unweighted “average”

treated outcome with unweighted “average” control.

So all experimental subjects have the same weighting,

irrespective of their position in the design.

0.125 0.125 0.125 0.125

0.125 0.125 0.125 0.125

-0.125 -0.125 -0.125 -0.125

-0.125 -0.125 -0.125 -0.125

0.125 0.125 -0.125 -0.125

0.125 0.125 -0.125 -0.125

-0.125 -0.125 0.125 0.125

-0.125 -0.125 0.125 0.125

-0.125 -0.125 0.125 0.125

-0.125 -0.125 0.125 0.125

-0.125 -0.125 0.125 0.125

-0.125 -0.125 0.125 0.125

Parallel Cross-Over Before & After


Stepped Wedge

-0.04 0.20 0.14 0.08 0.02 -0.04

-0.02 -0.08 0.16 0.10 0.04 -0.02

0.00 -0.06 -0.12 0.12 0.06 0.00

0.02 -0.04 -0.10 -0.16 0.08 0.02

0.04 -0.02 -0.08 -0.14 -0.20 0.04

0.00 0.24 0.13 0.09 0.06 0.00

0.00 -0.06 0.13 0.09 0.06 0.00

0.00 -0.06 -0.09 0.09 0.06 0.00

0.00 -0.06 -0.09 -0.13 0.06 0.00

0.00 -0.06 -0.09 -0.13 -0.24 0.00

Vertical Analysis Standard Model

(ICC =

0.025)

• Suggests the design could be improved(?) – by transferring some

observational effort from low-weight to high-weight cells

• If so (… it is!) the standard Stepped-Wedge is not an efficient design!

• Q: What do the efficient designs look like?

“The weighting varies with the position in the design”


RESEARCH QUESTIONS:

• To identify Efficient designs under different logistical

constraints.

• E.g. (for cross-sectional designs)

1. Fix total number of subjects over all clusters

2. Fix individual cluster sizes

3. As above, but with enforced empty cells (e.g.

transition periods)

• The solution to 1. is known

• There is an algorithm to solve 2. and 3. but its

consequences remain unexplored.


1. Fix Total number of observations.

The best design looks like:

OR

(Numbers of observations in each coloured cell are not equal in general.

In the first design these depend on the precise value of the ICC)

EITHER ICC ‘large’ ICC ‘small’


E.g. An optimal ‘Ladder’ Design

(with 6 groups of clusters) looks like: C

luste

rs

Time

(Uniform sampling rate over time – No. of observations cell-width)

Treatment effect estimate is the difference between the (unweighted)

average treated outcome and the (unweighted) average control.


For 3 groups of clusters the ‘Best’ design is:

If the ICC is ‘small’, the first and last columns disappear,

and the best design is a “Dog-Leg” (cf. Hooper & Rourke 2014, 2015)


Prospects for optimal sampling

• When the total sample size is fixed, some of the optimal sampling

schemes offer realistic practical alternatives, especially if the number

of groups is small.

– Such designs may be more efficient than a complete SW with a

larger number of groups, or a simple parallel study.

– Details need further work.

• A similar approach can (probably) be applied where there are

enforced transition periods, or to handle more general cluster-level

constraints on the total sample size.

– WATCH THIS SPACE!


Further Information

Website: www.clahrc-wm.nihr.ac.uk

Twitter: @CLAHRC_WM

Sign up to our News Blog: http://eepurl.com/OMOEP

This work was funded by the National Institute of Health Research (NIHR)

Collaboration for Leadership in Applied Health Research and Care West

Midlands (CLAHRC WM). The views expressed are those of the author(s)

and not necessarily those of the NHS, NIHR, or Department of Health.

http://www.clahrc-wm.nihr.ac.uk/



https://twitter.com/clahrc_wm

http://eepurl.com/OMOEP

