!"#$%&&%'()%*%&+,'-(.,'/0123423&*%&3#,'23.0#5&%'657$"'6%*$%8+'-%)03"+'9#3:7-5&5;$&;'1$#%.)3#,'23.0#5&%'25&5"5,'-.-5+)%#'<&$*%#+$)=

!"#$%&'()$(*+*+,&*-".&%/&*(&01/*2

Content

• What are RRs• Focus on recent & empirical investigations arounds methods• Cochrane RR methods in context of Cochrane’s covid-19 response

Early papers on RRs

Seminal Paper - Ganaan et al. (2010)

• Scoped articles related to ‘rapid reviews’ (RRs) in the health literature

• 45 methodological articles on rapid reviews

• 25 rapid review examples

• Concluded that RRs use a variety of labels, timelines and streamlining approaches (methods variable)

• Variable nomenclature– Rapid Review– Rapid HTA– Rapid Evidence Assessment– Ultra rapid review….etc.– Tech Notes/ Tech Briefs

• Variable timeframes– 1-9 months– No time reported

• Variable streamlining methods– Restricted searching*– Restricted screening– Restricted quality appraisal– Restricted data extraction

*Limits included restricting by language; dates; # of sources searched; geographical location and setting to increase applicability

What is a Rapid Review?

• No universal definition

• RRs: akin to the general SR process but with one or more methods shortcuts and accelerated process…timely• Expediency to assemble evidence• Transparency to methods, process• Undertaken to support a decision• Guided by stakeholder/knowledge user input

• SRs and/or primary studies• Spectrum of methods exist; decisions across stages of conduct

(.37$&;'#%*$%8]'

_'(5A7B%'3D'66+'I`5&'JKFa4GL_'(:77B%A%&)%"'8$)0'"%D$&$)$3&+'D#3A'66'A%)03"+'B$)%#5):#%'IFGGabL

c')0%A%+'D#3A'"%D$&$)$3&+'$&'JFc'66+'5&"'cd'A%)03"+'757%#+

Q5A%B'%)'5B,':&7:PB$+0%"

Variable terminology

Credit to Tricco AC et al. BMC Medicine 2015;13:224.

Systematic review

Living SR Rapid Review Overview Scoping Review

Defined Q Adjust scope? (time)Protocol Yes but flexible Yes but flexibleSearch ongoing Modified?Selection Modified?Inclusion unit study study SR and/or study? SR SR and/or studyExtraction Modified? From SR,

re-assess/ analyse,

and/or consult studies

ChartingRoB/Appraisal Modified? +/-Synthesis Meta-analysis if time

permits, appropriate?Descriptive;

not synthesising results

GRADE Consider, even if narrative; cautious

interpretation of findings

From SR or potential to undertake

Not applicable

Distinguish RRs from other KS family members

A##'/"B-&./&!!+! i&%4+$j%4D$)+45BB' *%#+:+ R5$B3#%"')3')37$.

-%)03"+'5#%'"%D$&$)$*%,'%C5A7B%+]'

!1%D$&%"'J4e'"5)5P5+%+!1%D$&%"'"5)%'#5&;%'I%H;H,'75+)'O=L!1%D$&%"'80%)0%#V038')83'7%37B%':+%"'I+%B%.)$3&,'%C)#5.)$3&L!S&;B$+0'3&B=

`:"$.$3:+&%++'5..3#"$&;')3')37$.,'.3&)%C)H'SC5A7B%+]

!?%8%#'"P5+%+,'A3#%';#%= B$)'I)37$.L!!"f:+)'"P5+%+')3')37$.! Z5&;:5;%'D3#'.3&)%C)]'S&; b'?#!h%5#'#5&;%'"%7%&"+'3&'B$)'*3B:A%,'.B$&$.5B'+%&+$P$B$)=!-$&'k'":5B'+.#%%&'3D'5P+)#5.)+'P:)')5^%'3&'A3#%'$D')$A%'7%#A$)+!()5;%"]'(6+'D$#+),')0%&'"38&'0$%#5#.0=

!"f:+)'.3A75#$+3&+,'3:).3A%+ !"f:+)'.3A75#$+3&+,'3:).3A%+6%A3*%'5:)03#'.3&)5.)'@'%B5P3#5)%'5&5B=+$+ 6%A3*%'5:)03#'.3&)5.)'@'%B5P3#5)%'5&5B=+$+

C"1%&96&-$('"'B-6D&E&:("1

(6+'D$#+)]'7#$A5#='B$)%#5):#%'*$%8%")0#3:;0'5'B%&+

R0%&'"38&')0%'7#$A5#='+):"=0$%#5#.0=,'^&38$&;')05)'$++:%+'3D'P$5+'.5&'*5#='I85*='B$&%+L

Methods

Focus

• Scoping of methods• Scoping of evaluations of methods

Rapid review modifications, examplesScope � Limit (or stage, priority order) PICOS elements

e.g., interventions, comparisons, setting, outcomes

Examples, literature1,2

Search � Limit bibliographic databases

� Reduce/omit grey literature searching

� Whether use of supplemental searching

� Limit language (e.g., English)

� ‘PRESS’ (or not) the search strategy

>1, 82%; 1 only, 2%2

Grey lit used, 70%2

Scanned refs, 50%2

Limits, 68%2

Org internal PR, 38%3

Selection � One person or modify how two people are used

� Retrieve only electronically available articles

� Prioritise systematic reviews, where available

1 only, full-text, 11%2

Examples, literature4,5

Organisations, 100%3

1 Syst Rev 2015;4:50 2 BMC Med 2015:13:224 3 PLoS ONE 2016;11:e0165903 4 Impl Sci 2010;5:6 5 Evid Policy 2013;9:5

Rapid review modifications, examplesExtraction � One person or modify how two people are used

� Limit or stage extraction (breadth/depth)

1 only, 7%2

Some restriction, lit3

RoB/

appraisal

� One person or modify how two people are used 1 only, 7%2

Synthesis � Whether meta-analysis appropriate, feasible m-a seldom/not efficient 1,2

GRADE � Consider using GRADE

� Use minimum standards, not SoF/EP table format

Used GRADE, 8%4

EP not used in majority4

1 Syst Rev 2015;4:50 2 BMC Med 2015:13:224 3 Impl Sci 2010;5:6 4 Hersi et al, unpublished

<"-"%"0'0(0'"#,2'3 F)GHIJ(%5#.0%"'A3#%')05&'3&%'"5)5P5+% cJk(%5#.0%"'3&%'"5)5P5+%'3&B= Jk<+%"'7#%*$3:+'#%*$%8'5+'+)5#)$&;'73$&) FKkU6 dk

6%"#58&'D#3A'W-2'-%"'JKFO[Fe]JJN

7-".&%/&*(&01/*2&&&F(.-/%+

E#'5 &$-'#"-.#' F)GHIJ(%5#.0%"';#%='B$) aKkU3 ;#%='B$)'+%5#.0 JNkU6 dk

4,"))'3 #'/'#'),'0 F)GHIJh%+ OKkU3 FKkU6 NKk

K*)-",-'3 ".-2*#0 F)GHIJh%+ JJkU3 FFkU6 dak

<"-'(&$+$-0("::&$'3 F)GHIJU3'B$A$) FJkZ$A$)%"'P='"5)% dckU3)'#%73#)%" JKk

6%"#58&'D#3A'W-2'-%"'JKFO[Fe]JJN

8")1."1'(&$+$-0("::&$'3 F)GHIJU3'B$A$) FakZ$A$)%"'P='B5&;:5;% NGkU3)'#%73#)%" eNk

1$DD$.:B)')3'^&38'80%)0%#'B$A$)+'f:+)$D$%"'D3#'#%B%*5&.%#%5+3&+

7-".&%/&*(&01/*2&&&F(.-/%+

lA73#)5&.%'A5='"$DD%#'P=')37$.

L.&&(-'A-(0'&',-$*) F)GHIJJb'$&"%7%&"%&)'7%37B% JNkJ&" 7%#+3&'*%#$D$.5)$3& dki&% 7%#+3&'3&B= FFk\'#%*$%8%#+':&^&38& JckU3)'"3&% FkU6 JGk

M$-&'C"%0-#",- 0'&',-$*) F)GHIJJb'$&"%7%&"%&)'7%37B% eNkJ&" 7%#+3&'*%#$D$.5)$3& Oki&% 7%#+3&'3&B= Fck\'#%*$%8%#+':&^&38& JNkU3)'"3&% FkU6 Fak

nNKk':&^&38&nNKk'J'7%37B%

7-".&%/&*(&01/*2&&&F(.-/%+

nOak':&^&38&neKk'J'7%37B%

6%"#58&'D#3A'W-2'-%"'JKFO[Fe]JJN

=."&$-5C#$0N(*/(%$"0("::#"$0"& F)GHIJJb'$&"%7%&"%&)'7%37B% FakJ&" 7%#+3&'*%#$D$.5)$3& Feki&% 7%#+3&'3&B= ak\'#%*$%8%#+':&^&38& JGkU3)'"3&% akU6 Jdk

<"-" 'A-#",-$*) F)GHIJJb'$&"%7%&"%&)'7%37B% FKkJ&" 7%#+3&'*%#$D$.5)$3& Jeki&% 7%#+3&'3&B= ak\'#%*$%8%#+':&^&38& eakU3)'"3&% FkU6 JJk

nOGk':&^&38&neek'J'7%37B%

7-".&%/&*(&01/*2&&&F(.-/%+

nOOk':&^&38&neKk'J'7%37B%

6%"#58&'D#3A'W-2'-%"'JKFO[Fe]JJN

<"-" 05)-2'0$0 F)GHIJ-%)545&5B=+$+ 3#'.B%5#'#%5+3&+&3)')3'733B'#%+:B)+

JJk

U5##5)$*%V"%+.#$7)$*% ack

<&^&38&'80%)0%#'+3A%'3D'&5##5)$*%V"%+.#$7)$*%'8%#%P%.5:+%'3D'.3&.%#&'3D'73)%&)$5BB='A$+B%5"$&;'733B%"'"5)5

7-".&%/&*(&01/*2&&&F(.-/%+

6%"#58&'D#3A'W-2'-%"'JKFO[Fe]JJN

E9D<@ F)GHOJ<+%" 96!1S Jk-3"$D$%" 96!1S dkU3 GJk

iD'ck,'%*$"%&.%'7#3D$B%'&3)':+%"'$&')0%'A5f3#$)=

7-".&%/&*(&01/*2&&&F(.-/%+

Q%#+$ %)'5B,':&7:PB$+0%"

657$"'#%*$%8+'JKFN,'JKFd

Scoping of evaluations of methods

• English literature, 1997 onward• Methods papers evaluating approaches applied or related to RR

conduct

(.37$&;'#%*$%8]'

?%8'+):"$%+'05*%D3#A5BB='%*5B:5)%"'66'M+03#)4.:)E'A%)03"+

Q5A%B'%)'5B,':&7:PB$+0%"

Q5A%B'%)'5B,':&7:PB$+0%"

Marshall J Clin Epi 2019;109:30-41

>80% of results with 5% ormore change

Q5A%B'%)'5B,':&7:PB$+0%"

Marshall J Clin Epi 2019;109:30-41

19% of results with 5% ormore change

Nussbaumer-Streit J Clin Epi 2018;102:1-11

Fig 2: for conclusions in opposite direction…combination of two databases was always noninferior or one databaseplus reference lists except MEDLINE.

Fig 3: combination of MEDLINE, Ebmase, CENTRAL with reference list searches (omitting specialized database and grey lit)led to lowest proportion of conclusions with any changes. Searching reference lists almost always improved searches

Non-inferiority: study• International survey of English,

German, Spanish decision-makers and guideline developers, provided 3 hypothetical scenarios (clinical treatment, public health intervention, clinical prevention), n=350

• Willing to accept a 10% risk of getting an incorrect answer as a trade-off for using rapid reviews

Wagner BMC Med Res Methodol2017;17:121

15 of 30 meta-analyses affected; One review missed 15/36 studies

Affected 5 of 23 meta-analyses; wider CI (direction not affected)

Omitted 4/73 studies

Pham Res Syn Meth 2016;7:433-446

Q5A%B'%)'5B,':&7:PB$+0%"

Gartlehner Syst Rev 2019;8:277

AI can be used for priority ordersorting

Gartlehner J Clin Epi 2020;121:20-28

~13% missed studieswith single-person screening of abstracts

Single screening maybe a viable option forRRs

Shortcut dependent on reviewer:Impacted 4 or 12 meta-analysesOmitted 4 or 21 studies

Large variation between reviewers

Pham Res Syn Meth 2016;7:433-446

• Findings supported by this methodological review: single screening misses substantially more studies but could be considered for RRs• Full text screening not evaluated

Structured P-I-C for 10 reviewsSynonyms list developed5 reviewersScreen titles only

• 100% recall for 9/10 reviews• Median 53% reduction in screening effort• how does it fare withmore complex topics?

Rathbone Syst Rev 2017;6:233

Q5A%B'%)'5B,':&7:PB$+0%"

OG'23.0#5&%'#%*$%8+p

Fd'#%*$%8+'8$)0'.05&;%"'+):"='$&.B:+$3&

pJ'#%*$%8+'8$)0'.05&;%"'+)5)$+)$.5B'+$;&$D$.5&.%]

o''23&.B:+$3&+'&3)'"$DD%#%&)'P:)'B%++'.%#)5$&

4GB5=%$1;&1/1I41;5$+-&+.=%$(+,&:$1$:"5&$:#"B.

U:++P5:A%#4()#%$) %)'5BH'`'2B$&'S7$'JKFG'I%47:PL

Q5A%B'%)'5B,':&7:PB$+0%"

Martyn-St James Evid & Policy 2017;13:517-38

Case study: pharmacological topic

RR: extracted data from existing SRs where available

• same RCTs in both (n=41)• data extracted from SRs in 25 RCTs

• findings and conclusions did not differ b/w RR and SR

• Data extraction by two independent reviewers seems to result in less extraction errors than data extraction by one reviewer and verification by a second.• Sparse data, more studies

needed

What have RRs addressed?

-5&='66+'5""#%++'3&%'Y:%+)$3&

1$DD%#%&)'q+'5#%'73++$PB%,'P:)'A5$&B='%DD%.)$*%&%++

J

eN

U6

UoGF'IJKFN,'JKFdL

1R!

q:5B$)5)$*%

Reporting is poor as perPRISMA…

Stevens et al, unpublished

What are key considerations?What else about the process should I know?

?(6&B/1+$%('".$/1+

! >05)'8$BB'A%%)':+%#+E'&%%"+g

! >05)'.5&'P%'5..3A7B$+0%"'D3#')0%'5*5$B5PB%')$A%g

! W%$&;')03:;0)D:B'3&'577#35.0,'A$&$A$j$&;'P$5+%+

! -5&5;$&;'%C7%.)5)$3&+'8$)0'%&"':+%#

! 1%.$+$3&')3':&"%#)5^%'A5='P%')37$.4)34)37$.

K(6/1%&.-(&:(.-/%+,&#'/B(++&B/1+$%('".$/1+

! (%))$&;')0%'Y:%+)$3&! ?%5+$P$B$)=,'%C7%.)5)$3&+

l&)5^%'7#3.%++,'$)%#5)$*%2:#+3#='B$)%#5):#%'+.37$&;

-5&='05&"+'A5^%'B$;0)'83#^'*+(A5BB%#')%5A+'IA5='83#^'P%))%#L'''''(A5BB%#')%5A+'IA5='83#^'P%))%#L'''''

L$:(5$1(&M&$1.('"B.$1;&*$.-&B/::$++$/1('+,&(G":#5(

! S+)5PB$+0%"'JKFO! 23&*%&3#+]'!:+)#$5,'<(,'25&5"5

E1.('$:&;=$%"1B(

! (:#*%='3D'#%7#%+%&)5)$*%+'D#3A'JK'23.0#5&%';#3:7+

! SC$+)$&;'%A7$#$.5B'%*$"%&.%'7#3*$"%"'3&')0%'+:#*%=

! 65)%'3#'#5&^'66'A%)03"+'5.#3++'+)5;%'3D'.3&":.)

! 6%.3AA%&"5)$3&+'P5+%"'3&'$)%A+'D3#'80$.0')0%#%'85+'5'0$;0'IraKk'#%+73&"%&)+L'3#'A3"%#5)%'IOK4dGkL'%&"3#+%A%&)'3#'0$;0%+)'#5&^$&;'$&'5P+%&.%'3D'%&"3#+%A%&)

Cochrane Rapid Review

“Rapid review is a form of knowledge synthesis that accelerates the process of conducting a traditional systematic review through streamlining or omitting specific methods to produce evidence for stakeholders in a resource-efficient manner.”

Cochrane RRs should be driven primarily by requests for timely evidence for decision-making…urgent/emergent or high priority

Cochrane Rapid Review: 1 week to 6 months(interim methods)

Defined Q Limit interventions/comparatorsLimit number of outcomes

Protocol Yes but flexible: allow post hoc changes as part of efficient, iterative processSearch MEDLINE, CENTRAL + EMBASE if access

Specialized databases limited & if feasibleEnglish language (other languages only if justified)Limit grey literature and supplemental searching

Selection(+ calibration pilot)

ABSTRACT: Dual screen for ≥20%; after, 2nd person to verify excludes & conflictsFULL TEXT: 2nd person screen all excluded articles

Inclusion unit Emphasis on SRs first, where available; then, stepwise through hierarchyExtraction Use data from existing SRs; limit extraction items

2nd person checks correctness, completenessRoB/Appraisal 2nd person risk of bias checkSynthesis Meta-analysis if time permits, appropriate

GRADE 2nd person check judgements, footnotes, final rating

• Involve key stakeholders in setting protocol; consult throughout

#57$"#%*$%8+v.3.0#5&%H5)

23.0#5&%'657$"'6%*$%8+'-%)03"+'9#3:7

v23.0#5&%66-9

!))#$P:)$3&'D3#'+3A%'.3&)%&)'7#3*$"%"'8$)0$&]9%#5B"'95#)B%0&%#,'W5#P5#5'U:++P5:A%#4()#%$),'205&)%BB%'95##$)=,'25&"=.% Q5A%B,'!&"#%5'R#$..3 I5""'.$)5)$3&L

+)%*%5JNvA.A5+)%#H.5