Systematic Reviews: the researcher's perspective and the research question. Edoardo Aromataris

www.joannabriggs.edu.au

Systematic reviews: the researcher’s perspective and the

research question

Edoardo Aromataris

www.joannabriggs.edu.au

Questions for you• Where do you work? What settings?

– Health care? Hospital, University, government health service?

• Who’s had some experience with systematic reviews?– Was it...pleasant?

• What is a systematic review?

www.joannabriggs.edu.au

Researcher’s perspective• The review question• What do we know?• What should we know?

– PICO/PICo• Depends on who we are...• Interaction between researchers and librarians• What to expect from us

www.joannabriggs.edu.au

Steps in a Systematic Review• Formulate review question• Define inclusion and exclusion criteria• Locate studies• Select studies• Assess study quality• Extract data• Analysis/summary and synthesis of relevant studies• Present results• Interpret results/determining the applicability of results

(Egger & Davey Smith, 2001:25; Glasziou et al., 2004:2)

www.joannabriggs.edu.au

Question Development

• Aim is to provide a framework for the development and conduct of the review

• A good question supports the review, a poor question risks confounding the review

• A good question responds to identified priorities and needs

www.joannabriggs.edu.au

Question Development

• Reviews of effects & economics:– Population– Intervention/Exposure– Comparator– Outcome– Types of Study Design

• Reviews of qualitative & Textual data:– Population– Phenomena of Interest– Context

Divide question following the PICO/PICo model

www.joannabriggs.edu.au

Questions of the effects of interventions• Population:

–The most important characteristics, including:• demographic factors of the population (e.g. age,

gender, ethnicity) • socioeconomic factors• the setting (e.g. hospital, community etc)• illness

www.joannabriggs.edu.au

Questions of the effects of interventions

• Intervention and Comparator–Primary intervention of interest (treatment group)–Comparator (control group)

• Passive (placebo, no treatment, standard care, or a waiting list control)

• Active (variation of the intervention, a drug, or kind of therapy)

www.joannabriggs.edu.au

Questions of the effects of interventions• Outcomes

– Identify the primary outcome/s in order to reach a clinically relevant conclusion

– Secondary outcomes may be required– Avoid use of surrogate outcomes unless clearly

reasoned in the background– Consider how the type and timing of outcome

measurements impacts on outcome measurement

www.joannabriggs.edu.au

Example

• Are antiseptic washes more effective than non-antiseptic washes at preventing nosocomial infections in patients undergoing surgery?

intervention comparison

outcome population

www.joannabriggs.edu.au

Example

• Are antiseptic washes more effective than non-antiseptic washes at preventing nosocomial infections in patients undergoing surgery?

intervention comparison

outcome population

http://guides.library.upenn.edu/content.php?pid=192036&sid=1610308

www.joannabriggs.edu.au

Questions of the experiential evidence• Qualitative and textual reviews:

– Re-focus to phenomena of Interest, not intervention, – and Context not comparator

• The phenomena of Interest relates to a defined event, activity, experience or process

• Context is the setting or distinct characteristics

www.joannabriggs.edu.au

Example

• What are caregivers experiences of providing home-based care to persons with HIV/AIDS in Africa?

Phenomena ofinterest

population

Context

www.joannabriggs.edu.au

Example

• What are caregivers experiences of providing home-based care to persons with HIV/AIDS in Africa?

Phenomena ofinterest

population

Context

www.joannabriggs.edu.au

PICO / PICo

• Constructing a well-built clinical question is a fundamental skill

• Question should be arranged following the PICO/PICo model

• The question operationalises the review by forming the basis for inclusion and exclusion criteria and the search strategy

www.joannabriggs.edu.au

Inclusion Criteria: Effects• Draws upon:

– Population characteristics– Intervention or exposure– Comparator - active or passive– Outcomes of interest

• Study type and other elements of the review such as language, year of publication etc

www.joannabriggs.edu.au

Inclusion Criteria: Experience

• Draws upon: – Population characteristics– Phenomena of Interest– Context

• Study type and other elements of the review such as language, year of publication etc

www.joannabriggs.edu.au

Different questions lead to different evidence• Effectiveness of therapy• Adverse events and harmful outcomes of therapy• Aetiology of disease• Diagnosis and diagnostic test accuracy• Prognosis

www.joannabriggs.edu.au

Focused versus broad question• Does SSRI antidepressant increase the risk of suicide in

teenagers?– Clear guide to the data the reviewer will look for and will arrive at

clear conclusions. – Identified outcome - miss full safety profile of drug

• What are common adverse events when starting on antidepressant therapy?– Find new events across a range of drugs– Vast amounts of heterogeneous data difficult to synthesise and

draw conclusions from.

www.joannabriggs.edu.au

Broader question?• Is a systematic review appropriate?

– ‘What smoking cessation interventions (population level) does the published evidence suggest are effective and represent ‘value for money’?

– ‘What are the main risks to patient safety in primary care?’

• Consider a scoping review

www.joannabriggs.edu.au

Question Development

• Verify that the question has not already been addressed – Search protocols and systematic review reports in the

JBI and Cochrane Libraries and others• May be similar topic – shouldn’t be the same

question unless justification is provided– Critically appraise and assess quality of review

www.joannabriggs.edu.au

Protocol• Guide process

– Reasoned approach to the question asked• Detailed review methods a priori• Decrease biased post hoc changes”• Main details are question, eligibility or inclusion

criteria, approach to search and remainder of steps

www.joannabriggs.edu.au

Review Team• Funder• Content experts, researchers• Methodologists with experience in systematic

reviews• Statistician • Librarian• End users - policy makers/decision makers

www.joannabriggs.edu.au

Researchers and Systematic Reviews

• Types/breeds of researchers– Academic/expert in their field– Consultant– Clinician– Student/novice

• Not always mutually exclusive• Experience?

www.joannabriggs.edu.au

Breeds...• Academic

– Know the topic, do they know the method?• Consultant

– May know it all, may know nothing, or somewhere in between...– Still have some grounding in research and what its all about– Methodologist advantage

• Clinician– Know the need/questions– Not how to approach the research

www.joannabriggs.edu.au

PICO / PICo

• Constructing a well-built clinical question is a fundamental skill

• Divide question following the PICO/PICo model

• The question also forms the basis for the search strategy– PICO concepts

From PICO to search• Nursing management of cancer fatigue in patients• Concept map

www.joannabriggs.edu.au

Researcher's knowledge• Led from the question through PICO....• ...define eligibility criteria....• ...into an example search strategy...• ....told where they can search...

• Onto the next step

www.joannabriggs.edu.au

Knock, Knock....• ...who’s there?

– Academic/methodologist/clinician– Experience with systematic reviews?

• Not everyone understands what a systematic review is!

• Who are they and where are they coming from?• The question....what is it?

www.joannabriggs.edu.au

What’s the question...?• What concepts?

– Same words - different definitions and meanings– Different words – same definitions and meanings

• Not everyone comes armed with their own question– Contract/consultancy– Researcher may be the middle man

www.joannabriggs.edu.au

Researchers may fail to appreciate...• Search is incredibly important part of the review

process• No evidence = no review• How and where are some of the defining features of

the review process• Will understand ‘reproducibility’

www.joannabriggs.edu.au

Researcher’s question• “...major difficulty being able to conceptualise their

question...”• “...can’t describe what they’re trying to do...”

• You may be left to (help) interpret aims of the project!

www.joannabriggs.edu.au

Search strategy• “...no capacity to be logical...”• “..logic grid doesn’t reflect what she’s trying to do...”• “...its not rocket science...”• “...essential elements are language and logic...”

www.joannabriggs.edu.au

Search strategy• Many researchers understand PICO, they just fail to

follow it• Many questions wont have all of the elements of a

PICO question• Maybe different elements

– Settings, study design, aspects of population/intervention

www.joannabriggs.edu.au

Search Strategy• Strategy is more important than the search itself and

requires a great deal more time– Results driven researcher

www.joannabriggs.edu.au

Databases• Fail• No idea• “...I’m going to do a search of OVID....”• Search “PubMed and OVID Medline...”

• How much to explain?

www.joannabriggs.edu.au

...works both ways...• Can’t do the experiment unless you understand the

tools you’re working with

• Researchers and reviewers are told to refer to the librarian

• Need to understand the nature of the work and maybe even the topic

www.joannabriggs.edu.au

Summary• Question• PICO and conceptualising a question• Researchers come in all shapes and sizes• Search

– Critical for review success• Many will not understand what you do, why you do it

and the impact it will have on their systematic review!