Postgraduate anaesthesiology education:protocol for a scoping review

Charlotte Ringsted,1 Sheena Bance,2,3 Laura Leigh Murgaski,4 Jodi Herold,4

Susan Glover Takahashi4

To cite: Ringsted C, Bance S,Murgaski LL, et al.Postgraduate anaesthesiologyeducation: protocol for ascoping review. BMJ Open2014;4:e004667.doi:10.1136/bmjopen-2013-004667

▸ Prepublication history forthis paper is available online.To view these files pleasevisit the journal online(http://dx.doi.org/10.1136/bmjopen-2013-004667).

Received 11 December 2013Revised 22 April 2014Accepted 25 April 2014

1Department of Anesthesia,The Wilson Centre, Universityof Toronto and UniversityHealth Network, Toronto,Ontario, Canada2Department of PostgraduateMedical Education, Faculty ofMedicine, University ofToronto, Toronto, Ontario,Canada3Ontario Institute for Studiesin Education (OISE),University of Toronto,Toronto, Ontario, Canada4Post Graduate MedicalEducation, Faculty ofMedicine, University ofToronto, Toronto, Canada

Correspondence toDr Charlotte Ringsted;charlotte.ringsted@utoronto.ca

ABSTRACTIntroduction: The link between education and clinicalpractice is vital, yet the current state of researchsuggests there is a substantial gap between medicaleducation research and practice. This too is the case inthe domain of anaesthesiology education research, asmuch of the research focuses on simulation studies,and a narrow range of research methods. The aim ofthis study is to comprehensively review the existingliterature in postgraduate anaesthesiology educationresearch in order to identify key research priorities. Thefindings from this review will be used to establish abase for developing a strategic research programme inanaesthesia education and practice.Methods and analysis: We will employ the scopingreview methodology outlined by Arksey and O’Malley(2005) to comprehensively search the literaturepertaining to postgraduate anaesthesiology education.We will search relevant electronic databases(eg, MEDLINE, EMBASE) and grey literature. Afterconducting calibration exercises, two authors willindependently apply inclusion criteria to all titles andabstracts and perform full-text review of all eligiblearticles. Data to be extracted will include basicinformation about the study (eg, location, design) aswell as detailed information regarding the context ofthe research and type of education examined. Ourresults will be used to develop a framework of themesthat outline the research currently being conducted andidentify gaps in research.Ethics and dissemination: This review is the firststep in a strategic research plan in postgraduateanaesthesiology education. We plan to disseminate thisresearch through publications, presentations andmeetings with relevant stakeholders. Ethical approvalwas not sought for this scoping review.

INTRODUCTIONAnaesthesiology is one of the largest medicalspecialties, and research on anaesthesiologyeducation and anaesthesiology practice con-tinues to grow and diversify. Although thelink between education and practice is crucialfor patient safety and quality of care, it hasbeen noted that a gap between education andclinical practice exists in the field of anaesthe-siology.1 A paucity of literature has

comprehensively surveyed what is currentlyknown, and what is currently unknown fromanaesthesiology education research. Philibert1

suggests that important topics in anaesthesi-ology that should be studied in more detailinclude issues surrounding transitions withinanaesthesiology training, duty hour limits, andteaching and evaluating competencies (eg, theCanMEDS framework), and importantly, sug-gests that although important to knowledgeand innovation, commentaries and editorialsare often employed as an evidence base,ahead of alternative methodologies or articletypes. This concern is echoed by otherresearchers, who have noted a number ofpotential concerns with the current research,such as methodological issues (eg, smallsample sizes, brief follow-up, threats to valid-ity),1 lack of conceptual frameworks,2 and apreponderance of descriptive or justificationstudies (ie, those that focus on only the begin-ning or end of the scientific method, ratherthan employing all of the steps from makingpredictions through to testing them).3

A general overview of the existing literaturesuggests that much of the current research inanaesthesiology education is focused on tech-nical skills, where simulation studies domin-ate the literature.4–6 However, an expertpanel recently identified a number of areaswithin simulation-based training that need tobe addressed in the future.7 Moreover, itappears that less is currently known aboutthe clinical training environment, includingimportant questions of clinical instructionand feedback, the structure of clinical

Strengths and limitations of this study

▪ The strength of this study is that it focuses onall aspects of clinical training.

▪ The limitation is that the study is confined to onespecialty.

▪ A second limitation is that effect of educational inter-ventions cannot be estimated - rather issues ofstudies on educational research will be compiled.

Ringsted C, Bance S, Murgaski LL, et al. BMJ Open 2014;4:e004667. doi:10.1136/bmjopen-2013-004667 1

Open Access Protocol

on October 25, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2013-004667 on 19 M

ay 2014. Dow

nloaded from

training and developmental aspects of competence andtraining.8 A thorough understanding of the current evi-dence, as well as the consideration of what still needs tobe examined, is essential to patient safety and quality ofhealthcare. However, at present, no study has systematic-ally reviewed the literature in anaesthesiology educationresearch to identify current practices, and the conse-quent gaps in current knowledge. To address this gap,we propose a comprehensive review of the literature per-taining to postgraduate anaesthesiology education. Thisreview is the first step in a strategic research plan in post-graduate anaesthesiology education, and will seek tocollect information on the methodologies and types ofstudies conducted, the conceptual or theoretical frame-works most often discussed (if any), the education com-ponents described, context of studies and articles anddesign. The overall purpose of this review will be toidentify key research priorities in postgraduate anaesthe-siology education to advance effective practice.

METHODSA scoping review methodology will be used to conductthis study. Unlike systematic reviews, scoping studies/reviews aim for breadth and comprehensiveness, ratherthan depth.9 Arksey and O’Malley’s9 article outlining theframework for a scoping study is widely used in the plan-ning and execution of scoping studies. The authorspoint to several reasons for conducting a scoping review,one being to identify research gaps in the existing litera-ture. In particular, scoping studies are of use in areas inwhich little research has been conducted, as is the casein anaesthesiology education and research.A protocol for our review was prepared and circulated

to all team members, and will be revised as needed. Weemployed the five steps as outlined in Arksey andO’Malley’s article (and further delineated byLevacet al10) to organise our review. These steps include(1) identifying the research question, (2) identifying rele-vant studies (developing the search strategy), (3) studyselection, (4) charting the data (data extraction), and (5)collating, summarising and reporting the results.

Data collectionSearch strategyThe basic structure of our search will include the terms‘postgraduate, ‘anaesthesia’/‘anaesthesiology’ and ‘edu-cation’. With the assistance of an information specialist,we will develop search strategies for the following data-bases: MEDLINE, EMBASE, ERIC and the CochraneDatabase. We will also search reference lists of identifiedarticles, hand search key journals (eg, Canadian Journalof Anesthesiology) and search reports (grey literature)from relevant organisations published from inception toDecember 2013, in order to identify other relevant infor-mation not captured by our search. We will not limit thesearch by study design or year, but due to a lack ofresources for translation, will limit articles to those in

English. All references will be imported in RefWorks, anonline bibliographic management programme, withduplicates removed.

Study selectionThe inclusion criteria will include studies that mentionanaesthesiology in some capacity, have a focus on thepostgraduate level/residents and discuss education.More specifically we will include articles that mentionanaesthesiology residents alongside other medical resi-dents, in order to examine anaesthesiology residents’outcomes separately from other medical specialties. Forspecificity, however, we will only include articles thatclearly mention postgraduate or a synonym for this levelof training (ie, residents, house staff, intern etc). Finally,we conceptualise education as falling under the follow-ing categories: training, learning, teaching and assess-ment. We will use these terms to determine eligibility.Prior to running the full database search, we will

conduct a pilot calibration exercise with a random sampleof 50 articles chosen from a MEDLINE search. Fourauthors (SGT, JH, LLM and SB) will independently screenthe 50 articles for inclusion, using an excel sheet withheadings for each inclusion criterion. We will compare theresults between raters and determine the κ coefficient. Ifthe κ is less than 90%, we will repeat the calibration exer-cise as needed, and revise our inclusion criteria untilreaching a κ of 90%. We will then run the full databasesearch and the full set of references will be independentlyscreened for inclusion by title and reference by twoauthors (SB and LLM). Any discrepancies will be resolvedthrough discussion with a third reviewer (SGT).

Data extractionThe full texts of all included articles will be retrieved.Two authors (SB and LLM) will complete the data extrac-tion process independently. The data will be extracted viaan online survey data collection tool, where data to beextracted will be entered as a question/answer in asurvey. We chose this method in order to organise ourdata in a way that could be easily collated after beingexported into Microsoft Excel. As suggested by Arkseyand O’Malley9 we will extract general information abouteach study/article, such as the author, year of publica-tion, location, purpose, main findings or conclusions,and recommendations, suggestions, or implications asstated by the article. In addition, we will extract informa-tion specific to addressing the gaps in anaesthesiologyeducation and research. This includes information aboutthe topic of the article (eg, biomedical/technicalstudies), the conceptual or theoretical frameworkemployed (eg, adult learning theory), the type of article(eg, review, commentary, primary research), the studydesign (if primary research study selected), the contextof the study (eg, classroom; clinical) and the educationalcomponent (eg, assessment, teaching, training, includingmode of delivery of teaching and training and assessmentformat; figure 1). Information on study design will be

2 Ringsted C, Bance S, Murgaski LL, et al. BMJ Open 2014;4:e004667. doi:10.1136/bmjopen-2013-004667

Open Access

on October 25, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2013-004667 on 19 M

ay 2014. Dow

nloaded from

extracted using the ‘research compass’ framework as out-lined by Ringsted et al11 which conceptualises medicaleducation designs as experimental (eg, randomisedcontrol trial), explorative (eg, descriptive study), observa-tional (eg, cohort study) and translational (eg, review orknowledge translation). Since we are including all studydesigns, including exploratory research and reviewswhich do not necessarily strive to provide evidence, wewill not assess the quality of evidence (as recommendedby Arksey and O’Malley9).As with the title and abstract screening phase, we will

first complete a calibration exercise for data extraction.Three authors (CR, LLM and SB) will, in duplicate,complete data extraction on a random sample of 10 arti-cles that meet for inclusion in the title and abstractreview. Using an online survey data collection tool, theresults will be exported into Excel and compared usingthe κ statistic. If the agreement level is sufficient, twoauthors will independently extract data from the full setof articles. If needed, a second calibration exercise willbe conducted. The calibration exercises will be used forboth refining the information to be extracted as well asimproving the questions used in the online survey datacollection tool. A third reviewer (CR) will adjudicate thecalibration process in the case of discrepancies betweenreviewers.

Synthesis and presentation of resultsWe will use a mixed-method approach to summarise thedata. We will provide a numerical summary of the included

studies, including the overall number of studies, countrieswhere studies/articles were conducted/written and yearsof publication. Next, we will summarise, using an iterative,content analysis approach, the gaps in postgraduate anaes-thesiology education, following a grounded theorymethodology. Specifically, the information extracted willbe used to develop a framework of themes that outlinesthe research currently being conducted, and the potentialgaps in research.

DISCUSSION AND DISSEMINATIONTo the best of our knowledge, this study will be the first toreview the literature on postgraduate anaesthesiology edu-cation using comprehensive scoping review methodology.This study is the first step in establishing an empirical basefor developing a strategic research programme in anaes-thesiology education and practice. The findings from thereview will be submitted for publication in relevantmedical education and anaesthesiology journals, whichwill provide information to researchers regarding topicsand themes in postgraduate anaesthesiology educationthat are under-researched. The findings will also be disse-minated at medical education conferences such as TheInternational Conference on Residency Education (ICRE)and the Association for Medical Education in Europe(AMEE) Annual Conference. We will also share our resultswith key stakeholders in education, practice andcollaborators.

Figure 1 Data extraction plan flowsheet.

Ringsted C, Bance S, Murgaski LL, et al. BMJ Open 2014;4:e004667. doi:10.1136/bmjopen-2013-004667 3

Open Access

on October 25, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2013-004667 on 19 M

ay 2014. Dow

nloaded from

Acknowledgements The authors thank the PGME office at the University ofToronto for operational funding support and Laure Perrier for guidance andassistance with the search strategy.

Contributors CR conceived the study; all the authors worked collaborativelyto draft and edit; all the authors read and approved the final manuscript.

Funding CR received support from a University of Toronto, Department ofAnesthesia Merit Award for undertaking this project.

Competing interests None.

Provenance and peer review Not commissioned; externally peer reviewed.

Open Access This is an Open Access article distributed in accordance withthe Creative Commons Attribution Non Commercial (CC BY-NC 3.0) license,which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, providedthe original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/

REFERENCES1. Philibert I. Review article: closing the research gap at the interface of

learning and clinical practice. Can J Anesth 2012;59:203–12.

2. Boet S, Sharma S, Goldman J, et al. Review article: medicaleducation research: an overview of methods. Can J Anesth2012;59:159–70.

3. Cooke DA, Bordage G, Schmidt HG. Classifying the purpose ofresearch. Med Educ 2008;42:128–33.

4. Bould MD, Naik VN, Hamstra SJ. Review article: new directions inmedical education related to anesthesiology and perioperativemedicine. Can J Anesth 2012;59:136–50.

5. Leblanc VR. Simulation in anesthesia: state of the science andlooking forward. Can J Anesth 2012;59:193–202.

6. Boulet JR, Murray DJ. Simulation-based assessment inanesthesiology: requirements for practical implementation.Anesthesiology 2010;112:1041–52.

7. Issenberg SB, Ringsted C, Østergaard D, et al. Setting a researchagenda for simulation-based healthcare education. A synthesis ofthe outcome from an Utstein style meeting. Sim Healthcare2011;6:155–67.

8. Ringsted C. Developmental aspects of medical competency andtraining: issues of curriculum design. Med Educ 2010;45:12–16.

9. Arksey H, O’Malley L. Scoping studies: towards a methodologicalframework. Int J Soc Res Methodol 2005;8:19–32.

10. Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing themethodology. Implement Sci 2010;5:69.

11. Ringsted C, Hodges B, Scherpbier R. ‘The research compass’: anintroduction to research in medical education: AMEE Guide No. 56.Med Teach 2011;33:695–709.

4 Ringsted C, Bance S, Murgaski LL, et al. BMJ Open 2014;4:e004667. doi:10.1136/bmjopen-2013-004667

Open Access

on October 25, 2020 by guest. P

rotected by copyright.http://bm

jopen.bmj.com

/B

MJ O

pen: first published as 10.1136/bmjopen-2013-004667 on 19 M

ay 2014. Dow

nloaded from