Designing a proficiency-based, content validated virtual reality curriculum for laparoscopic colorectal surgery: A Delphi approach

  • View

  • Download

Embed Size (px)


  • Designing a proficienvalidated virtual reallaparoscopic colorec


    lity (descased

    e explorecn a

    which they thought that a particular task should be included in a final technical skills curriculum.

    s a was 0.715; after the second round, consensus was

    SURGERally useto trainrect sucreaseregardtime oogy sufeasibla portiroom

    thatment,ealityent in, theulatorperat-tagesabilitytheiraram-idualsey areculty

    rtingtrain-f theed on



    Reprint600 Uniada. E-m0039-6060/$ - see front matterloped 2012doi:10.1reached at 0.865. Consensus was reached for 7 basic tasks and 1 advanced suturing task. Medianexpert time and economy of movement scores were defined as benchmarks for all curricular tasks.Conclusion. This study used Delphi consensus methodology to create a curriculum for an advancedlaparoscopic procedure that is reflective of current clinical practice on an international level andconforms to current educational standards of proficiency-based training. (Surgery 2012;151:391-7.)

    From the University of Toronto,a Toronto, Ontario, Canada; the Department of Surgery,b Academic MedicalCenter Amsterdam, Amsterdam, The Netherlands; and the Department of Surgery,c St. Michaels Hospital,Toronto, Ontario, Canada

    Y RESIDENCY TRAINING PROGRAMS have tradition-d the operating room to teach surgical skillsees through graded responsibility under di-pervision. Because of the mandated de-in resident work hours, ethical concernsing trainees learning procedures for the firstn patients, and the advent of new technol-ch as laparoscopy, this strategy is no longere. As such, it has become necessary to shifton of residency training from the operatingto the surgical skills laboratory. Several

    recent systematic reviews have demonstratedtechnical skills training in an ex vivo environwhether on a bench-top model, or a virtual r(VR) simulator, translates into an improvemoperating room performance.1-4 Moreovertechnical improvements seen on a VR simhave shown to persist for $10 cases in the oing room.5 VR simulators have several advanover bench-top simulators, including theirto simulate complications, such as bleeding,ability to automatically generate assessment peters allowing for comparison between indivand performances, as well as the fact that thable to simulate tasks at varying levels of diffiallowing for a natural gradation of training.6

    With the strong body of evidence suppothe role of VR simulation in technical skillsing, it is somewhat surprising that outside orealm of research studies, few curricula bastraining using VR simulation have been deve

    by the Royal College Fellowship in Education.

    d for publication August 4, 2011.

    requests: Vanessa N. Palter, MD, University of Toronto,versity Ave, Rm. 440, Toronto, Ontario M5G 1X5 Can-ail:

    Mosby, Inc. All rights reserved.Results. In the first round of the survey, CronbachResults of the survey were sent back to participants until consensus (Cronbachs a >0.8) was reached. Across-sectional design was utilized to define the benchmark scores for the identified tasks. Nine expertsurgeons completed all identified tasks on the easy, medium, and hard settings of the simulator.Delphi approachVanessa N. Palter, MD,a Maurits Graafland, MD,b

    and Teodor P. Grantcharov, MD, PhD,c Toronto, O

    Background. Although task training on virtual reaoperating room, to date no VR curricula have beenpurpose of this study was to develop a proficiency-bcolorectal surgery.Methods. The Delphi method was used to determinsimulator) are relevant to teaching laparoscopic cointernational experts rated all the LapSim tasks o016/j.surg.2011.08.005cy-based, contentity curriculum fortal surgery: A

    lies P. Schijven, MD, PhD, MHSc,b

    o, Canada, and Amsterdam, the Netherlands

    VR) simulators has been shown to transfer to theribed for advanced laparoscopic procedures. TheVR technical skills curriculum for laparoscopic

    ert consensus on which VR tasks (on the LapSimtal surgery. To accomplish this task, 19Likert scale (15) with respect to the degree tofor minimally invasive procedures. Several groups

    SURGERY 391

  • Surgery392 Palter et alhave described curricula for basic laparoscopy,including curricula for basic minimally invasivetasks, or for less complex procedures such aslaparoscopic cholecystectomy.7-9 Although thesecurricula represent an important step in definingVR curricula for minimally invasive procedures,and largely conform to current educational theo-ries regarding proficiency-based learning and dis-tributed practice, as a group, they have beendeveloped largely using local expertise. Specifi-cally, experts at 1 institution determine which tasksor components are included in the final curricu-lum. To ensure the applicability of the developedcurricula, it is essential that the final educationalproduct be reflective of practice across diverse in-stitutions. In addition, to our knowledge, no tech-nical skills curricula using VR simulation havebeen described for advanced minimally invasiveprocedures.

    Laparoscopic colorectal surgery is consideredan advanced, minimally invasive procedure. Per-forming this procedure successfully involves ligat-ing large blood vessels, working in multiplequadrants of the abdomen, and creating a viableanastomosis.10 A long, variable learning curve11

    has been described for laparoscopic colorectal sur-gery, which underscores the necessity of develop-ing a technical skills curriculum for learning thisprocedure, ideally in a simulated environment.The purpose of this study is 2-fold. Our first aimwas to use consensus methodology to develop atechnical skills curriculum based on VR for laparo-scopic colorectal surgery. Our second aim was todefine expert benchmarks of proficiency for thiscurriculum. We hypothesized that the Delphimethod of consensus would be a feasible meansof developing a technical skills curriculum forthis advanced minimally invasive procedure thatis reflective of international practice.


    Study design. This study used Delphi methodol-ogy to obtain consensus on the essential compo-nents of a VR curriculum for laparoscopiccolorectal surgery. In addition, a cross-sectionaldesign was utilized to determine expert levels ofproficiency for the defined curriculum. The studywas approved by the local institutional reviewboard.

    Participants for Delphi consensus. Participantsfor the Delphi consensus portion of the study wererequired to be leaders in their clinical field asevidenced by their role as opinion leaders within

    organizations such as The American Society ofColon and Rectal Surgeons, the Society ofGastrointestinal and Endoscopic Surgeons, orother national surgery societies. Furthermore,they were required to be familiar with the VRsystem used for the curriculum. Finally, the expertswere required to be practicing surgeons who wereinvolved in training laparoscopic colon and rectalsurgery both at the resident and continuing pro-fessional development level. Twenty experts wererecruited by e-mail to respond to an on-line survey.The experts were intentionally selected to repre-sent a wide geographic area. In North America, 11experts were contacted, and in Europe, 9 expertswere contacted with the assistance of the DutchSociety for Endoscopic Surgery. Membership ofthe expert panel was not revealed to the surveyparticipants.

    On-line survey. The VR system that was utilizedfor the technical skills training portion of thecurriculum was the LapSim laparoscopy trainer(Surgical Science, Gothenburg, Sweden). Con-struct validity, learning curves, and transfer of skillslearned on the LapSim have been demon-strated.5,12-17 The system consists of 11 basic tasks,10 advanced tasks, and 6 procedural tasks specificto general surgery. Not all tasks on the LapSim,however, are relevant to laparoscopic colorectalsurgery. The role of the Delphi panel was to deter-mine through expert consensus those tasks thatare relevant to teaching the technical skills re-quired to perform laparoscopic colorectal surgery.These tasks were compiled into an on-line surveyusing via Survey Monkey (Palo Alto, CA). The par-ticipants in the expert panel were required to rateeach identified task on a Likert scale from 1 to 5detailing the degree to which they agreed ordisagreed that a particular component should beincluded in a final technical skills curriculum. Re-sults of the survey were sent back to participantswith group averages and standard deviations untilexpert consensus was reached. Expert consensuswas pre-defined as Cronbachs a >0.8, which hasbeen shown to be an acceptable method of consen-sus determination.18

    Final structure of the technical skills curricu-lum. After Cronbachs a >0.8 was achieved for theon-line survey, an outline of the final technicalskills portion of the curriculum was created. Spe-cific curricular tasks that over 80% of the expertsrated as either 4 (agree) or 5 (strongly agree) onthe final scale were included in the final technicalskills curriculum. The technical portion of the fi-nal curriculum will require execution of the iden-tified tasks on the easy, medium, and hard

    March 2012levels of the simulator. The settings for each levelwere taken from a recent European study in which

  • Surgery Palter et al 393consensus was reached on defining levels for theLapSim Basic Skills 3.0 package.19 Levels for thetasks that were not discussed in the European con-sensus document (handling intestines, and stitchand square knot) were defined using an identicalconcept with local expertise.

    Participants for expert benchmark levels. Nineexperts in minimally invasive surgery were identi-fied. An expert was defined as an individual whohas completed >100 advanced minimally invasiveprocedures.

    Tasks. Each expert was familiarized to the sim-