ORIGINAL REPORTS

A Proficiency-Based Skills TrainingCurriculum for the SAGES Surgical TrainingFor Endoscopic Proficiency (STEP) Program

Victor Wilcox Jr, MD,*,† Ted Trus, MD,‡ Nilson Salas, MD,*,† Jose Martinez, MD,§

and Brian J. Dunkin, MD*,†

*Methodist Institute for Technology, Innovation & Education (MITIE); †Department of Surgery, The MethodistHospital, Houston, Texas; ‡Department of Surgery, Dartmouth-Hitchcock Medical Center, Lebanon, NewHampshire; and §Department of Surgery, University of Miami Health System, Miami, Florida

INTRODUCTION: The surgical training for endoscopicproficiency program is a collaborative project between Societyof American Gastrointestinal and Endoscopic Surgeons andOlympus America Inc. dedicated to providing flexible endoscopytraining to surgery residency programs. Currently it lacks modelsfor proficiency-based training. This study developed 2 novelflexible endoscopy simulators, purchased a third, and establishedface and content validity as well as proficiency metrics for all 3.

METHODS: Three simulators were tested—a foam andcardboard upper gastrointestinal tract model, a commer-cially available colonoscopy model (CM-15, Olympus,Japan), and an endoscopic targeting model created fromthe Operation Game (Hasbro). Time and errors for theperformance of 12 expert surgical endoscopists on eachmodel were used to calculate proficiency scores. Facevalidity and content validity were established throughposttest questionnaires using a 5-point Likert scale.

RESULTS: Experts had a mean of 8 years of endoscopicpractice (range: 1-24 y). Among them, 83% teach residentsor fellows using simulation. Most perform more than 50upper endoscopies (51 to4500) and 100 colonoscopies (101to 4500) per year. The average time for completing theupper gastrointestinal tract model with correct identificationof all targets was 133 � 56 seconds. Complete navigation ofthe colonoscopy model averaged 325 � 156 seconds. Properorientation and targeting using the Operation Game modelaveraged 273 � 109 seconds with 3 errors.

CONCLUSIONS: This study proves face and contentvalidity for 3 physical flexible endoscopy simulators that

Disclosures: Ted Trus, Jose Martinez, and Brian Dunkin have honoraria forspeaking and consulting from Olympus.

Correspondence: Inquiries to Victor Thomas Wilcox, MD, Department of Surgery,The Methodist Hospital, 6550 Fannin Street, Suite 1661, Houston, TX 77030;E-mail: VWilcox@houstonmethodist.org, VWilcox2000@hotmail.com

282 Journal of Surgical Education � & 2014 Association of ProElsevier Inc. All rights reserv

can be used to train upper and lower endoscopy as well asinstrument targeting. It also establishes expert proficiencymetrics that can be used by trainees for structured rehearsal.These relatively inexpensive models will be incorporatedinto the surgical training for endoscopic proficiency curric-ulum. ( J Surg 71:282-288.JC 2014 Association of ProgramDirectors in Surgery. Published by Elsevier Inc. All rightsreserved.)

KEY WORDS: Surgical o Technical, Endoscopy, Profi-ciency, Training o Endoscopy, SAGES, STEP

COMPETENCIES: Patient Care, Medical Knowledge, Prac-tice-Based Learning and Improvement

INTRODUCTION

Surgical training in flexible gastrointestinal (GI) tract endos-copy has come under scrutiny recently, with the medicalcommunity questioning if the experience required duringresidency training (35 esophagogastroduodenoscopies and 50colonoscopies) is adequate to perform these procedures inpractice.1,2 This has caused the surgical community toreassess its commitment to keeping flexible endoscopy in itsarmamentarium for the care of patients with GI diseases.3,4

As endoscopy is a field developed and advanced by surgeonsand a procedure done more commonly in rural surgicalpractice than any other, the surgical community maintainsthat it is critical for surgical training to continue to preparetrainees to perform these examinations in practice.In 2009, as part of an ongoing effort to promote hands-

on skills training in flexible GI endoscopy, the Society ofAmerican Gastrointestinal and Endoscopic Surgeons(SAGES) partnered with Olympus America, Inc. to developa program to provide endoscopy equipment and a trainingcurriculum for all surgical residency training programs inthe United States. The program—surgical training for

gram Directors in Surgery. Published byed.

1931-7204/$30.00http://dx.doi.org/10.1016/j.jsurg.2013.10.004

FIGURE 1. Trus upper endoscopy model.

endoscopic proficiency (STEP)—provides suggested train-ing models and didactic resources for teaching surgicalresidents about flexible GI endoscopy as well as an endo-scopic tower and an upper and lower endoscope for skillsrehearsal. Participating programs are required to have a localchampion attend a 1-day training course conducted bySAGES that reviews the STEP curriculum, provides hands-on experience with multiple endoscopic models, and givesguidance as to how to use the flexible endoscopy tower in adry or wet laboratory. The programs are then required tosubmit regular reports back to SAGES on how the equip-ment is being used and barriers encountered to training.One element missing from the STEP program is a train-

to-proficiency skills curriculum. STEP members are given anice summary of models that can be used with the endos-copy tower, but no validated metrics that would allow for asurgical resident to rehearse their skills until they reach acertain expert proficiency. This study developed 2 novelflexible endoscopy simulators, purchased a third, and thenestablished face and content validity as well as proficiencymetrics for training using the STEP endoscopic equipment.

FIGURE 2. CM-15 colonoscope training model - Olympus Japan

MATERIALS

Three flexible endoscopy simulators were tested. The first wasan upper gastrointestinal (UGI) tract model constructed fromfoam, a cardboard box, plastic tablecloth, and stickers (TrusUGI model; Fig. 1). It is meant to mimic the anatomy of the

Journal of Surgical Education � Volume 71/Number 3 � May/June

UGI tract and requires scope traversal, tip deflection, torque,and retroflexion. The user must identify all of the sticker targetswithin the model, and time to completion of the task is therecorded metric. Estimated cost for construction is less than $5.The second simulator is a commercially available colono-

scopy model (CM-15, Olympus Inc, Japan) that consists ofa lifelike latex colon mounted via springs to a base board(Fig. 2). The configuration of the spring mounts can bealtered to create different types of anatomy (e.g., redundantsigmoid and redundant transverse colon) and the springsthemselves allow for stretching of the colon when traversingflexures. The model also has a pneumatic sphincter at the

2014 283

FIGURE 3. The MITIE flexible endoscopy targeting model

FIGURE 4. The MITIE flexible endoscopy targeting model. (A) Trainee praendoscopy tower with affixed “Smiley” face indicating proper target alignmoperation game probe. (D) Hot biopsy forceps with marked line indicating m

284 Journal of

scope insertion site which allows for use of insufflation. Forthis study, the model was arranged into configuration no. 5,which creates a very redundant sigmoid colon that ischallenging to traverse. Scope traversal, torque, tip deflec-tion, loop reduction, and management of insufflation arethe skills required to successfully navigate to the cecum.Time to cecal intubation was the measure of performance.The estimated cost of the model is approximately $1800.The third model was developed to practice targeting. It is

constructed from pool hose, plastic peg board, and theOperation Game from Hasbro (MITIE Flexible EndoscopyTargeting model; Fig. 3). A colonoscope is passed to the end ofthe pool hose and a hot biopsy forceps wired to the OperationGame is passed down the working channel (Fig. 4A and C).Numbered targets are affixed to each of the wire-rimmed wellson the game. The user must align the forceps between the 5-and 7-o’ clock position on the screen and touch the wire rimof the well next to the target (Fig. 4B). Correct orientation isassured by visualizing the forceps on the endoscopy monitorpositioned within a “smile” drawn on a plastic overlay affixedto the monitor (Fig. 4B). The user receives an audio alert whencontact with the wire rim has been made. If the user touchesthe target without the proper tool orientation, touches thesurface of the board, or extends the tool too far outside of the

cticing on the MITIE flexible endoscopy targeting model. (B) Flexibleent. (C) Hot biopsy forceps with attached electrode connected to theaximal proper deployment range.

Surgical Education � Volume 71/Number 3 � May/June 2014

TABLE 1. Demographics of 12 Expert Endoscopist Participants

Demographics for Expert Surgical Endoscopists(n ¼ 12)

Average age 40.6 (33-51 y)Gender 12 malesSpecialty 42% MIS, 58% GSYears in practice 7.6 (1-21)Fellowship training 12 100%Handedness 12 rightYears doing endoscopy 8.8 � 6.9 (1-24)Train residents in endoscopy 10 83%Train fellows in endoscopy 10 83%Use flexible endoscopy simulatorsto train 9 75%

Upper endoscopies per year

5 51þ2 101þ4 201þ1 500þ

Total upper endoscopy cases 2 101þ10 500þ

Lower endoscopies per year

2 o103 11þ4 51þ1 101þ1 201þ1 500þ

Total lower endoscopy cases1 11þ6 101þ5 500þ

TABLE 3. Recalculated Proficiency Metrics

Data From Expert Trials on the STEP EndoscopyModels

Model Time (s) Errors

Trus Upper Endoscopy model 133.1 (�55.5) N/ACM-15 Colonoscopy model 325.1 (�155.5) N/AMITIE Flexible EndoscopyTargeting model

272.8 (�109.0) 3.2(�3.2)

working channel (as indicated by seeing a black mark drawnon the biopsy forceps; Fig. 4D), an error is assessed. Scopetraversal, tip deflection, torque, and targeting are the skillsrequired for successful performance. Time to identify all targetsand errors were the metrics of performance. The estimated costof the model is approximately $50.To measure face and content validity, questionnaires were

developed with responses recorded along a 5-point Likertscale with descriptive anchors at 1, 3, and 5. Each studysubject was asked to complete the questionnaires immedi-ately after working on all 3 models.

METHODS

After obtaining informed consent, 12 experienced surgicalendoscopists were enrolled in this Institution Review

TABLE 2. Performance Metrics of All Experts

Data From Expert Trials on the STEP EndoscopyModels

Model Time (s) Errors

Trus Upper Endoscopy model 136.9 (�65.6) N/ACM-15 Colonoscopy model 342.0 (�187.4) N/AMITIE Flexible EndoscopyTargeting model

274.8 (�118.1) 3.5(�3.7)

Journal of Surgical Education � Volume 71/Number 3 � May/June

Board–approved study. The demographics for this groupare shown in Table 1. Each subject was given a verbalorientation to the models and told what performancemetrics would be recorded. They were then allowed toperform 1 repetition on each model. For the Trus UGImodel, an Olympus GIF180 gastroscope connected to theSTEP tower (camera box, light source, insufflator, irriga-tion, suction, and LCD monitor) was used. An OlympusCFL-180 colonoscope was used for the CM-15 and MITIEmodels. The built-in timer on the STEP tower was used totrack time. One of the experts was only available to performon the MITIE Targeting model leaving 11 experts forperformance on the Trus and CM-15 models. During aninitial trial on the CM-15 model in “case 1” configuration,the colonoscopy was found to be too simple to challenge thegroup of experts; results of that trial were thrown out, andthe model was subsequently reconfigured to “case 5” with aredundant sigmoid and transverse colon. Navigation of thisconfiguration proved somewhat challenging for most of theremaining 10 expert surgeons. After working on all 3 mod-els, each participant completed exit surveys to assess faceand content validity. Performance on the Trus and CM-15model recorded time and task completion only. Time anderrors were recorded for the MITIE Targeting model.

RESULTS

All experts were right-handed males, average age 40 years,with a mean of 8 years of endoscopic practice (range:1-24 y). They all had completed a general surgery residency

TABLE 4. Recommended Proficiency Metrics for Trainees

Suggested STEP Proficiency Metrics

Model Time(s) Errors Repetition

Trus Upper Endoscopy model 133 N/A 2consecutive

CM-15 Colonoscopy model 325 N/A 2consecutive

MITIE Flexible EndoscopyTargeting model

273 r3 2consecutive

2014 285

TABLE 5. Exit Survey Results for the STEP Endoscopy Models

TABLE 6. Deconstructed Skills Required to Perform FlexibleGI Endoscopy

Scope Navigation� Tip deflection� Scope traversal� Torque� Use of 2-handed technique

Loop reductionRetroflexionTraversing a sphincterManagement of insufflationTargeting

286 Journal of

followed by a specialty fellowship in advanced laparoscopyor flexible endoscopy or both (Table 1). Among them, 83%teach residents or fellows and use simulation to do so. Mostperform more than 50 upper endoscopies (51 to 4500 peryear) and 100 colonoscopies (101 to 4500 per year) peryear. The performances for the group are shown in Table 2.To decrease variability in the data (some experts performedvery few upper or lower endoscopies), the highest andlowest scores were eliminated and the mean group perform-ance recalculated (Table 3). The adjusted data demonstratethat an average time for complete navigation of the TrusUGI model with correct identification of all targets was

Surgical Education � Volume 71/Number 3 � May/June 2014

FIGURE 5. GI mentor II simulator.

133 � 56 seconds. Cecal intubation in the colonoscopymodel with successful loop reduction averaged 325 � 156seconds. Proper orientation and successful targeting usingthe Operation Game model averaged 273 � 109 secondswith 3 � 3 errors. Based on this performance, a recom-mended proficiency for all 3 models is outlined in Table 4,with learners having to achieve times equivalent to the meanof experts (with 3 or fewer errors on the MITIE Targetingmodel) for 2 consecutive repetitions. Table 5 shows theresults of the exit surveys. The MITIE Targeting model hadthe strongest face and content validity results (100% agreedor strongly agreed that the technical skills required reflectedthose needed in clinical endoscopy and 100% that the task

TABLE 7. FES Test Modules

FES Test Modules on GI Mentor II

Module I Navigation with traversal, tip deflection,and torque

Module II Loop reductionModule III Upper GI endoscopy with retroflexionModule IV Mucosal evaluationModule V Targeting

Journal of Surgical Education � Volume 71/Number 3 � May/June

encompassed skills sets an experienced endoscopist shouldhave) followed by the CM-15 Colonoscopy model (82%each) and the Trus UGI model (64% and 73%, respec-tively). The MITIE model was also the most favorablyreviewed regarding appropriate difficulty (100% agreed),usefulness for training (100% agreed), and suitability forinitial training in flexible endoscopy (82% agreed). Oneparticipant felt that it was possible to “cheat” on the CM-15by using excessive speed and force to rapidly pass throughthe colon, taking advantage of low kinetic friction imme-diately after initial lubrication; whereas, another felt that theCM-15 lacked consistency in the level of difficulty owing tothe lubrication settling over time causing greater frictionand resistance to scope traversal and loop reduction.

DISCUSSION

The progression of laparoscopic surgery has led to generalsurgeons becoming experts in the use of a rigid laparoscopeand interpreting images projected in 2 dimensions on avideo monitor. However, over a similar period, manysurgeon’s skills in using a flexible endoscope have declinedcreating a deficit in the surgical work force as the minimallyinvasive surgery world progresses to less and less invasiveprocedures using a combination of rigid and flexible endos-copy. It is not unreasonable to believe that disease processessuch as gastroesophageal reflux, morbid obesity, and GIleaks and fistulas would be commonly managed using aflexible endoscopic platform. In addition, rural surgeonsapplying to take their recertifying examination list colono-scopy as the most common procedure they do. As a result,general surgeons have a need to acquire and maintain skillsin flexible GI endoscopy. To meet this need, better trainingcurricula are required during general surgery residency. TheSTEP program was developed to meet this need.The STEP curriculum describes numerous models that

can be used for developing hands-on skills in flexibleendoscopy. These range from very inexpensive physicalmodels to more costly computer-generated ones. This studychose to focus on physical models that are relativelyinexpensive, readily available, and useful for learning multi-ple skills in GI endoscopy. To illustrate which skills thesemodels encompass, it is useful to deconstruct esophagogas-troduodenoscopy and colonoscopy into their essential tasks.Such a deconstruction was conducted by experts whendeveloping the Fundamentals of Endoscopic Surgery (FES)—a validated test of knowledge and skill in GI endoscopy—and shown in Table 6.1 FES incorporates most of theseskills into a 5-module test conducted on the GI Mentor IIsimulator (Simbionix Ltd., Israel; Fig. 5, Table 7). Table 8maps the skills required to complete the tasks on the3 models in this article to the 5 modules on the FES testand shows that training to proficiency on these inexpensivephysical models would enable a trainee to rehearse the same

2014 287

TABLE 8. STEP Model Deconstructed Skills Mapped to FES Skills

Deconstructed Task ListSTEP Models GI Mentor II FES Testing Modules

Trus CM-15 Operation Targeting Game I II III IV V

Tip deflection ● ● ● ● ● ● ● ●Scope traversal ● ● ● ● ○ ● ●Torque ● ● ● ● ● ● ● ●Two-handed technique ● ● ● ● ● ● ● ●Loop reduction ● ●Retroflex ● ●Traverse sphincterManagement of insufflation ○Targeting ● ● ●●, major utilization; ○, minor utilization.

skills that are required to pass the FES examination. This isparticularly important because trainees are not able topractice the FES modules before taking the examination.Strengths of this study include the relatively large number

of experts that provided performance metrics used to establishproficiency and that the skills required to perform the taskson the model encompass almost all of those determined to beimportant in the FES program. We also believe that requiring2 consecutive repetitions of expert performance is importantto achieve a level of mastery that is likely to be retained.Experience in our laboratory during rigorous Fundamentalsof Laparoscopic Surgery proctoring sessions has shown thatonce a trainee is able to achieve a proficient score on 2 con-secutive repetitions, their subsequent performance remains inthe proficient range and more trials are not required. Conver-sely, trainees that only achieve a proficient score on 1 repe-tition do not remain consistent and would require morerepetitions to achieve durable proficiency.Weaknesses of the study include having the experts per-

form only 1 repetition of each skill and no prior validation ofthe metrics used for determining proficiency. Because we arenot seeking to develop a high stakes test of skill in endoscopybut merely a target performance for structured rehearsal, it wasfelt that taking advantage of large group of expert performanceefficiently was adequate for “setting the bar” for practice. Inaddition, we are not attempting to create a high-fidelity simu-lator de novo, but simply determining how to use commonlyavailable tools for rehearsal. Further study would determinethe effectiveness of training to proficiency on these models byeither linking them to performance on the FES skills test or toglobal assessment of gastrointestinal endoscopic skills—avalidated measure of the clinical performance of endoscopy.5

288 Journal of

CONCLUSION

This study proves face and content validity for 3 physicalflexible GI endoscopy simulators that can be used to train inupper and lower endoscopy. It also establishes expertproficiency metrics that can be used by trainees forstructured rehearsal. These relatively inexpensive modelswill be incorporated into the STEP curriculum.

REFERENCES

1. Johna S, Klaristenfeld D. Surgery resident training inendoscopy. The saga continues. Arch Surg. 2011;146(8):899-900.

2. Dunkin BJ. Fundamentals of endoscopic surgery. InTichansky DS, Morton J, Jones DB, eds. The SAGESManual of Quality, Outcomes, and Patient Safety. NewYork: Springer, 2012;473-484.

3. The American Board of Surgery (US). ABS Statementon GI Endoscopy [Internet]. Feb 24, 2011. Available at:⟨http://www.absurgery.org/default.jsp?newsgiresponse⟩.

4. Morales MP, Mancini GJ, Miedema BW, et al. Inte-grated flexible endoscopy training during surgical resi-dency. Surg Endosc. 2008;22(9):2013-2017.

5. Vassiliou MC, Kaneva PA, Poulose BK, et al. GlobalAssessment of Gastrointestinal Endoscopic Skills (GAGES):a valid measurement tool for technical skills in flexibleendoscopy. Surg Endosc. 2010;24(8):1834-1841.

Surgical Education � Volume 71/Number 3 � May/June 2014