USE OF A VIRTUAL REALITY SIMULATOR FOR URETEROSCOPYTRAINING

LUCAS JACOMIDES, KENNETH OGAN, JEFFREY A. CADEDDU AND MARGARET S. PEARLE*,†From the Department of Urology, The University of Texas Southwestern Medical Center, Dallas, Texas

ABSTRACT

Purpose: Virtual reality surgical simulators may shorten operative time and reduce thepotential for iatrogenic injury by providing training outside the operating room. We hypothesizedthat training on a virtual ureteroscopy (VU) simulator would allow novice endoscopists toovercome the initial learning curve before entering the operating room.

Materials and Methods: We evaluated 16 medical students on their ability to perform specificureteroscopic tasks on a VU simulator. The students trained on the simulator for a total of 5hours over multiple sessions using different training modules and then were retested on theinitial module. Likewise, 16 urology residents with varying degrees of endoscopic experience wereassessed on the same test module twice, without additional simulator training.

Results: The students improved task completion time from 17.4 to 8.7 minutes (p �0.05), whilethe residents performed the task in 7.6 minutes at baseline and 6.7 minutes at the second trial.Stratification of residents by years of urology training revealed that the mean completion time forthe students after training did not differ statistically from that of first year residents who hadperformed a median of 14 clinical ureteroscopies. Furthermore, the subjective performance scoresof the students were comparable to those of the first year residents.

Conclusions: Novice medical students trained on a VU simulator improved task completiontime by 50% after training, and performed comparably to residents who had completed nearly 1year of urology training. VR training may allow beginning urology residents to shorten the initiallearning curve associated with ureteroscopy training, although this hypothesis requires furthervalidation.

KEY WORDS: virtual system, ureteroscopy, simulation, medical education

Endourology and laparoscopy pose unique challenges forresident training because the visual field is completely unlikeopen surgery. Altered depth perception, decreased tactilefeedback and increased demand on hand-eye coordination1

contribute to a steep learning curve associated with someprocedures, which can then translate into longer operativetime and increased patient morbidity.2, 3 Until now, therehave been few practical training substitutes for live patientsas teaching resources. The use of live animals for surgicaltraining has been discouraged due to ethical concerns andhigh costs.4 Likewise, the use of human cadavers has alsobeen limited by high cost and limited supply.1, 5

Inanimate bench models, which include basic skills train-ers and virtual reality (VR) simulators, provide a potentialtraining resource. These models are widely used in otherindustries, such as aviation training,6 and have recently beenintroduced in several medical disciplines, including urology,pulmonology, gastroenterology, cardiology and anesthesiolo-gy.7–9 We recently evaluated the use of the Uromentor virtualureteroscopy (VU) simulator (Simbionix, Lod, Israel) to traininexperienced medical students, and found that studentswho trained for 2.5 hours on the simulator outperformeduntrained control students on subsequent testing.10 Our cur-

rent goal was to determine if inexperienced medical studentscould overcome the clinical learning curve faced by urologyresidents by training on a VU simulator.

MATERIALS AND METHODS

We recruited 16 novice medical students (10 first year and6 second year) to participate in this Institutional ReviewBoard approved study. Mean student age � SD was 26.1 �6.9 years (range 19 to 45), with an equal number of males andfemales.

After orientation to the simulator, each student performeda series of tasks comprising a training module, includingrigid cystoscopy, ureteral orifice cannulation with a Bentsonwire, flexible ureteroscopy and systematic inspection of thecollecting system (figs. 1 and 2). This training module con-sisted of a collecting system containing 4 upper pole calices,2 middle calices, and a single lower pole calix (fig. 3). Thesimulator is capable of recording total time to task comple-tion, fluoroscopy time, number of attempts at ureteral orificecannulation and number of traumatic injuries to the collect-ing system caused by the instruments. Each student was alsosubjectively assessed by 2 experienced observers using aglobal rating scale of 1 to 5, with 5 being the best and 1 theworst in the 4 individual categories of anatomy, instrumentmanipulation, use of tools and task completion), as well asoverall performance.

The students then trained on the simulator for a total of 5hours during a 2-week period under the direct supervision of1 urology resident involved in the study. During these ses-sions the students practiced various endoscopic training sce-narios on the simulator, all of which differed from the initial

Accepted for publication August 8, 2003.This investigation was conducted at the Southwestern Center for

Minimally Invasive Surgery, which is supported by a research grantfrom United States Surgical Corporation, a division of TYCO Health-care Group.

Study received Institutional Review Board approval.* Correspondence and requests for reprints: Department of Urology,

The University of Texas Southwestern Medical Center, 5323 HarryHines Blvd., Dallas, Texas 75390-9110 (telephone: 214-648-6853; FAX:214-648-8786; e-mail: margaret.pearle@utsouthwestern.edu).

† Financial interest and/or other relationship with Applied Medi-cal, Circon ACMI and Boston Scientific.

0022-5347/04/1711-0320/0 Vol. 171, 320–323, January 2004THE JOURNAL OF UROLOGY® Printed in U.S.A.Copyright © 2004 by AMERICAN UROLOGICAL ASSOCIATION DOI: 10.1097/01.ju.0000101515.70623.4a

320

test module. After training the students were then reas-sessed by the same 2 observers on the original module.

All 16 urology residents in our residency program weresimilarly assessed on the test module twice (separated by 2weeks) without receiving interim simulator training. Resi-dent performances were then stratified by years of urologytraining (U1 to U4, with each class comprised of 4 residents)for the purposes of comparison to the students. The residentswere also polled on the number of ureteroscopic cases theyhad performed in residency to date, as prospectively recordedin the surgical logs.

The task completion times and subjective assessments ofall participants were analyzed using commercially availableSigma Stat software (SPSS Inc., Chicago, Illinois). Compar-ison between pre-test and post-test performances was per-formed using paired t tests, while subgroup comparisonswere performed using unpaired t tests.

RESULTS

The paired task completion times for each group are shownin figure 4. The medical student group improved mean taskcompletion time from 17.4 minutes before training to 8.7minutes after training (50% time reduction, p �0.05). Inaddition, they significantly reduced the need for fluoroscopy(fluoroscopy time decreased from 33.2 to 14.3 seconds), at-tempts at cannulation of the ureteral orifice (14.4 to 4.9) andinstances of instrument trauma (0.6 to 0.1). In fact, duringthe second trial only 1 episode of mucosal trauma occurredamong all 16 students.

The resident group completed the test module in a mean of7.6 minutes at baseline but improved to only 6.7 minutes atthe second test (12% time reduction, p not significant). Ofnote, the trained medical students required only 2 minutesmore than the urology residents to complete the ureteroscopytest module, although this difference remained statisticallysignificant (p �0.05). Interestingly, during the second trialthe trained students used significantly less fluoroscopy andcaused significantly less iatrogenic trauma than did the res-idents. The trained students also required a comparablenumber of attempts to cannulate the ureteral orifice (table 1).Review of the global rating scores revealed that the residentswere consistently rated higher than the students in all 5categories at the second assessment, although in the categoryof ureteroscope manipulation the difference did not reachstatistical significance (table 1).

Stratification of the residents by years of urology trainingrevealed that the first and second task completion timescorrelated directly with the level of training, with improvedperformance on the simulator with each year of residency

FIG. 1. Virtual ureteroscopy simulator

FIG. 2. VU instruments

FIG. 3. VU monitor depicting human collecting system

FIG. 4. Task completion times of students and residents

VIRTUAL URETEROSCOPY SIMULATOR 321

(fig. 5). In addition, the incremental improvement betweenthe 2 trials was inversely related to the level of training, iethere were negligible differences between the 2 trials for themore advanced residents but relatively greater improvementon the second trial for the less experienced residents.

At the time of this study first year residents had almostcompleted year 1 of urology training and had performed amedian of 14 clinical ureteroscopy procedures. However,their second trial time (7.6 minutes) did not differ statisti-cally from that of the trained student group (8.7 minutes). Inaddition, the subjective evaluations of the 2 groups werestatistically similar, except for superior anatomy recognitionby the residents (table 2).

DISCUSSION

Laparoscopy and endourology are particularly well suitedfor inanimate training models.11 The purported advantagesof bench models include availability and learning in an opti-mal unsupervised, stress-free environment. In contrast tohumans or animals, they are standardized, predictable andreproducible with unlimited opportunity for repetition, feed-back and correction of errors. In addition, simulators may beprogrammed to simulate or plan a particular procedure.

The utility of bench models for improving surgical skillshas been previously demonstrated.1, 5, 7–17 Indeed, some stud-ies have suggested that novices could potentially perform aswell as experienced surgeons after skills training.8, 12, 13 In astudy similar to ours Colt et al compared 5 inexperiencedpulmonary/intensive care fellows to a control group of 4 ex-perienced pulmonologists on a virtual bronchoscopy simula-tor.8 After 4 hours of training on the simulator, the fellowsimproved dexterity and accuracy, and performed as well as,and in some instances better than, the experienced group onthe simulator. Eubanks et al showed that medical studentscould improve basic laparoscopy skills after dry lab trainingby the same magnitude as surgery residents participating in

the same dry lab training module.12 Likewise, Rosser et alreported that inexperienced surgery residents could betrained to perform basic laparoscopic skills as well as trainedfaculty surgeons after both groups participated in a 2.5-dayskills training program.13

Use of virtual reality simulators during medical studentand resident training has the potential to reduce the opera-tive learning curve typically required to master endoscopicskills. Indeed, after training on the VU simulator, the med-ical students in our study were able to perform as well on thesimulator as urology residents who had performed approxi-mately 14 clinical ureteroscopy cases. As such, VU trainingcould potentially lower the incidence of operative injury dueto inexperience and reduce overall operative cost. Daniels etal found that major complications from ureteroscopy weresignificantly more common early rather than later in theexperience of the surgeon (14% complication rate earlyin experience vs 3% complication rate late).3 Bridges andDiamond estimated that $53 million are spent annually inadditional operating room cost training surgical residents,translating into a $48,000 cost for each graduating resident.2Therefore, although virtual reality simulators are costly, thepotential savings in reduced operative time and morbiditymust be considered.

Our study also confirmed the construct validity of thesimulator, ie those with more clinical experience demon-strated an increased level of competence on the simula-tor.14, 15 As such, the VR simulators could potentially be usedto objectively assess resident ureteroscopic skills throughouttraining and ensure an appropriate level of competence com-mensurate with level of training. Similar objective skillslaboratories are already in place in several general surgerytraining programs.5, 9, 13–16 Others in fact have suggested thatVR simulators be used to predict operative skill of prospec-tive residency applicants during the selection process.17 Ob-viously, this application would be controversial and wouldrequire strict validation of the simulator as a predictive tool,as even 1 mistaken assessment could cost a potentially train-able candidate their chosen career.

We acknowledge several limitations of our study. Ourstudy population was a small heterogeneous sample withouta control group of untrained students. However, we previ-ously reported that students who trained on the VU simula-tor for 2.5 hours outperformed their untrained counterpartson subsequent testing.10 In addition, our observers were un-blinded to the status of the individual being tested (studentversus resident). Consequently, subjective evaluation waspotentially biased in favor of the residents. For instance thestudents actually caused fewer traumatic injuries and re-quired fewer cannulation attempts compared to the resi-dents, although the residents were consistently rated higherin subjective parameters (table 1). The observers may haveperceived the shorter performance times and perhaps moreconfident demeanor of the residents as suggestive of subjec-tively better, albeit not more careful, performance.

TABLE 1. Measurements and evaluations of residents and trainedstudents

Students Residents p Value

Objective measurements(mean � SE):

Time (mins) 8.7 � 0.5 6.7 � 0.3 �0.01*Fluoroscopy time (secs) 14.3 � 1.7 23.7 � 2.6 �0.01*Cannulation attempts 4.9 � 1.5 6.3 � 1.1 0.45Traumatic injuries 0.1 � 0.1 1.4 � 0.2 �0.01*

Subjective evaluations ofresidents (mean � SE):†

Anatomy 3.8 � 0.2 4.8 � 0.1 �0.01*Manipulation 3.6 � 0.2 3.9 � 0.1 0.12Tools 3.5 � 0.2 4.0 � 0.2 0.03*Tasks 3.5 � 0.2 4.1 � 0.1 0.02*

Overall 3.6 � 0.2 4.1 � 0.1 0.03** Statistically significant.† Parameters measured on a scale from 1 to 5, with 1 being the worst and 5

the best.

FIG. 5. Comparison of task completion times between studentsand residents stratified by year of training. MS, medical students.U1 to 4, urology year of training.

TABLE 2. Objective and subjective scores of trained students andfirst year residents

Mean � SEp ValueStudents

(16)1st Yr Residents

(4)

Time (mins) 8.7 � 0.5 7.6 � 0.4 0.08Fluoroscopy time (secs) 14.3 � 1.7 20.8 � 3.9 0.12Cannulation attempts 4.9 � 1.5 6.5 � 1.8 0.62Traumatic injuries 0.1 � 0.1 1.0 � 0 �0.01*Anatomy 3.8 � 0.2 4.7 � 0.1 0.01*Manipulation 3.6 � 0.2 3.5 � 0.1 0.83Tools 3.5 � 0.2 3.4 � 0.2 0.90Tasks 3.5 � 0.2 3.5 � 0.2 0.94

Overall 3.6 � 0.2 3.6 � 0.1 0.97* Statistically significant.

VIRTUAL URETEROSCOPY SIMULATOR322

Another concern is that our study only measured outcomeson the simulator, while not assessing the ability of simulatortraining to improve clinical operative performance. The find-ings that urology residents performed better on the simulatorwith increasing years of training and that the performance ofthe trained medical students was inferior to that of moreexperienced residents indirectly validate the ability of thesimulator to faithfully represent the clinical experience. Nev-ertheless, the ultimate question of whether the skills learnedon the VU simulator are transferable to the operating roommust be addressed. While there are obvious limitations toassessing medical students in the operating room, we arecurrently in the process of validating the simulator using ahuman cadaver model.

CONCLUSIONS

Novice medical students tested on a virtual ureteroscopysimulator significantly improved task completion time andoperative skills with training. Moreover, the performance ofthe students on the simulator after training was statisticallysimilar to that of residents who had completed nearly 1 yearof urology training. This pilot study suggests that VU skillstraining may help first year residents overcome the initialclinical ureteroscopy learning curve, although further clini-cal validation is required.

REFERENCES

1. Scott, D. J., Bergen, P. C., Rege, R. V., Laycock, R., Tesfay, S. T.,Valentine, R. J. et al: Laparoscopic training on bench models:better and more cost effective than operating room experience?J Am Coll Surg, 191: 272, 2000

2. Bridges, M. and Diamond, D. L.: The financial impact of teachingsurgical residents in the operating room. Am J Surg, 177: 28,1999

3. Daniels, G. F., Jr., Garnett, J. E. and Carter, M. F.: Uretero-scopic results and complications: experience with 130 cases.J Urol, 139: 710, 1988

4. Rowan, A. N.: Is justification of animal research necessary?JAMA, 269: 1113, 1993

5. Anastakis, D. J., Regehr, G., Reznick, R. K., Cusimano, M.,Murnaghan, J., Brown, M. et al: Assessment of technical skillstransfer from the bench training model to the human model.Am J Surg, 177: 167, 1999

6. Rolfe, J. M. and Staples, K. J.: Flight Simulation. Cambridge,United Kingdom: Cambridge University Press, p. 232, 1986

7. Issenberg, S. B., McGaghie, W. C., Hart, I. R., Mayer, J. W.,Felner, J. M., Petrusa, E. R. et al: Simulation technology forhealth care professional skills training and assessment.JAMA, 282: 861, 1999

8. Colt, H. G., Crawford, S. W. and Galbraith, O., 3rd: Virtualreality bronchoscopy simulation: a revolution in proceduraltraining. Chest, 120: 1333, 2001

9. Derossis, A. M., Bothwell, J., Sigman, H. H. and Fried, G. M.:The effect of practice on performance in a laparoscopic simu-lator. Surg Endosc, 12: 1117, 1998

10. Wilhelm, D. M., Ogan, K., Roehrborn, C. G., Cadeddu, J. A. andPearle, M. S.: Assessment of basic endoscopic performanceusing a virtual reality simulator. J Am Coll Surg, 195: 675,2002

11. Kuo, R. L., Delvecchio, F. C. and Preminger, G. M.: Virtualreality: current urologic applications and future developments.J Endourol, 15: 117, 2001

12. Eubanks, T. R., Horgan, S., Resnick, B., Pellegrini, C. andSinanan, M.: Laparoscopic drills do not improve objectivelymeasured motor skills in surgery residents. Proceedings ofSociety of American Gastrointestinal Endoscopic Surgeons an-nual meeting, Seattle, Washington, April 1998

13. Rosser, J. C., Jr., Rosser, L. E. and Savalgi, R. S.: Objectiveevaluation of a laparoscopic surgical skill program for resi-dents and senior surgeons. Arch Surg, 133: 657, 1998

14. Derossis, A. M., Fried, G. M., Abrahamowicz, M., Sigman, H. H.,Barkun, J. S. and Meakins, J. L.: Development of a model fortraining and evaluation of laparoscopic skills. Am J Surg, 175:482, 1998

15. Martin, J. A., Regehr, G., Reznick, R., MacRae, H., Murnaghan,J., Hutchison, C. et al: Objective structured assessment oftechnical skill (OSATS) for surgical residents. Br J Surg, 84:273, 1997

16. Reznick, R., Regehr, G., MacRae, H., Martin, J. and McCulloch,W.: Testing technical skill via an innovative “bench station”examination. Am J Surg, 173: 226, 1997

17. Macmillan, A. I. and Cuschieri, A.: Assessment of innate abilityand skills for endoscopic manipulations by the AdvancedDundee Endoscopic Psychomotor Tester: predictive and con-current validity. Am J Surg, 177: 274, 1999

EDITORIAL COMMENT

As a medical student 20 years ago I recall a surgery intern saying“Monkeys can be trained to perform surgery.” Are we there yet? Theauthors trained medical students to perform virtual ureteroscopycomparably to junior urology residents. This finding implies thatother novice ureteroscopists (beginning junior residents or olderpracticing urologists with no prior ureteroscopy experience) mightacquire ureteroscopy skills using VU.1 The incremental improve-ment decreased as training level increased, implying that educa-tional benefits of VU training are greatest for (and perhaps limitedto) ureteroscopy novices.

However, the study end point was VU performance. The authorsdid not show content validity (does VU testing actually assess clin-ical ureteroscopy skills?) or criterion validity (does VU predict per-formance on clinical ureteroscopy?). Criterion validity is necessary ifVU is used for resident applicant screening. Thus, all we know is thatnovice ureteroscopists can be trained to play an expensive ureteros-copy video game and play it as well as junior urology residents. Weneed a study in which junior residents are randomized to VU versusno VU training and then assess clinical ureteroscopy skills, withattending observers blinded to VU training status.

So what is the role for VU? I believe that VU will flatten thelearning curve for novice ureteroscopists. Furthermore, virtual sim-ulators are already being tested for other skills.2 It seems likely thaturology trainees will have increased opportunities to learn throughvirtual surgery simulators. It may not be far-fetched that we cansoon train monkeys to perform ureteroscopy.

Joel M. H. TeichmanDivision of UrologySt. Paul’s HospitalVancouver, BC, Canada

1. Watterson, J. D., Beiko, D. T., Kuan, J. K. and Denstedt, J. D.: Arandomized, prospective blinded study validating the acquisi-tion of ureteroscopy skills using a computer based virtualreality endourological simulator. J Urol, 168: 1928, 2002

2. Beiko, D. T., Watterson, J. D., Knudsen, B. E., Bennett, J. D.,Pace, K. T., Honey, R. J. D’a. et al: PERC mentor: a newcomputer-based virtual reality simulator for percutaneous re-nal access. J Urol, suppl., 169: 473, abstract 1775, 2003

REPLY BY AUTHORS

We wholeheartedly agree that VU simulators need to be validatedin the clinical setting. Before training programs line up to purchasethem and VU training becomes pervasive in our educational curric-ulum, it is imperative that content and criterion validity be demon-strated. Otherwise, these simulators are simply glorified videogames.

VIRTUAL URETEROSCOPY SIMULATOR 323