Computer-assisted training and learning in surgery

Review Article

Computer-Assisted Training and Learning in SurgeryPaul J. Gorman, M.D., Andreas H. Meier, M.D., and Thomas M. Krummel, M.D.

Department of Surgery, School of Medicine, Stanford University, Stanford, California

ABSTRACT The teaching and learning of surgery is a time-honored tradition based upon the “seeone, do one, teach one” apprenticeship model. Recent improvement of this model has centered uponincremental change in skills teaching and testing and curricular development. Economic pressureshave strained the resources of academic health centers and faculty responsible for teaching surgery,even as information technology has opened new avenues for obtaining and benefitting from relevantinformation. Combining the tools of simulation theory, virtual reality, and the principles of adulteducation offers new opportunities to optimize surgical education as we enter a more highlyconnected and interdependent era, where the boundaries between teacher and student blur as themodern surgeon truly becomes a lifelong learner. Comp Aid Surg 5:120–130 (2000). ©2000 Wiley-Liss, Inc.

Key words: surgical simulation, surgical education, computer-assisted learning

INTRODUCTIONThe practice of surgery is a learned art, built upona strong foundation of didactic learning, reading,observation, doing under guidance, and repetition.This apprenticeship model has changed little overthe years. Surgical education today, primarily theresponsibility of the academic medical center, iscarried out within the context of teams that com-prise the surgical resident body of an organizedresidency program. Core surgical residencies inthe United States (general surgery, orthopedics,otolaryngology, neurosurgery, and urology) con-sist of a minimum of five years of full-timeclinical training, with progressive responsibilityas the years progress.

This traditional structure has remained rela-tively unchanged over many years. Rapid advance-ment in information technology, and thus the wayknowledge is disseminated and absorbed, threatens

the stability of the old model. Advances in instru-mentation, visualization, and monitoring have en-abled continual growth in minimally invasive tech-niques in surgery, radiology, and cardiology,among others. The operating room is becoming anincreasingly complex environment as technologyimpacts the ways in which surgery is practiced,with marked change over the last ten years andlittle to suggest otherwise for the future.14

The advent, growth, and development ofcomputer-assisted technologies as adjunctive edu-cational, training, and certification modalities insurgery will likely affect current surgical practicein ways that are difficult to predict. The purpose ofthis paper is to explore the growing impact of thetechnology on surgical training and education aswe examine the history of surgical education, theconcepts of adult education, simulation, and virtual

Address correspondence/reprint request to: Thomas M. Krummel, M.D., Stanford University, School of Medicine, Departmentof Surgery, 300 Pasteur Drive, H3680, Stanford, CA 94305-5655; Telephone: (650) 498-4292; Fax: (650) 725-0791; E-mail:[email protected].

* This work is sponsored in part by funds provided by Cisco Systems, Inc.

† Key link: http://catss.stanford.edu

Computer Aided Surgery 5:120–130 (2000)

©2000 Wiley-Liss, Inc.

reality (VR), and review recent advances and ap-plications in the field.

HISTORICAL OVERVIEWSurgical education and training as a subset of grad-uate medical education has drawn increasing inter-est in recent years. Prior to the twentieth century,medical education in the US was erratic and poorlyregulated. Advanced surgical training was oftenobtained in Europe in the mid-nineteenth cen-tury.137 The effects of ether anesthesia and aseptictechniques upon surgical practice radically in-creased the number of operations performed duringthe latter half of the nineteenth century. Combinedwith the surgical training system instituted by Dr.William Halsted of the Johns Hopkins UniversitySchool of Medicine in the 1890s, these advanceslaid the groundwork for the future of surgical sci-ence and training at the turn of the century.63,64

The standard of training in the early 1900sremained poor, however. Growing concerns aboutthe quality of education highlighted by Flexner inhis analysis of 191048 led to widespread activity toimprove surgical and general medical training,81,108

and scientific medical education became a primaryfocus for the first time.61 In 1912, a “minimumstandard of requirements . . . to perform indepen-dent operations” was defined.108 Over the next tenyears, the length of time required for formal surgi-cal training increased. The first approved surgicalinternships appeared in 1914 as the criteria to enterresidency became more stringent.50

With the development of specialty examiningboards in the 1930s and 40s,61 interest in the anal-ysis of surgical education outcomes increased. In1934, the Committee on Graduate Training forSurgery began to develop criteria for graduate sur-gical training.108 Evarts Graham stated that “resi-dency training . . . should be distinctly educationalrather than a means of supplying cheaply an assis-tant to the staff. . .”59 Creation of the ResidencyReview Committee in 1955 enhanced quality con-trol in residency training by introducing an inde-pendent oversight committee and a feedback mech-anism.62

The traditional surgical teaching method thathas developed through this process is based uponthe preceptor or apprenticeship model, in which theresident surgeon learns with small groups of peersand superiors, over time, in the course of patientcare. Surgeons have always acquired most of theiroperative and judgment skills through “learning bydoing.”49 Though an essential portion of surgicalpractice, the majority of technical skill instruction

occurs through fairly unstructured operating roomexposure. In many programs, this hands-on expe-rience excludes the more junior resident learner,and faculty evaluations rarely provide constructivefeedback. The use of videotapes followed by de-briefing, however, has successfully countered thistrend in some cases.96 This method allows for ef-fective feedback, but does require significant fac-ulty input. Computer editing technology may makethis approach more efficient.11

Ideally, exposure to operative practice shouldcommence at an early training level and in anorganized fashion that allows the breakdown oftasks into simple steps7 in a time-flexible environ-ment.61 These prerequisites cannot be fulfilled intoday’s operating rooms. Even if this were possi-ble, accurate assessment of the resident’s progresswould pose another significant challenge.

Due to the nature of surgical practice, theapproach described above will likely remain a cor-nerstone of the surgical education process. Studiesanalyzing learning style preferences of surgical res-idents show that most favor a problem-solving andhands-on approach,6,38 which may partly explainwhy this form of teaching has been so successful.The “learning by doing” approach, though, fails toprovide skill acquisition in an organized fashion.Teaching opportunities are dependent upon the ran-dom flow of patients through the office, clinic,emergency unit, and operating room. The operatingroom itself provides a venue to demonstrate tech-nique and place the operation in the context ofoverall patient management. Indeed, the OR hasbeen termed “the surgeon’s classroom and labora-tory extraordinaire.”120 However, the variability inpatient-flow results in significant unpredictabilityin the educational content provided to the trainees,and precludes the use of an organized curriculum.

As interest in the development of technicalskills training laboratories has grown in recentyears, several investigators have worked to de-velop methods to objectively evaluate surgicalskill.8,96,101,129,134Surgical skills laboratories havebeen successfully used for decades; they were firstintroduced with simple tie and suture boards andpig-skin suturing models in the 1960s.15,88Multipletools and materials have since been used.12,66,84,95,119

Recently, organ perfusion models that allow oper-ative practice and provide the means to simulateintraoperative complications have been intro-duced.56 All of these skills laboratories require aclear curriculum and constructive feedback in orderto be effective.8 The overall validity of skills train-ing in the laboratory setting has been shown re-

Gorman et al.: Surgical Training 121

cently,3 and increased use of this method has beenrecommended.133 However, the significant amountof faculty time necessary to provide this training,and the ethical questions that arise if live animalsare used, are issues that make this approach prob-lematic.

Despite this progress in skills training, eval-uating operative proficiency remains a challengingtask. Psychomotor and dexterity skill testing alonedoes not correlate well with performance in theoperating room.65,117,130The best correlation hasbeen established for visuo-spatial abilities andstress tolerance;33,54,118psychological testing maytherefore have some predictive validity for futuresurgical skill.111,112 Another valid tool is directobservation of technical skill in the operating roomusing predefined criteria based on a Likert Scalerating.96,112

Recently, the Objective Structured Assess-ment of Technical Skills score (OSATS) was foundto be valid and reliable for senior residents, but lessuseful for junior housestaff.45,84 The StructuredTechnical Skills Assessment Form (STSAF) hasdemonstrated inter-rater and construct validity.134

However, no-one has identified a valid, reliable,and sensitive measuring system that is easily ad-ministered, allows for the pre-emptive evaluationof residency candidates, and provides analysis of agiven resident’s progress throughout his or hertraining. With the advent of minimally invasivesurgery in the last decade, and rapid structural andfinancial change within the American healthcaresystem, traditional educational methods in surgeryhave come under increasing scrutiny. Radicalchange in the “see one, do one, teach one” meth-odology of surgical education has been proposedby integrating skill laboratories and surgical train-ers into the curriculum.56

In addition to the hands-on approaches de-scribed above, a variety of more or less formaleducational practices, such as bedside teachingrounds, case conferences, morbidity and mortalityconferences, and grand rounds have evolved.10,51

Though the standard surgical learning and trainingsystem has proven sufficient over time, variousefforts have been undertaken to improve this pro-cess. Problem-based learning, the single-observermethod of evaluation, and organized strategies forteaching in the ambulatory setting have been de-scribed as educational modalities for third- andfourth-year medical students.39,46,85Another inno-vative educational tool, the Objective StructuredClinical Examination (OSCE), has proven useful in

the evaluation of clinical competence of surgicalresidents.115,116

Studies have uncovered significant problemswith the current surgical education curriculum.These include the lack of continuity from under-graduate to graduate surgical training101 and thelack of supervision when acquiring physical exam-ination skills,136ultimately resulting in poor perfor-mance.21,43 Curricular changes within clerkshipshave recently been made or suggested, and objec-tives and curricula for surgeons in training havebeen created.28,52,67It is crucial that this develop-ment process include residents and students as wellas educators; the curricular concerns of the traineesand their feedback regarding changes can then beintegrated.87,135 Such feedback mechanisms havebeen shown to increase the overall educational im-pact of the curriculum.21,36,109Examples of changeinclude the work of Sachdeva and colleagues inadjusting the surgical clerkship at HahnemannUniversity to increase student exposure to basicsurgical principles,100 while Jacobsen proposed aresidency program that focused on developingproblem-solving skills.73 Recent publications haveshown that learning from such analysis, and adjust-ing the training accordingly, improves overall per-formance.13

Why are these issues important? Multiple ex-ternal factors are exerting pressure upon the tradi-tional surgery residency training structure. Thefunding of graduate medical education in surgery isthreatened, while the per-capita workload increasesdue to plateaus in the number of post-graduatetraining positions.26,31 The explosion in healthcarecosts,4,49,97,124 followed by decreasing Medicaresupport for medical education,30,60 has resulted insignificant hardship for surgical training programs.Time in the OR, the traditional learning ground forsurgical residents, has become more precious andcostly.96 These economic changes have importantsecondary effects, such as shorter in-patient stays,further diminishing a traditional learning re-source.34,92 Surgical residents thus take care ofmore acutely ill patients,5 which has the effect ofhindering the educational process by decreasing theamount of time available for formal teaching.61

Surgical educators have not remained oblivi-ous to these changes. Junior faculty members faceincreasing research requirements in order to pursuea successful academic career, leaving less time forformal teaching.91 The resultant lack of facultysupervision has a negative impact on student de-velopment,136 though surveys have shown that fac-ulty members are aware of these shortcomings.39

122 Gorman et al.: Surgical Training

Also, fear of liability and malpractice litigation hasalso put significant restraints on teaching in theOR.24,96

Recognizing these forces, several authorshave suggested that the next step in surgical edu-cation is the adoption of advanced computer-basedsimulators for surgical education and training.41,105

Bridges and Diamond17 estimate that the annualcost of training chief residents in the operatingroom amounts to $53 million per year (generalsurgery alone). They suggest that adjunctive train-ing environments employing traditional and virtualteaching aids may serve to alleviate this cost overtime. Over the last several years, many medicalschools have begun to utilize computers in prob-lem-based learning, thereby decreasing requiredfaculty time. Results with this approach have beenpromising.34,47 Studies in other specialties haveshown that developing and measuring problem-solving skills with a computerized decision analy-sis model is feasible.37 Further alternatives includedistributed education via the Internet, although lit-tle is known about this growing area.35

ADULT EDUCATIONOver the last two decades, educational research hasattempted to define the salient aspects of adultlearning. Adults usually act as self-directed, inter-nally motivated, and experienced students whoseek specific knowledge in their chosen area ofstudy.79 Adult students learn by doing, and areoften most successful when the experience is self-directed.82 Focused upon practical applications, theadult learner gains insight as information is placedwithin a contextual framework.

Historically, mainstream educational systemshave regarded learning as an individual process.Recent evidence suggests that doing so fails toprepare the learner for high achievement in themodern workplace, which is characterized by theneed to successfully use technology to collect, an-alyze, and act upon information.25 To counter this,group or collaborative learning is increasingly usedas a networked learning environment to enablestudents to work together on learning tasks. Effec-tive collaboration can create high levels of interde-pendence and ingenuity among group learners, andit allows enhanced learning not easily attained us-ing other methods.19

Traditional medical training, however, isrigidly structured, lecture-based, and focuses onthe memorization of facts, leaving little room forself-directed education. Therefore, many medicalschools have begun to change their curricula and

have introduced principles of problem-based learn-ing.9 Within the last ten years, these methods havebeen integrated into surgical education.85 Studieshave shown that problem-oriented small groupsmay perform better compared to lecture-basedgroups, especially when asked to solve clinicalproblems,40,113 and that students appear more mo-tivated.22

It remains to be seen whether this approachactually results in better knowledge transfer.85 Thepreviously mentioned OSCE model has also shownpromising results. It has been implemented intolicensing examinations for family medicineabroad,2,16 and there are plans underway to inte-grate the OSCE format into the United States Med-ical Licensure Examination (USMLE).132 It stilllacks widespread use today, however, due to thehigh faculty time commitment required.

SIMULATIONThe concept of simulation in training is not unique,and its utility in education has been recognized forsome time. Perhaps it is most well-known for itsrole in civilian and military pilot and astronauttraining. The idea of simulation in action mayevoke images of game- or role-playing, though itmay be most instructive to consider a simulation asa case study, with the participants “on the inside.”From a functional standpoint, a good simulationrepresents simplified reality, free of the need toinclude every possible detail.77 It is important torealize that simulations are not completely identicalto actual events. Rather, an effective simulationplaces the learner in life-like situations that providereal-time feedback on decisions, actions, and ques-tions.72

Simulation, loosely construed, is the act ofassuming the outward qualities or appearances of agiven object(s) or process or series of processes.Application areas for real-time simulation (whichinvolves the computer modeling of events so thatthey proceed within a defined range of their naturaloccurrence) include training, testing, analysis, andresearch into and development of new products.55

In addition to air and space flight training, trainingsimulators exist for military and commercial vehi-cles, mechanical system maintenance, and nuclearpower plant operation. Transport companies usesimulators to prototype and test ground and airtransport vehicles, primarily because they providetesting environments that are controllable, secure,and safe. The cost-effective use of simulators asdescribed has demonstrated the utility of real-timesimulation as a training tool, and has sparked in-


terest in the development of simulators for otherpotentially dangerous environments (e.g., new orcomplex medical procedures).55

Simulation in medical education has been un-dertaken in a variety of settings. Paramedical per-sonnel are taught triage and assessment skills withthis technique, and Advanced Trauma Life Support(ATLS) and Advanced Cardiac Life Support(ACLS) courses rely upon simulated scenarios toteach and test skills. Screen- and mannequin-basedsimulators have been used in anesthesia training toensure that clinicians will be exposed to unusualsituations not otherwise routinely experienced,such as malignant hyperthermia, anaphylaxis, andcardiac ischemia.114 Efforts to show that these sim-ulators improve clinical performance have beenequivocal. Chopra et al.23 showed that anesthesiol-ogists trained on a high-fidelity anesthesia simula-tor responded more quickly and appropriately whenhandling crises on the simulator. Controlled studiesinvolving human patients to validate this findingwould present an unacceptable risk, however.

Further development of the simulation con-cept evolved out of recognition that two-thirds ofall accidents or incidents in anesthesia can be at-tributed to human error. To counter this, Howardand colleagues71 developed a training program en-titled “Anesthesia Crisis Resource Management” inorder to optimize anesthesiologist and team perfor-mance during stressful incidents. Success in thisarena has lead to the use of mannequin-based sim-ulators in surgery training as an alternative to “real”trauma resuscitations for teaching teamwork andcrisis-management skills.57,86 Deliberate practice,defined as attempting “a well-defined task with anappropriate difficulty level for the particular indi-vidual, informative feedback, and opportunities forrepetition and correction of errors,” has been shownto separate the elite performer from other practi-tioners.44 The use of high-fidelity simulators tomodel variable human conditions may enable de-liberate practice and help fill the void created byreduced attending teaching time and the relativescarcity of in-patients.72

VIRTUAL REALITYIvan Sutherland wrote that the computer “screen isa window through which one sees a virtual world.”While much has been made of virtual reality (VR)in the media, it is important to realize that it basi-cally represents a unique interface to a variety ofthree-dimensional (3D) computer applications.127

The term “virtual reality” was coined by JaronLanier, founder of VPL Research, in the late 1980s.

Virtual reality has also been defined as a humancomputer interface that simulates realistic environ-ments while enabling participant interaction, as a3D digital world that accurately models actual en-vironments, or simply as cyberspace.1,20

Early work on interactive head-mounted dis-plays in the mid-1960s set the tone for developmentin 3D graphical visualization, though it was notuntil the mid-1980s that evolving components al-lowed Lanier et al.110 to develop viable head-mounted displays (HMDs), body suits, and gloves.VR development was supported in part by efforts tobuild better flight simulators, particularly in thehuman factors and input design areas (e.g., HMDsfor NASA).104,110

Virtual reality has been used in a variety ofeducational, training, and entertainment settings.127

The highly visual and interactive nature of VR hasproven to be useful in understanding complex 3Dstructures and for training in visual-spatial tasks.69

Recognition of this has led to increasing interest indeveloping VR-based applications for surgical ed-ucation and training.

The concept of developing and integratingcomputer-based simulation and training aids forsurgical skills education has begun with VR simu-lators. Interest in controlling training risk and costthrough VR simulation has appealed to surgicaleducators,96 but due to the complexity of surgicalanatomy and the limitations of interacting directlywith computer-generated images, surgical simula-tors of this nature have only recently become avail-able. Initially funded by the military to simulateorthopedic injuries, the quality of these devices hasmarkedly increased.32,76,102,122These devices allowfor the simulation of tasks on virtual tissues createdby high-end graphics workstations. The manipula-tion is performed through haptic interfaces, thusallowing measurement of the trainee’s performancethrough precise movement analysis. For the firsttime, objective data regarding motion, tissue tear-forces, precision and error rates can be acquired,which can be compiled into a “surgical report-card.”90,122

TRAINING AND LEARNINGAPPLICATIONSVirtual environments (VE) have been created andused in many areas of medicine. Early develop-ments in the surgical field included the virtual ab-domen created by Satava and Lanier and the hiparthroplasty planning application by Delp and col-leagues.98,103Applications ranging from virtual en-doscopy to interactive anatomy teaching modules,


to acrophobia treatment modalities, and to soft-tissue modeling have been described.29,70,78,99 Aseries of dedicated conferences have sparked inter-est in this field, and reports on VR applications inmedicine can be found in the medical, computerscience, engineering, and popular lay literature.Interest in simulated environments for surgicaltraining is growing. Issenberg et al.72 suggest thatsimulators are ideal for mastering techniques thatdemand repeated practice, and that their use shouldbe considered before allowing trainees to performinvasive maneuvers on actual patients.

The similarities between pilot and surgeonresponsibilities are striking: both must be ready tomanage potentially life-threatening situations indynamic, unpredictable environments. The longand successful use of flight simulators in air andspace flight training has inspired the application ofthis technology to surgical training.107 Perhaps dueto the number of complications resulting from theuncontrolled growth of laparoscopic procedures inthe early 1990s, many groups have pursued simu-lation of minimally invasive and endoscopic pro-cedures. Tendick et al.123 have developed laparo-scopic camera handling and cholecystectomysimulations based on a graphics workstation, whileTseng and co-workers have built a real-time forcefeedback cholecystectomy simulator based on apersonal computer.125

Advances in tissue modeling, graphics, andhaptic instrumentation have enabled the develop-ment of open abdominal and hollow-tube anasto-mosis simulators.18,89 Initial validation studies us-ing these and other simulators have showndifferences between experienced and novice sur-geons, improvement in training scores over time,and that simulator task performance is correlated toactual task performance.58,121,131O’Toole et al.90

have further demonstrated skill improvement andconstruct validity with a VR-based needle-drivingsimulator.

Not surprisingly, there is movement towardthe development of web-based surgical training andlearning applications. El-Khalili et al.42 have dem-onstrated a web-based surgical training system forabdominal aneurysm stent graft deployment. Thisunique system includes instructional web-pagesand measuring and other rudimentary interactivetools to teach and simulate this complex procedure.Though the question of focusing on realism versusaccuracy remains unanswered, this system demon-strates the feasibility of creating web-based surgi-cal training applications. CardioOp53 represents an-other web-based surgical learning system. This

environment allows for the flexible composition ofmultimedia fragments, enabling the re-use of vari-ous data sets to suit the needs of individual learnersin cardiac surgery. John and Philips74 have devel-oped a web interface for simulating ventricularcatheterization, pedicle screw insertion, and lumbarpuncture. Though limited by a lack of tactile orforce feedback, these systems do include perfor-mance assessment tools and the capability of al-lowing many users to access one simulator over theInternet.

VR technology has been used to create sev-eral learning environments. Cotin et al.27 have de-veloped the Interventional Cardiology TrainingSystem (ICTS), which combines a simulated cath-eter and fluoroscope with a haptic interface andanatomical models to augment the learning of car-diologists. A curriculum of standardized virtualpatients has been created, which allows the learnerto experience typical and atypical cases. Anatomyeducation using interactive 3D graphics has re-cently been undertaken at the UC San DiegoSchool of Medicine, and early evaluation of virtualbronchoscopies suggests that it may prove useful inpre-bronchoscopy planning and training.68,128A re-cent report on VR-based flexible sigmoidoscopysimulator training has shown significant improve-ments in exam times and hand-to-eye skill mea-sures.126 Medical students and residents using aVR-based module for intravenous catheter place-ment showed improvement in the VE, but wereunable to transfer that improvement to physicalreality.93

CONCLUSIONSThough in its infancy, the field of VR and simula-tion in surgical education and training is gainingrecognition. The Committee on Emerging SurgicalTechnology and Education of the American Col-lege of Surgeons has sponsored demonstrations ofvarious simulation- and VR-based educational en-vironments at recent meetings, and reports on theseevolving modalities are beginning to appear in themainstream medical literature.80,83,90

The exact effect of this change upon the sur-gical education process is impossible to predict,though evidence suggests positive outcomes willresult. The adult learner succeeds by “doing,” par-ticularly when the experience is self-directed.82 Fo-cused upon practical applications, the adult learnergains insight if information is placed within a con-textual framework. Providing this context within arich visual, auditory, and touch enhanced virtualworld has enabled the transfer of VR-based training


to actual skill.75,94Incorporation of networked VR-based simulations into the surgical curriculumwould leverage the collaborative strength of thepresent team-based structure of most surgical resi-dency and clerkship programs.

It has been reported that information manage-ment comprises 80 to 90 percent of a physician’sdaily workload.106Failure to adapt to the increasingdependence on information (of all kinds) would bea mistake. Use of the new technologies describedabove may help prevent such an outcome, in part,by enhancing the current educational process. Inshort, for reasons of educational quality, safety, andcost, simulation and virtual reality can enhancesurgical training and learning now, and their rolewill almost certainly expand as computer powerand availability increase.

REFERENCES1. Ahmed M, Meech JF, Timoney A. Virtual reality in

medicine. Br J Urol 1997;80(Suppl 3):46–52.2. Allen J, Evans A, Foulkes J, French A. Simulated

surgery in the summative assessment of general prac-tice training: results of a trial in the Trent and York-shire regions. Br J Gen Pract 1998;48:1219–1223.

3. Anastakis DJ, Regehr G, Reznick RK, et al. Assess-ment of technical skills transfer from the bench train-ing model to the human model. Am J Surg 1999;177:167–170.

4. Bagley JS. The problems of surgeons in training. JRoyal Coll Surg Edinburgh 1996;41:206–207.

5. Bahnson HT. Education of a surgical chairman. AnnSurg 1988;208:247–253.

6. Baker JD 3rd, Reines HD, Wallace CT. Learningstyle analysis in surgical training. Am Surg 1985;51:494–496.

7. Barnes RW. Surgical handicraft: teaching and learn-ing surgical skills. Am J Surg 1987;153:422–427.

8. Barnes RW, Lang NP, Whiteside MF. Halstediantechnique revisited. Innovations in teaching surgicalskills. Ann Surg 1989;210:118–121.

9. Barrows HS. Problem-based, self-directed learning.JAMA 1983;250:3077–3080.

10. Bartlett RH. Teaching from patients. In: Bartlett RH,Strodel WE, Zelenock GB, Harper ML, Turcotte JG,editors: Medical Education: A Surgical Perspective.Chelsea, MI: Lewis Publishers; 1986. p 7–22.

11. Beckmann CR, Lipscomb GH, Ling FW, BeckmannCA, Johnson H, Barton L. Computer-assisted videoevaluation of surgical skills. Obstet Gyn 1995;85:1039–1041.

12. Bevan PG. The craft of surgery. The anastomosisworkshop, March 1981. Annals of the Royal Collegeof Surgeons of England 1981;63:405–410.

13. Blue AV, Griffith CH 3rd, Wilson J, Sloan DA,Schwartz RW. Surgical teaching quality makes adifference. Am J Surg 1999;177:86–89.

14. Bodily KC. Presidential address. Surgeons and tech-nology. Am J Surg 1999;177:351–353.

15. Boyle DE, Gius JA. Tie and suture training board.Surgery 1968;63:434–436.

16. Brailovsky CA, Grand’Maison P, Lescop J. A large-scale multicenter objective structured clinical exam-ination for licensure. Acad Med 1992;67:S37–S39.

17. Bridges M, Diamond DL. The financial impact ofteaching surgical residents in the operating room.Am J Surg 1999;177:28–32.

18. Bro-Nielsen M, Helfrick D, Glass B, Zeng X,Connacher H. VR simulation of abdominal traumasurgery. In: Westwood JD, Hoffman HM, StredneyD, Weghorst SJ, editors: Medicine Meets VirtualReality: Art, Science, and Technology. Amsterdam:IOS Press; 1998. p 117–123.

19. Bruffee KA. Making the most of knowledgeablepeers. Change 1994:39.

20. Burt DE. Virtual reality in anaesthesia. Br J Anaest1995;75:472–480.

21. Chalabian J, Garman K, Wallace P, Dunnington G.Clinical breast evaluation skills of house officers andstudents. Am Surg 1996;62:840–845.

22. Chang G, Cook D, Maguire T, Skakun E, YakimetsWW, Warnock GL. Problem-based learning: its rolein undergraduate surgical education. Can J Surg1995;38:13–21.

23. Chopra V, Gesink BJ, de Jong J, Bovill JG, SpierdijkJ, Brand R. Does training on an anaesthesia simulatorlead to improvement in performance? Br J Anaest1994;73:293–297.

24. Coe NP, Hirvela E, Garb JL, Friedmann P. Surgicaleducation: a decade of change. Current Surg 1990;47:317–321.

25. Companion MA, Tarr RW, Jacobs JW, Moshell JM.Collaborative learning: Migration of distributed in-teractive simulation to educational applications. In:Clarke TL, editor: Distributed Interactive SimulationSystems for Simulation and Training in the Aero-space Environment. Bellingham, WA: SPIE OpticalEngineering Press; 1995. p 136–148.

26. Copeland EM 3rd, Flynn TC, Ross WE. Impact ofmanaged care on one training program. University ofFlorida at Gainesville. Arch Surg 1995;130:930–931.

27. Cotin S, Dawson S, Meglan D, et al. ICTS, AnInterventional Cardiology Training System. In:Westwood J, Hoffman H, Mogel G, Robb R, Stred-ney D, editors: Medicine Meets Virtual Reality 2000.Envisioning Healing: Interactive Technology and thePatient-Practitioner Dialogue. Amsterdam: IOSPress; 2000. p 59–65.

28. DaRosa DA, Dunnington GL, Sachdeva AK, et al.A model for teaching medical students in an am-bulatory surgery setting. Acad Med 1992;67:S45–S47.

29. De S, Srinivasan MA. Thin walled models for hapticand graphical rendering of soft tissues in surgical


simulations. In: Westwood JD, Hoffman HM, RobbRA, Stredney D, editors: Medicine Meets VirtualReality. Amsterdam: IOS Press; 1999. p 94–99.

30. Debas HT. Impact of the changing economy and newtechnology on surgical practice and education. InvestRadiol 1993;28(Suppl 3):S23.

31. Debas HT. Impact of managed care on funding ofsurgical residencies. Arch Surg 1995;130:929–930.

32. Delp SL, Loan JP, Hoy MG, Zajac FE, Topp EL,Rosen JM. An interactive graphics-based model ofthe lower extremity to study orthopaedic surgicalprocedures. IEEE Trans Biomed Eng 1990;37:757–767.

33. DesCoˆteaux JG, Lecle`re H. Learning surgical tech-nical skills. Can J Surg 1995;38:33–38.

34. Devitt P, Cehic D, Palmer E. Computers in medicaleducation 2. Use of a computer package to supple-ment the clinical experience in a surgical clerkship:an objective evaluation. Aust NZ J Surg 1998;68:428–431.

35. Doty JR, Liddicoat JR, Salomon NW, Greene PS.Surgical education via the Internet: the Cardiotho-racic Surgery Network. Maryland Med J 1998;47:264–266.

36. Downing SM, English DC, Dean RE. Resident rat-ings of surgical faculty. Improved teaching effective-ness through feedback. Am Surg 1983;49:329–332.

37. Downs SM, Friedman CP, Marasigan F, Gartner G.A decision analytic method for scoring performanceon computer-based patient simulations. In: MasysDR, editor: The Emergence of “Internetable” HealthCare: Systems That Really Work. Proceedings of the1997 AMIA Annual Fall Symposium (formerlySCAMC). Philadelphia: Hanley & Belfus; 1997.

38. Drew PJ, Cule N, Gough M, et al. Optimal educationtechniques for basic surgical trainees: lessons fromeducation theory. J Royal Coll Surg Edinburgh 1999;44:55–56.

39. Dunnington G, Reisner L, Witzke D, Fulginiti J.Structured single-observer methods of evaluation forthe assessment of ward performance on the surgicalclerkship. Am J Surg 1990;159:423–426.

40. Dunnington G, Witzke D, Rubeck R, Beck A, MohrJ, Putnam C. A comparison of the teaching effective-ness of the didactic lecture and the problem-orientedsmall group session: a prospective study. Surgery1987;102:291–296.

41. Dunnington GL, DaRosa DA. Changing surgical ed-ucation strategies in an environment of changinghealth care delivery systems. World J Surg 1994;18:734–737; discussion 733.

42. El-Khalili N, Brodlie K, Kessel D. Wester. A web-based surgical training system. In: Westwood J,Hoffman H, Mogel G, Robb R, Stredney D, editors:Medicine Meets Virtual Reality 2000. EnvisioningHealing: Interactive Technology and the Patient-Practitioner Dialogue. Amsterdam: IOS Press; 2000.p 69–75.

43. Endean ED, Sloan DA, Veldenz HC, Donnelly MB,Schwarcz TH. Performance of the vascular physicalexamination by residents and medical students. JVasc Surg 1994;19:149–154; discussion 155–156.

44. Ericsson K, Krampe R, Tesch-Ro¨mer C. The role ofdeliberate practice in the acquisition of expert per-formance. Psychol Rev 1993;100:363–406.

45. Faulkner H, Regehr G, Martin J, Reznick R. Valida-tion of an objective structured assessment of techni-cal skill for surgical residents. Acad Med 1996;71:1363–1365.

46. Ferenchick G, Simpson D, Blackman J, DaRosa D,Dunnington G. Strategies for efficient and effectiveteaching in the ambulatory care setting. Acad Med1997;72:277–280.

47. Fletcher-Flinn CM, Gravatt B. The efficacy of com-puter assisted instruction (CAI): a meta-analysis. JEducational Computing Res 1995;12:219–242.

48. Flexner A. Medical Education in the United Statesand Canada. New York: Carnegie Foundation; 1910.

49. Folse JR. Surgical education — addressing the chal-lenges of change. Surgery 1996;120:575–579.

50. Folse R. Conflicts in accreditation of postgraduatesurgical education. Current Surg 1986;43:368–372.

51. Folse R. Surgeons as educators — College aims toimprove teaching skills. ACS Bulletin. 1993;78:31–33.

52. Folse RJ, Andriole DP, Debas HT, et al. Prerequisiteobjectives for graduate surgical education: a study ofthe Graduate Medical Education Committee Ameri-can College of Surgeons. J Am Coll Surg 1998;186:50–62.

53. Friedl R, Preisack M, Schefer M, et al. CardioOp: Anintegrated approach to teleteaching in cardiac sur-gery. In: Westwood J, Hoffman H, Mogel G, RobbR, Stredney D, editors: Medicine Meets Virtual Re-ality 2000. Envisioning Healing: Interactive Tech-nology and the Patient-Practitioner Dialogue. Am-sterdam: IOS Press; 2000. p 76–82.

54. Gibbons RD, Baker RJ, Skinner DB. Field articula-tion testing: a predictor of technical skills in surgicalresidents. J Surg Res 1986;41:53–57.

55. Goldiez BF. History of networked simulations. In:Clarke TL, editor: Distributed Interactive SimulationSystems for Simulation and Training in the Aero-space Environment. Bellingham, WA: SPIE OpticalEngineering Press; 1995. p 39–58.

56. Gorey TF. Training in minimally invasive therapyand its impact on traditional surgical education [ed-itorial]. Irish J Med Sci 1997;166:1–2.

57. Gorman PJ, Murray WB, Bholat OS, Henry J,Schelper D, Krummel TM. Crisis resource manage-ment training: optimizing surgeon performance un-der stress. Association for Surgical Education, Nine-teenth Annual Meeting. Boston, MA; 1999:39.

58. Gorman PJ, Lieser JD, Murray WB, Haluck RS,Krummel TM. Evaluation of skill acquisition using aforce feedback, virtual reality based surgical trainer.


In: Westwood JD, Hoffman HM, Robb RA, StredneyD, editors: Medicine Meets Virtual Reality. Amster-dam: IOS Press; 1999. p 121–123.

59. Graham EA. Graduate training for surgery from theviewpoint of the American Board of Surgery. Bulle-tin Am Coll Surg 1938;23:33–34.

60. Greenfield LG. Support of graduate medical educa-tion. Current Surg 1986;43:271.

61. Griffen WO Jr. Surgical residency: on-the-job train-ing or education? Am J Surg 1980;140:720–723.

62. Grillo HC. To impart this art: the development ofgraduate surgical education in the United States. Sur-gery 1999;125:1–14.

63. Grillo HC. To impart this art: the development ofgraduate surgical education in the United States [seecomments]. Surgery 1999;125:1–14.

64. Grosfeld JL. Presidential address. Visions: medicaleducation and surgical training in evolution. ArchSurg 1999;134:590–598.

65. Harris CJ, Herbert M, Steele RJ. Psychomotor skillsof surgical trainees compared with those of differentmedical specialists. Br J Surg 1994;81:382–383.

66. Heppell J, Beauchamp G, Chollet A. Ten-year expe-rience with a basic technical skills and perioperativemanagement workshop for first-year residents. CanJ Surg 1995;38:27–32.

67. Herrmann JB. A self-education program for generalsurgical residents. Current Surg 1987;44:93–96.

68. Hoffman H, Murray M. Anatomic VisualizeR: Real-izing the vision of a VR-based learning environment.In: Westwood JD, Hoffman HM, Robb RA, StredneyD, editors: Medicine Meets Virtual Reality. Amster-dam: IOS Press; 1999. p 134–140.

69. Hoffman H, Vu D. Virtual reality: teaching tool ofthe twenty-first century? Acad Med 1997;72:1076–1081.

70. Hoffman HM. Virtual reality meets medical educa-tion. In: Satava RM, Morgan K, Sieburg H, MattheusR, Christensen J, editors: Interactive Technology andthe New Paradigm for Healthcare. Amsterdam: IOSPress and Ohmsha; 1995. p 130–136.

71. Howard S, Gaba D, Fish K, Yang G, Sarnquist F.Anesthesia crisis resource management training:teaching anesthesiologists to handle critical inci-dents. Aviation, Space, and Environmental Medicine1992;63:763–770.

72. Issenberg SB, McGaghie WC, Hart IR, et al. Simu-lation technology for health care professional skillstraining and assessment. JAMA 1999;282:861–866.

73. Jacobsen DC. The pursuit of excellence in graduatesurgical education. Visions of the Arizona experi-ence. Am J Surg 1980;139:673–676.

74. John N, Phillips N. Surgical simulators using theWWW. In: Westwood J, Hoffman H, Mogel G, RobbR, Stredney D, editors. Medicine Meets Virtual Re-ality 2000. Envisioning Healing: Interactive Tech-nology and the Patient-Practitioner Dialogue. Am-sterdam: IOS Press; 2000. p 146–152.

75. Johnson WL, Rickel J, Stiles R, Munro A. Integrat-ing pedagogical agents into virtual environments.Presence 1998;7:523–546.

76. Johnston R, Bhoyrul S, Way L, et al. Assessing avirtual reality surgical skills simulator. Studies inHealth Technology and Informatics 1996;29:608–617.

77. Jones K. Simulations : a handbook for teachers.London-New York: Kogan Page – Nichols Pub Co;1980. p 125.

78. Kay CL, Evangelou HA. A review of the technicaland clinical aspects of virtual endoscopy. Endoscopy1996;28:768–775.

79. Knowles MS. Andragogy in action. The Jossey-Bassmanagement series. 1st edn. San Francisco: Jossey-Bass; 1984. p 9.

80. Krummel TM. High-tech training tools available.Bulletin Am Coll Surg 1998;83:44–45.

81. Larson CB. History and organization of the tradi-tional orthopaedic residency. Clin Orthop Rel Res1971;75:17–21.

82. Macala JC. Sponsored experiential programs—Learning by doing in the workplace. In: Lewis LH,editor: Experiential and Simulation Techniques forTeaching Adults. San Francisco: Jossey-Bass Inc.;1986. p 57–70.

83. Marescaux J, Cle´ment JM, Tassetti V, et al. Virtualreality applied to hepatic surgery simulation: the nextrevolution. Ann Surg 1998;228:627–634.

84. Martin JA, Regehr G, Reznick R, et al. Objectivestructured assessment of technical skill (OSATS) forsurgical residents. Br J Surg 1997;84:273–278.

85. McGregor DB, Arcomano TR, Bjerke HS, Little AG.Problem orientation is a new approach to surgicaleducation. Am J Surg 1995;170:656–658; discus-sion 658–659.

86. McLellan BA. Early experience with simulatedtrauma resuscitation [see comments]. Can J Surg1999;42:205–210.

87. Mennin SP, Kalishman S. Student assessment. AcadMed 1998;73:S46–S54.

88. Oneal RM, Dingman RO, Grabb WC. The teachingof plastic surgical techniques to medical students.Plastic Reconstr Surg 1967;40:494–498.

89. O’Toole R, Playter R, Blank W, Cornelius N, To-berts W, Raibert M. A novel virtual reality surgicaltrainer with force feedback: surgeon vs. medical stu-dent performance. In: Salisbury JK, Srinivasan MA,editors. Proceedings of the Second Phantom UsersGroup Workshop. Dedham, MA: MIT; 1997.

90. O’Toole RV, Playter RR, Krummel TM, et al. Mea-suring and developing suturing technique with a vir-tual reality surgical simulator [see comments]. J AmColl Surg 1999;189:114–127.

91. Pelletier MP. Undergraduate surgical education inthe twenty-first century. Can J Surg 1995;38:42–44.

92. Provan JL. Surgical education: time for a change?[editorial; comment]. Can J Surg 1995;38:8–9.


93. Prystowsky JB, Regehr G, Rogers DA, Loan JP,Hiemenz LL, Smith KM. A virtual reality module forintravenous catheter placement. Am J Surg 1999;177:171–175.

94. Regian JW, Shebilske W, Monk J. A preliminaryempirical evaluation of virtual reality as an instruc-tional medium for visual-spatial tasks. J Communi-cation 1992;42:136–149.

95. Reznick R, Regehr G, MacRae H, Martin J, McCul-loch W. Testing technical skill via an innovative“bench station” examination. Am J Surg 1997;173:226–230.

96. Reznick RK. Teaching and testing technical skills.Am J Surg 1993;165:358–361.

97. Ritchie WP Jr. Graduate surgical education in the eraof managed care: a statement from the AmericanBoard of Surgery [editorial]. J Am Coll Surg 1997;184:311–2.

98. Rosen JM, Soltanian H, Redett RJ, Laub DR. Evo-lution of virtual reality. IEEE Eng Med Biol 1996:16–21.

99. Rothbaum BO, Hodges LF, Kooper R, Opdyke D,Williford JS, North M. Effectiveness of computer-generated (virtual reality) graded exposure in thetreatment of acrophobia. Am J Psychiatry 1995;152:626–628.

100. Sachdeva AK, Blair PG, Kelliher GJ, et al. Rede-signing the surgery clerkship at MCP HahnemannSchool of Medicine to address the educational needsof generalists. Acad Med 1999;74:S98–S101.

101. Sachdeva AK, Loiacono LA, Amiel GE, Blair PG,Friedman M, Roslyn JJ. Variability in the clinicalskills of residents entering training programs in sur-gery. Surgery 1995;118:300–308; discussion 308–309.

102. Satava RM. Medical applications of virtual reality.J Med Systems 1995;19:275–280.

103. Satava RM. Virtual reality and telepresence for mil-itary medicine. Comp Biol Med 1995;25:229–236.

104. Satava RM. Virtual reality, telesurgery, and the newworld order of medicine. J Image Guided Surg 1995;1:12–16.

105. Satava RM. Advanced simulation technologies forsurgical education. ACS Bulletin. 1996;81:77–81.

106. Satava RM. Cybersurgery: A new vision for generalsurgery. In: Satava RM, editor. Cybersurgery: Ad-vanced Technologies for Surgical Practice. NewYork: John Wiley and Sons, Inc.; 1998. p 3–14.

107. Satava RM, Jones SB. Virtual environments for med-ical training and education. Presence 1996;6:139–146.

108. Sawyers JL. Presidential address. Graduate surgicaleducation. Am Surg 1981;47:1–5.

109. Schirmer WJ, Galat JA, Morris JB, Persons ML,Shuck JM. The impact of a resident-directed surveyon the education curriculum of a university surgicaltraining program. Surgery 1991;110:405–410.

110. Schroeder R. Virtual reality in the real world: history,applications and projections. Futures 1993:963–972.

111. Schueneman AL, Pickleman J, Freeark RJ. Age, gen-der, lateral dominance, and prediction of operativeskill among general surgery residents. Surgery 1985;98:506–515.

112. Schueneman AL, Pickleman J, Hesslein R, FreearkRJ. Neuropsychologic predictors of operative skillamong general surgery residents. Surgery 1984;96:288–295.

113. Schwartz RW, Donnelly MB, Nash PP, Young B.Developing students’ cognitive skills in a problem-based surgery clerkship. Acad Med 1992;67:694–696.

114. Schwid HA. Anesthesia and critical care simulators.Curr Op Anaesthesiol 1995;8:532–535.

115. Sloan DA, Donnelly MB, Johnson SB, SchwartzRW, Strodel WE. Use of an Objective StructuredClinical Examination (OSCE) to measure improve-ment in clinical competence during the surgical in-ternship [see comments]. Surgery 1993;114:343–350; discussion 350–351.

116. Sloan DA, Donnelly MB, Schwartz RW, StrodelWE. The Objective Structured Clinical Examination.The new gold standard for evaluating postgraduateclinical performance. Ann Surg 1995;222:735–742.

117. Squire D, Giachino AA, Profitt AW, Heaney C.Objective comparison of manual dexterity in physi-cians and surgeons. Can J Surg 1989;32:467–470.

118. Steele RJ, Walder C, Herbert M. Psychomotor test-ing and the ability to perform an anastomosis injunior surgical trainees. Br J Surg 1992;79:1065–1067.

119. Stotter AT, Becket AJ, Hansen JP, Capperauld I,Dudley HA. Simulation in surgical training usingfreeze dried material. Br J Surg 1986;73:52–54.

120. Strodel WE, Zelenock GB. Teaching and learning inthe operating room. In: Bartlett RH, Strodel WE,Zelenock GB, Harper ML, Turcotte JG, editors:Medical Education: A Surgical Perspective. Chelsea,MI: Lewis Publishers; 1986. p 47–51.

121. Taffinder N, Sutton C, Fishwick R, McManus IC,Darzi A. Validation of virtual reality to teach andassess psychomotor skills in laparoscopic surgery:Results from randomised controlled studies using theMIST VR laparoscopic simulator. In: Westwood JD,Hoffman HM, Stredney D, Weghorst SJ, editors:Medicine Meets Virtual Reality. Amsterdam: IOSPress; 1998. p 124–130.

122. Taffinder N, Sutton C, Fishwick RJ, McManus IC,Darzi A. Validation of virtual reality to teach andassess psychomotor skills in laparoscopic surgery:results from randomised controlled studies using theMIST VR laparoscopic simulator. Studies in HealthTechnology and Informatics 1998;50:124–130.

123. Tendick F, Downes M, Cavusoglu CM, Gantert W,Way LW. Development of virtual environments fortraining skills and reducing errors in laparoscopic


surgery. In: Bogner MS, Charles ST, Grundfest WS,et al., editors: Proceedings of Surgical-Assist Sys-tems. Bellingham, WA: SPIE; 1998:36–44.

124. Thompson JC. Impact of Managed Care on SurgicalEducation and Research. Boston, Massachusetts, Oc-tober 20, 1994. Proceedings. Arch Surg 1995;130:925–941.

125. Tseng CS, Lee YY, Chan YP, Wu SS, Chiu AW. APC-based surgical simulator for laparoscopic sur-gery. In: Westwood JD, Hoffman HM, Stredney D,Weghorst SJ, editors: Medicine Meets Virtual Real-ity: Art, Science, Technology. Amsterdam: IOSPress; 1998. p 155–160.

126. Tuggy ML. Virtual reality flexible sigmoidoscopysimulator training: impact on resident performance.J Am Board Family Pract 1998;11:426–433.

127. Vince J. Virtual reality systems. SIGGRAPH series.Wokingham, England — Reading, MA: Addison-Wesley Pub Co; 1995. pp xv, 388, and [8] of plates.

128. Vining DJ, Liu K, Choplin RH, Haponik EF. Virtualbronchoscopy. Relationships of virtual reality endo-bronchial simulations to actual bronchoscopic find-ings. Chest 1996;109:549–553.

129. Walt AJ. The uniqueness of American surgical edu-cation and its preservation. Bulletin Am Coll Surg1994;79:8–20.

130. Watson DC, Matthews HR. Manual skills of traineesurgeons. J Royal Coll Surg Edinburgh 1987;32:74–75.

131. Weghorst S, Airola C, Oppenheimer P, et al. Vali-dation of the Madigan ESS Simulator. In: WestwoodJD, Hoffman HM, Stredney D, Weghorst SJ, editors:Medicine Meets Virtual Reality. Amsterdam: IOSPress; 1998. p 399–405.

132. Weidenbach K. Staged encounters. Stanford Medi-cine 1999:8–12.

133. Weigelt J, Brasel K, Olson C, Thal E. Opinions ofpracticing general surgeons on surgical education.Am J Surg 1998;176:481–485.

134. Winckel CP, Reznick RK, Cohen R, Taylor B. Reli-ability and construct validity of a structured technicalskills assessment form. Am J Surg 1994;167:423–427.

135. York NL, DaRosa DA, Folse R. The learning needsof first-year surgical residents in the intensive careunit. Am J Surg 1996;171:608–611.

136. York NL, Niehaus AH, Markwell SJ, Folse JR. Eval-uation of students’ physical examination skills duringtheir surgery clerkship. Am J Surg 1999;177:240–243.

137. Zuidema GD, Lound JL. History, current concepts,and surgical manpower. In: Bartlett RH, Strodel WE,Zelenock GB, Harper ML, Turcotte JG, editors.Medical education: a surgical perspective. Chelsea,MI: Lewis Publishers; 1986:247–258.


Documents

Computer-assisted training and learning in surgery