SourcesSecretarW81XW

Accepte

ReprintGrant Spurdue.

0039-60

� 2016

http://d

1646

Medical telementoring usingan augmented reality transparentdisplay

Daniel Andersen,a Voicu Popescu, PhD,a Maria Eugenia Cabrera, MSc,b Aditya Shanghavi,b

Gerardo Gomez, MD, FACS,c Sherri Marley, BSN, RN,c Brian Mullis, MD,c and Juan P. Wachs, PhD,b

West Lafayette and Indianapolis, IN

Background. The goal of this study was to design and implement a novel surgical telementoring systemcalled the System for Telementoring with Augmented Reality (STAR) that uses a virtual transparentdisplay to convey precise locations in the operating field to a trainee surgeon. This system was comparedwith a conventional system based on a telestrator for surgical instruction.Methods. A telementoring system was developed and evaluated in a study which used a 1 3 2 between-subjects design with telementoring system, that is, STAR or conventional, as the independent variable. Theparticipants in the study were 20 premedical or medical students who had no prior experience withtelementoring. Each participant completed a task of port placement and a task of abdominal incision undertelementoring using either the STAR or the conventional system. The metrics used to test performance whenusing the system were placement error, number of focus shifts, and time to task completion.Results. When compared with the conventional system, participants using STAR completed the 2 taskswith less placement error (45% and 68%) and with fewer focus shifts (86% and 44%), but moreslowly (19% for each task).Conclusions. Using STAR resulted in decreased annotation placement error, fewer focus shifts, butgreater times to task completion. STAR placed virtual annotations directly onto the trainee surgeon’sfield of view of the operating field by conveying location with great accuracy; this technology helped toavoid shifts in focus, decreased depth perception, and enabled fine-tuning execution of the task to matchtelementored instruction, but led to greater times to task completion. (Surgery 2016;159:1646-53.)

From the Departments of Computer Sciencea and Industrial Engineering,b Purdue University, West Lafayette;and the School of Medicine,c Indiana University, Indianapolis, IN

BECAUSE MANY SURGEONS remain less well-trainedoutside their subspecialization,1 some surgeonsmay be less well-prepared to treat trauma caseseffectively in austere environments, such as thoseoccurring in the military. A broad range of exper-tise for optimal treatment is required that is notreadily available at the point of injury or at therequired time.2 Traumatic injuries suffered in com-bat that require immediate care must becompleted in forward operating bases lackingdirect access to a wide array of specialist surgeons.Similarly, general surgeons in rural areas are

of Funding: Supported by the Office of the Assistanty of Defense for Health Affairs under Award No.H-14-1-0042.

d for publication December 19, 2015.

requests: Juan P. Wachs, PhD, Purdue University, 315 N.treet, West Lafayette, IN 47907. E-mail: jpwachs@edu.

60/$ - see front matter

Elsevier Inc. All rights reserved.

x.doi.org/10.1016/j.surg.2015.12.016

SURGERY

disadvantaged when required to perform surgicalprocedures requiring expertise beyond what isavailable at the local hospital. In such situations,transporting the patient to a major medical facilitymay not be feasible. The prospect of using infor-mation technology, computer graphics, and com-puter vision to provide remote instruction is apossible solution. In this paper, we discuss our im-plemented surgical telementoring system.

Telementoring, the use of information technol-ogy to provide real-time, remote guidance to atrainee surgeon from an expert mentor surgeon,can address these issues. Telementoring has shownthe following benefits. By introducing new surgicaltechniques to remotely located trainees, telemen-toring has been used successfully to provideguidance to surgeons in the completion of pro-cedures that were initially unfamiliar to them,3 andby providing assistance closer to the point of injury,transportation-related delays that can affect the pa-tient adversely can be avoided.4

Existing surgical telementoring systems rely ontelestrators used by the mentor to overlay graphic

SurgeryVolume 159, Number 6

Andersen et al 1647

or textual annotations onto imagery of the surgicalenvironment. These images are displayed to thetrainee typically on a nearby computer monitor.Budrionis et al5 suggested that telestration is a corefeature of telementoring solutions and that aresearch gap exists regarding the impact and de-tails of telestration itself.

The conventional telestrator-based approachhas a substantial disadvantage; trainees must shiftfocus repeatedly between the operating field andthe telestrator. When a mentor is co-locatedphysically with the trainee, the mentor can usehands or surgical instruments to indicate locationsdirectly within the operating field. For example,hand gestures were used to interact with MRIimages in the operating room in our previouswork.6,7 When using a telestrator, however, focusshifting from the operating field to some othermonitor (like a computer screen) adds additionalindirection. The trainee must look at the telestra-tor, understand the provided guidance, then lookback at the operating field and map the instruc-tions mentally onto the real-world scene.3 Thisindirection creates additional cognitive load, risk-ing decreases in surgical performance.

Recent research into surgical telementoring hasemphasized the importance of reevaluating therole of the telestrator-based approach. Ereso et al2

developed a telementoring system using atelestrator-based video connection with mentor-controlled pan/tilt functionality, but most notablyusing a laser pointer that the mentor could controlremotely to point virtually at areas of interestdirectly on the operating field. Satisfaction surveyscompleted afterward indicated that although thetelestrator was unnecessary for several trainees,the ability of the mentor to indicate areas directlyin the trainee’s field of view was useful.3 Otherresearch has focused on enhancing the ability ofthe mentor to demonstrate proper actions of thetrainee. Vera et al8 implemented and validatedan augmented reality telementoring platform,which overlaid a live view of the surgical instru-ments manipulated by a remotely located mentoronto the laparoscopic monitor viewed by a traineeto conduct a simulated laparoscopy. This approachshowed the effectiveness of overlaying mentorguidance directly onto the trainee’s view of theoperating field; because the trainee normally viewsthe operating field through the laparoscopicmonitor, there was no active focus shifting.8

Another approach to augmented reality telemen-toring called Virtual Interactive Presence andAugmented Reality used a set of binocular video-scopes through which both trainee and mentor

could view the operating field augmented withmentor-provided overlays. This system allowed amentor to “see what the local surgeon sees,” andwas used successfully while performing a carotidendarterectomy and pterional craniotomy; a majordisadvantage, however, was that the bulky eyewearof the apparatus forced the trainee to operatefrom a fixed, rigid location.9 Augestad et al10 re-viewed recently the state of surgical telementoringresearch and determined that the impact of tele-mentoring on surgical training has improvedover time, but emerging technologies related toportable and mobile devices, such as tablets orother smaller, less cumbersome devices, remainan open area of research.

We designed and developed a surgical tele-mentoring system called the System for Telemen-toring with Augmented Reality (STAR) based onan augmented reality display that simulates atransparent effect.11 This work was begun byLoescher et al,12 who investigated an augmentedreality approach to surgical telementoring, display-ing annotations on a tablet suspended between thetrainee and the operating field. Instead of display-ing mentor annotations on a screen that waslocated separate from the operating area, the cur-rent STAR system places annotations directly intothe field of view of the trainee surgeon (Fig 1,top). As a result, the trainee surgeon does notneed to shift eye focus to get information from aremote mentor. In addition, because the annota-tions are overlaid directly onto the trainee’s viewof the operating area, the trainee does not needto remap annotations mentally to the currentcontext. This system was developed using forma-tive feedback provided by surgeons during anadvanced trauma operative management course(Fig 1, bottom). The purpose of the current studywas to evaluate the effectiveness of the novel surgi-cal telementoring system, STAR, when comparedwith the conventional telestrator approach.

SYSTEM DESCRIPTION

STAR provides the trainee with an augmented“window” through which the trainee views theoperating field while completing the operation.The following discussion describes the design andarchitecture of STAR and the subsystems (thetrainee subsystem and the mentor subsystem)through which the trainee and mentor interactto complete a telementored operation.

Fig 1 shows the trainee subsystem. The tabletcaptures and displays live video of the operatingfield using its camera. When the trainee surgeonconducts a telementored operation, this trainee

Fig 1. Top, The trainee subsystem for the system for tele-mentoring with augmented reality (STAR). An incisionline annotation (dark blue line) created by a remotementor is displayed on the tablet’s screen. Bottom, A pro-totype of the STAR trainee subsystem being tested by atrainee surgeon during an advanced trauma operativemanagement course. (Color illustration of figure is avail-able online.)

SurgeryJune 2016

1648 Andersen et al

tablet is held in between the operating fieldand the trainee’s head by a manipulator. This posi-tion allows the trainee to view the operating fieldand his or her hands by looking “through” thetablet while conducting a telementored procedure.The trainee tablet is networked wirelessly with aremote mentor subsystem. As the trainee subsys-tem captures live video of the operating field, ittransmits video frames to the mentor subsystem.

The mentor uses a tablet subsystem to view theoperating field remotely and to place graphicannotations. The mentor subsystem displays thelive video provided by the trainee subsystem,allowing the mentor to view the operating arearemotely. The mentor subsystem has a touch-baseduser interface, which lets the mentor create,delete, and modify annotations overlaid onto animage frame from the trainee subsystem. Sup-ported types of annotations include points, lines,loops, semitransparent icons of surgical instru-ments and hand gestures, and text labels. Using

multitouch controls, these annotations can berescaled, rotated, and repositioned.

Once annotations are created, the mentor sub-system transmits them to the trainee subsystem.When the trainee subsystem receives the annota-tions, with a delay of only about 100 ms, theannotations are displayed on screen directly inthe trainee’s view of the operating area. By meansof computer vision algorithms, these annotationsare anchored to relevant areas of the operatingfield, avoiding drift as the trainee tablet is reposi-tioned or as the operating field changes.

METHODS

Our study was conducted with 17 premedical and3 medical students at Purdue University. Eachparticipant completed 2 simulated tasks: placementof an incision for port placement and an abdominalincision. In each task, participants acted as a traineein a telementoring scenariowith a simulatedmentorproviding incremental graphic annotations as theparticipant completed each stage of the task. Tenstudents used the STAR surgical telementoringsystem, and the other 10 students used a conven-tional, telestrator-based system. For each task, wemeasured placement error, the number of focusshifts, and completion time.

Participants. Twenty students (ages 18–26) wererecruited from the premedical and medical pro-grams at Purdue University. No students had priorexperience with either STAR or surgical telemen-toring in general. The study was approved by thePurdue University Institutional Review Board, andwritten participant consent was acquired beforebeginning the experiment.

STAR condition. For the STAR condition, par-ticipants completed each task while using the STARtrainee subsystem. To ensure consistency, a simu-lated mentor controlled by the experimenters wasused to provide annotations to participants. Allannotations used during the experiment werepregenerated, and as each stage of each task wascompleted, the simulated mentor provided theannotation demonstrating the next step. Eachannotation was sent to the trainee subsystem, whereit was visible on screen for the participant to follow.

Conventional condition. For the conventionalcondition, a simulated telestrator interface wasdeveloped. Before the experiment, reference im-ages of the operating field were captured from thepoint of view of the trainee. These images weredisplayed on a 46-inch LCD monitor and placedon the other side of the patient simulator visible tothe participant. Images were advanced sequentiallyusing a keyboard user interface.

Fig 2. Top, The port placement task for the system fortelementoring with augmented reality (STAR) condi-tion. Bottom, Screenshot captured from the trainee tabletduring the port placement task. Mentor-provided anno-tations (blue circles) indicate locations for the trainee tomark (green adhesives). (Color illustration of figure isavailable online.)

Fig 3. The abdominal incision task for the system for tel-ementoring with augmented reality (STAR) condition.Annotations (blue circles and transparent clamp icons)show where the trainee should place the clamps. (Colorillustration of figure is available online.)

SurgeryVolume 159, Number 6

Andersen et al 1649

Tasks. Participants used either STAR or theconventional telestrator-based system (asdescribed) to complete 2 telementored tasks.Participants completed each task by interactingwith a patient simulator (developed in house)placed on a table.

Port placement. The first task evaluated how well atrainee can find locations in the human body forthe purpose of surgical port placement (Fig 2, top).These locations were indicated via a simulatedmentor, and each trainee was required to assignphysical identifiers to these regions using small ad-hesives. Participants were provided small circularadhesives (6.4 mm in diameter) to place over thepatient simulator, representing the location of sur-gical ports. Participants were given a verbal descrip-tion of the task and instructed to complete itquickly but accurately. During the task, the mentorwould annotate 7 locations one at a time on an im-age of the neck of the patient simulator. Partici-pants were tasked with placing an adhesive oneach corresponding physical location on the pa-tient simulator. For each participant, this task wasrepeated for a total of 3 trials. To avoid task memo-rization, subsequent trials displayed differing

sequences and positions of circle annotations. Aftereach trial, any placed adhesives were removed.

For the STAR condition, participants viewed thementor annotations using the STAR trainee tablet.For all participants, the trainee tablet was posi-tioned at an identical location, pointing the tabletcamera at the neck area of the patient simulator.Annotations appeared overlaid onto the live video(Fig 2, bottom). After each trial, an image wascaptured from the camera on the trainee tabletto record the placed locations of the adhesivesfrom a reference position.

For the conventional condition, participantsreceived mentor guidance by viewing the nearbyLCD monitor. The monitor displayed a referenceimage of the neck area of the patient simulator,overlaid with annotations displayed one at a time.After completing each trial of this task, werecorded an image of the adhesives as placed bythe participant.

Abdominal incision task. The second task consistedof performing an abdominal incision and exposureusing the same patient simulator (developed inhouse) described earlier (Fig 3). Participants werealso provided with a scalpel, retractor, surgical scis-sors, 4 clamps, and a felt-tipped marker. Beforethe task, each participant was shown a 4-minutevideo demonstrating the task to be completed. Inthe task, the surgeon used a scalpel and scissors tomake an incision and used the retractor and clampsto spread the linea alba. Before each incision andeach use of the retractor or a clamps, the traineedrew either an incision line or a circle at the pointof contact using the felt-tipped marker.

Next, the surgical instruments were placed on atray near the incision simulator, and the partici-pant was tasked with repeating the previously

SurgeryJune 2016

1650 Andersen et al

viewed procedure on the actual incision simulator,step by step as mentor-provided annotations weredisplayed to the participant. This procedure wasrepeated for a total of 2 trials by each participant.

For the STAR condition, participants viewed thesimulated operating field by viewing the traineetablet screen held in a fixed pose over theabdominal area of the incision simulator. As theparticipant completed each step of the procedure,pregenerated annotations were transmitted by thesimulated mentor to the trainee tablet and dis-played. These annotations showed the location ofthe incision and surgical instruments. For theconventional condition, participants receivedmentor guidance by looking at the nearby LCDmonitor, which displayed reference images of theincision simulator overlaid with annotations dis-played one at a time.

After the experiment. After completing all trials ofeach task under either the STAR condition or theconventional condition, participants were asked tofill out a form indicating their age and theircurrent year in their premedical or medicalstudies. Participants received $10 as compensationfor their participation.

Study design. Owing to the specific repeatednature of each task (especially the abdominalincision task), we used a between-subjects studydesign. The type of surgical telementoring systemwas the independent factor; each participant usedeither the conventional system or STAR. Thepremedical and medical population was com-bined, because their knowledge of the procedurewas the same (no previous experience).

Dependent variables. For this study, the dependentvariables were placement error, the number offocus shifts, and the time for completion of thetask.

Placement error: To determine the accuracy of sur-gical tool placement or incisions made, wemeasured the distance between marked locationsin the image provided to the participants asguidance and the actual locations marked byparticipants during each trial. The marked loca-tions included the locations of adhesives, surgicalinstruments, and incision lines made during thetask. This distance was the placement errormeasured in pixels on images captured from thesame position relative to the operating field.

Focus shifts: We measured the number of times aparticipant shifted focus while completing a singletrial. A focus shift was defined as a period of time

when the participant was not looking directly atthe operating field.

Task completion time: For each trial, we recorded thetime for completion of the task, starting when averbal indication was given to start and endingwhen the participant had completed the requestedtask and was no longer touching either the patientsimulator or any attached surgical instruments.

Data collection. To allow counting the focusshifts, participants wore a Google Glass head-mounted display (created by Google, Menlo Park,CA) while completing each trial. The on-boardcamera on the Google Glass captured a videorecording of the head movements of the partici-pants. After the experiment, we analyzed theserecordings and counted the focus shifts.

To measure placement error, images were re-corded of the operating field for each task andoverlaid onto reference images taken previouslyand used when generating the annotations. Anautomatic process was used to correspond theimages to compensate for minor variation in theposition of either the tablet or the patient simu-lator. After the images were overlaid, pixel dis-tances were recorded between marked locations inthe reference images and in the experimentalimages.

Statistical analysis. For all the metricsmeasured, we wanted to compare 2 sampledpopulations based on the tested conditions fortelementoring systems: a conventional approachand STAR. The responses used were the averageof each metric for the 10 subjects in each condi-tion; in the first task, each subject performed theplacement task 3 times giving a total of 30 samplesper condition. In the second task, the subjectsperformed the abdominal incision twice resultingin 20 samples for each condition. The firstdiagnostic to compare the populations was tocheck the normality assumption, using theShapiro-Wilks test; when the test provided resultspointing to non-normality of the collected datafor a given metric, the nonparametric test Wilcoxsign-rank was used to compare the medians andspreads of the responses from both conditions. Inthe cases where the Shapiro-Wilks test supportedthe normality assumption, a t test was conductedusing Satterthwaite condition for nonequal vari-ance. Continuous data were summarized for theresults section as mean (m) ± standard deviation(sx) for normally distributed data, or asmedian ± interquartile range (IQR) values fordata not normally distributed.

SurgeryVolume 159, Number 6

Andersen et al 1651

RESULTS

Placement error. For the task of port placement,placement error for participants using STAR (me-dian, 23.73; IQR, 13.28 px) and the conventionalsystem (median, 57.55; IQR, 32.80 px) wasdifferent (P < .001) with a median improvementof 33.8 px, representing a 59% decrease. Likewise,there was a 68% improvement (P < .001)decreasing 49.5 px in average the placement errorfor the abdominal incision task, from the conven-tional system (m, 72.6; sx, 16.9 px) to STAR(m, 23.1; sx, 8.4 px).

Focus shifts. For the port placement task, thenumber of focus shifts for participants using STAR(median, 0; IQR, 0.5) was less (P < .0001) than forparticipants using the conventional system (me-dian, 13; IQR, 3.75). For the abdominal incisiontask, the number of focus shifts for participants us-ing STAR (m, 11.61; sx, 10.46) was also less(P = .003) than for participants using the conven-tional system (m, 20.68; sx, 5.78).

Task completion time. For the port placementtask, participants completed the task more slowly(P = .0003) when using STAR (m, 48.0 s; sx, 15.5 s)than when using the conventional system(m, 40.4 s; sx, 16.6 s). For the abdominal incisiontask, no difference was found (P = .165) betweenthe times to task completion for participants usingSTAR (m, 274.9 s; sx, 86.9 s) and for participants us-ing the conventional system (m, 231.1 s; sx, 63.4 s).Both task completion times (port placement andabdominal incision) were normally distributed.

DISCUSSION

Placement error. In each task, participants usingSTAR completed the task with significantly lessplacement error than participants using the con-ventional system. One likely cause for the decreasein placement error is the direct overlay of annota-tions onto the operating field. When using STAR,trainees avoided the cognitive load involved inlooking at a separate image of the operating field,interpreting it, and mapping the separate instruc-tions onto the actual operating field.

Operations rely on a sequence of steps, eachenacted on a previously altered operating field.When using static imagery for telementoring,trainees using a conventional system rely onrelative estimates of location based on their ownprior actions, which can lead to accumulation oferror. For example, a participant placing an adhe-sive at an incorrect location may, when shown alater port location, place the later adhesive inrelation to the previous adhesive location rather

than an absolute position relative to the structuresof the operating field. In contrast, live feedbackusing STAR allowed for absolute reference pointsfor guiding trainee actions, thereby limiting accu-mulation of error.

During the port placement task, some of theadhesives were placed far from the location indi-cated by the annotation, even in the STAR condi-tion (Fig 2, top). Several participants reporteddifficulty in manipulating the adhesives with theirfingers and sticking them to the correct surface,which was a limitation of this task, possibly contrib-uting to the large variances in the data.

Focus shifts. For each task, participants shiftedfocus away from the operating field much lessoften when using STAR than when using theconventional system. This finding was expected,because the conventional system requires thetrainee to look away from the operating field tosee the mentor instruction to complete each task.In contrast, STAR placed annotations directly inthe trainee field of view overlaid onto the oper-ating field, meaning no focus shifts were necessaryfor the trainee to complete the tasks successfully(Figs 1 and 2). In fact, for each task, there wereseveral trials in the 10 subjects tested (6 trials forthe port placement task, and 20 trials for theabdominal incision task), all completed while us-ing STAR, where no focus shifts were made.

In 2 cases, participants using STAR moved theirheads routinely to look under the tablet afterviewing the annotations displayed on the tablet.A third participant using STAR behaved similarlyonly when using the scalpel during the abdominalincision task. One possible cause for this move-ment might have been owing to a lack of trainingor familiarity with STAR, causing participants torevert to a more familiar approach of regardingthe tablet as a reference screen rather than as avirtual window. Another cause could be the lack ofdepth perception on the tablet display, whichdecreases hand–eye coordination during higherrisk actions such as an incision.

Task completion time. Participants using STARcompleted tasks slightly more slowly than whenusing the conventional system. This slowdown maybe secondary to the decrease in placement errornoted in participants using STAR. When using theconventional system, participants lacked live feed-back to evaluate how their current placement ofsurgical instruments correlated with the requestedmentor instruction. As a result, these participantssimply completed each stage of a task relying on aninitial estimate of the correct placement. Incontrast, participants using STAR had the benefit

SurgeryJune 2016

1652 Andersen et al

of real-time annotations being overlaid directlyonto their view of the operating area as theycompleted the task.

Frequently, participants using STAR used asearching or hunting movement with their handsto complete the task. These trainees would placetheir hands or surgical instruments initially intothe tablet’s field of view at an initial estimatedlocation, then correct the position slowly until thesurgical instrument aligned with the providedoverlay.

Other contributing factors may include adecreased level of hand–eye coordination in thecurrent prototype system. One cause could belatency in the display of the operating field bythe tablet. There is some delay (approximately100 ms) between when the tablet camera capturesa video frame and when the video frame is avail-able in memory. In addition, the single camera ofthe STAR system provides only a 2-dimensionalview of the operating field to the trainee, whichlimits the depth perception of the trainee. Inaddition, when the trainee views the operatingfield through the tablet, the image is from thecamera’s perspective and not the trainee’s perspec-tive. As a result, the hands and the surgicalinstruments of the trainee appear unexpectedlymagnified when viewing them through the tablet.

Current limitations and future research. Themain limitation of STAR is that the trainee sub-system is not truly a transparent display; instead, itcreates the impression of a transparent display bydisplaying a live video on the trainee tabletscreen. The tablet displays only a single imageof the operating field from a single camera on thetablet, which affects the trainee’s depth percep-tion. Displaying a 3-dimensional view visible withspecial eyewear is a possible solution, as is usingcustom-made displays that simulate a 3-dimensional effect for a limited field of view,13

although such hardware would currently requirea bulky apparatus to be worn by the trainee.There is also a short latency of about 100 ms inthe video feed, both in displaying captured videoframes to the trainee and in transmitting them toa mentor. Finally, because of the difference be-tween the position of the camera of the tabletand the position of the trainee’s eyes, the videoas captured by the trainee tablet may not alwaysbe from the correct perspective of the trainee;this may cause minor mismatches between thetrainee’s view of the operating field inside thetablet “window,” and the trainee’s real-world viewof the surrounding area. Future research willinvestigate computer vision and 3-dimensional

reconstruction techniques to detect the head po-sition of the trainee and automatically correctfor this, achieving a true “transparent display”effect.

Compared with other telementoring system us-ing virtual reality haptic controlled instruments,14

or those using servos for controlling camerasand/or laser pointers to indicate specific regions,3

the proposed system enables richer and more real-istic annotations, it is versatile, mobile, and lessexpensive than the existing ones.

In conclusion, this study evaluated the ability ofa tablet-based virtual transparent display surgicaltelementoring system (STAR) to facilitate accurateperformance of surgical tasks by a trainee whenguided under remote telementoring. Participantsusing the STAR technology completed tasks withfewer errors and fewer focus shifts. Even thoughparticipants using STAR completed the surgicaltasks slightly more slowly, the decreases in place-ment error suggest that surgical telementoringsystems that provide guidance directly onto thefield of view of the trainee can improve surgicalperformance.

The opinions, interpretations, conclusions, and rec-ommendations are those of the authors and are notnecessarily endorsed by the Department of Defense. Theauthors thank Prof Thomas Kuczek and Jing Lu fromPurdue Statistical Consulting Services for their guidanceand suggestions. They also thank Chun-hao Hsu andAviran Malik for their help with the manipulator systemused to position the trainee tablet used in ourexperiments.

REFERENCES

1. Borman KR, Vick LR, Biester TW, et al. Changing demo-graphics of residents choosing fellowships: longterm datafrom the American Board of Surgery. J Am Coll Surg2008;206:782-8.

2. Ereso AQ, Garcia P, Tseng E, et al. Live transference of sur-gical subspecialty skills using telerobotic proctoring toremote general surgeons. J Am Coll Surg 2010;211:400-11.

3. Treter S, Perrier N, Sosa JA, et al. Telementoring: a multi-institutional experience with the introduction of a novelsurgical approach for adrenalectomy. Ann Surg Oncol2013;20:2754-8.

4. Garcia P. Telemedicine for the battlefield: present andfuture technologies. Surgical Robotics 2011:33-68.

5. Budrionis A, Augestad M, Patel HR, et al. An evaluationframework for defining the contributions of telestrationin surgical telementoring. Interact J Med Res 2013;2:e14.

6. Wachs JP, Stern HI, Edan Y, et al. A gesture-based tool forsterile browsing of radiology images. J Am Med Inform As-soc 2008;15:321-3.

7. Jacob MG, Wachs JP, Parker RA. Hand gesture-based sterileinterface for the operating room using contextual cues forthe navigation of radiological images. J Am Med Inform As-soc 2013;20:e183-6.

SurgeryVolume 159, Number 6

Andersen et al 1653

8. Vera AM, Russo M, Mohsin A, et al. Augmented reality tele-mentoring (ART) platform: a randomized controlled trialto assess the efficacy of a new surgical education technology.Surg Endosc 2014;28:3467-72.

9. ShenaiMB,DillavouM,ShumC, et al. Virtual interactive pres-ence and augmented reality (VIPAR) for remote surgicalassistance. Neurosurgery 2010;68(1 Suppl Operative):200-7.

10. Augestad M, Bellika JG, Budrionis A, et al. Surgical telemen-toring in knowledge translation---clinical outcomes andeducational benefits. Surg Innov 2010;20:273-81.

11. Andersen D, Popescu V, Cabrera ME, et al. Virtualannotations of the surgical field through an

augmented reality transparent display. Vis Comput2015:1-18.

12. Loescher T, Lee SY, Wachs JP. An augmented realityapproach to surgical telementoring. Conf Proc IEEE IntConf Syst Man Cybern 2014:2341–2346.

13. Maeda N, Sugimoto M. Pathfinder vision: tele-operationrobot interface for supporting future prediction usingstored past images. InACM SIGGRAPH 2014 Posters 2014(p. 52). ACM.

14. Allen BF, Jordan B, Pannell W, Lewis C, Dutson E,Faloutsos P. Laparoscopic surgical robot for remotein vivo training. Adv Robot 2010;24:1679-94.