Assessing suturing techniques using a virtual reality surgical simulator

ASSESSING SUTURING TECHNIQUES USING A VIRTUAL REALITYSURGICAL SIMULATOR

HAMED KAZEMI, M.Eng.,1 JAMES K. RAPPEL, M.Eng.,1 TIMOTHY POSTON, Ph.D.,2 BENG HAI LIM, M.B.B.S., M.Med., F.R.C.S.,3

ETIENNE BURDET, Ph.D.,4 and CHEE LEONG TEO, Ph.D.1*

Advantages of virtual-reality simulators surgical skill assessment and training include more training time, no risk to patient, repeatable diffi-culty level, reliable feedback, without the resource demands, and ethical issues of animal-based training. We tested this for a key subtaskand showed a strong link between skill in the simulator and in reality. Suturing performance was assessed for four groups of participants,including experienced surgeons and naive subjects, on a custom-made virtual-reality simulator. Each subject tried the experiment 30 timesusing five different types of needles to perform a standardized suture placement task. Traditional metrics of performance as well as newmetrics enabled by our system were proposed, and the data indicate difference between trained and untrained performance. In all tradi-tional parameters such as time, number of attempts, and motion quantity, the medical surgeons outperformed the other three groups,though differences were not significant. However, motion smoothness, penetration and exit angles, tear size areas, and orientation changewere statistically significant in the trained group when compared with untrained group. This suggests that these parameters can be used invirtual microsurgery training. VVC 2010 Wiley-Liss, Inc. Microsurgery 30:479–486, 2010.

Microsurgery is a key surgical speciality1 needing train-

ing and practice. The assessments of surgical competence

and dexterity have been broadly studied in the recent

years.2–5 The assessment principles are validity (whether

the parameters observed correspond to the performance

observed in real surgery) and reliability (how general

assessment criteria apply across tasks and people).6 Direct

observation, often without detailed and systematic crite-

ria, results in large interobserver variation that makes the

assessment not very reliable.5 Such assessments are not

standardized enough to compare surgical skills of train-

ees. Without agreement or conformity on criteria, the

results from any two senior surgeons on the same trainee

can vary, showing a lack of reliability.6

Direct observation with clear criteria is more objective.

Global rating scores with checklists are common criteria. In

1975, Harden et al.7 introduced checklists in the ‘‘objective

structured clinical examination.’’ These made assessment

more objective and are suitable for objective formative

feedback in training.5 However, the need for methods to

objectively assess the skills and competence of the surgeons

has become an important issue in the medical profession.2

Virtual-reality simulators have many potential advan-

tages in surgical skill assessment and training. These include

more opportunities for training, with no risk to patient;

repeatable degree of difficulty8; and reliable feedback for

both student and instructor. They also reduce resource and

ethical issues associated with animal-based training.9

We are developing a virtual-reality–based system for

microsurgical trainees, to train and assess suturing skill

using the virtual suture quality and to quantify differences

in needle shape efficacy. We show that parameters such

as needle design and skill level can be reliably measured

and assessed in the virtual environment.

MATERIALS AND METHODS

Virtual Reality Suturing Simulator

The digital microsurgical pretrainer is a virtual-real-

ity–based microsurgical suture platform for use, prior to

training on biological tissue. As shown in Figure 1, it

uses a dual Pentium Intel Workstation with CRT monitor

and a display setup from ReachIn Technologies (Stock-

holm, Sweden), comprising a 6DOF Desktop Phantom

haptic device from SensAble Technologies (Cambridge,

MA), Crystal Eyes stereo glasses, a stereo synchroniza-

tion emitter, a semitransparent mirror, and a metal frame.

The virtual environment consists of a level ‘‘tissue

membrane,’’ virtual needles of five different shapes, and

a needle holder. The operator interacts with it, using

shutter glasses, allowing depth perception of virtual

objects through a semitransparent mirror on the display

and manipulation of objects via a stylus. The Phantom

Desktop is a 6DOF positioning device placed in such a

way that the stylus movement workspace is below the

mirror. A button on the stylus adds input like mouse

click and drag. The workstation, emitter, and shutter

glasses provide stereo graphics.

1Department of Mechanical Engineering, National University of Singapore,Singapore2Natural Sciences and Engineering, National Institute of Advanced Studies,Bangalore, Karnataka, India3Department of Orthopaedic Surgery, National University of Singapore,Singapore4Department of Bioengineering, Imperial College of Science, Technologyand Medicine, London, UKHamed Kazemi and James K. Rappel contributed equally to the work.

*Correspondence to: Chee Leong Teo, Ph.D., Department of MechanicalEngineering, National University of Singapore, 9 Engineering Drive 1,Singapore 117576. E-mail: [email protected] and [email protected]

Received 26 October 2009; Accepted 7 January 2010

Published online 3 March 2010 in Wiley Online Library (wileyonlinelibrary.com). DOI 10.1002/micr.20766

VVC 2010 Wiley-Liss, Inc.

The monitor and mirror are mounted on the frame

such that the user looking into the mirror sees the reflec-

tion of the monitor image. By this arrangement, images

appear to be in the space below the mirror, where the

Phantom stylus is also located. By inverting the monitor

image, the reflected image appears upright. This arrange-

ment allows natural hand–eye coordination, as users inter-

act with the objects in the scene location where they feel

the stylus.

The software environment was developed using the

Reachin API, custom classes, and objects. The virtual

world consists of three objects: the stylus representing the

needle holder, the needle (five different shapes), and

the two-dimensional (2D) surface representing a tissue.

The needle holder in the virtual display follows the real

stylus in its position, orientation, and twist. The stylus is

held much the same way as a needle holder, and the

index finger presses the button. When the tip of the stylus

makes contact with the needle and the button is held

down, the needle is ‘‘grasped’’ and follows the stylus

movements until dropped by releasing the button.

Needles are polyhedral objects constructed with a

needle designer program, imported into the scene

graphics and made available to the simulator. The virtual

tissue is a translucent 2D surface that shows the needle

through it. A several-pixel-wide central gap (made of

fully transparent pixels) effectively divides the surface

into two parts (see Fig. 2).

Custom classes handle interactions between needle

holder and needle and also between needle and tissue.

When the stylus button is held down, collision detection

decides if the needle is near the holder’s tip, and the

holder should grasp it. Once grasped, it follows the trans-

lation, rotation, and twist of the holder. Twist is impor-

tant, providing rotation of the needle along its own axis,

as the user pronates or supinates the grasping hand’s

wrist. Once the button is released the needle is

‘‘dropped,’’ and the needle holder moves without it. The

dropped needle returns to home position, if dropped while

not touching the tissue; otherwise it stays unmoving.

So, we define the interaction of the needle with tis-

sue that the touched pixels become transparent (like the

gap) and the background color shows clearly. This tissue

is ‘‘unforgiving,’’ without tissue movement when the nee-

dle moves sideways, that in latex simulation limits tear-

ing. This was intended because in microvascular surgery,

visualization of damage is a more important cue than

feeling of tissue resistance. In the trial, the system tracks

and measures the position, orientation, current time and

status of the needle at a rate of 60 Hz, and writes them

to a data file. The file is analyzed after the trial, to

derive various characteristics of the suture placement

exercise.

Subjects and Protocol

The subjects, all right-handed volunteers who gave

informed consent, composed four groups (Table 1). The

first group consisted of six microvascular surgeons from

National University Hospital (NUH), Singapore, who rou-

tinely performed microsurgery in their practice, with an

average experience of 7 years. Seven medical students

who had undergone formal microsurgical training at

NUH made up another group of trained subjects. Groups

of untrained subjects consisted of six medical staff of the

National University Hospital with Bachelor of Medicine,

and seven nonmedical students from the National Univer-

sity of Singapore without microsurgical training or micro-

surgery experience.

Prior to starting the exercise, each subject was

instructed on the task and setup. No practice trial was

allowed, other than experiencing orientation to the equip-

ment and controls. The experiment was done under domi-

Figure 1. Hardware setup for virtual model. [Color figure can be viewed

in the online issue, which is available at wileyonlinelibrary.com.]

Figure 2. Video screen capture of model demonstrating the zero

resistance membranes represented in dark pink separated by a

gap, which is represented by the vertical black line. A needle

passed through the membrane makes visible entry and exit holes.

[Color figure can be viewed in the online issue, which is available

at wileyonlinelibrary.com.]

480 Kazemi et al.

Microsurgery DOI 10.1002/micr

nant hand. No subject had prior experience of virtual

reality or stereo glasses.

In a suturing attempt, the task is to grasp the needle

using the needle holder appropriately, bring it down with

the point orthogonal to the tissue, carefully pierce the tis-

sue to the right of the partition, surface the tip in the left

of the tissue by rotating the needle holder, release the

needle, grasp the tip of the needle on the left, pull it out,

and then release it. A set of five suturing attempts

presents all the five needle shapes in random order to

avoid learning bias in success rates with them. Comple-

tion of five sutures is defined as one trial. A typical eval-

uation session consists of 30 such suturing attempts.

We also wanted to assess the effect of various needle

shapes on performance. Five different shapes were used

(see Fig. 3): Three conventional needle shapes (2/8, 3/8, and

4/8 circles), and two nonconventional needles combining

curved and straight segments. The two unconventional nee-

dles were used to study whether needle familiarity conferred

additional performance advantage to the trained group.

Metrics of Performance Evaluation

Suturing performance was assessed with objective

scores (some of them drawn from the literature10),

thought to be related to suturing technique, as indicators

of skill. These parameters were: time to complete the

task; entry and exit tear size area; penetration and exit

angles; tool motion; average grasp number; and motion

smoothness. We hypothesize that these candidates may

be good to differentiate trained from untrained subjects.

Their descriptions are as follows:

Time (seconds). The total time to complete a task or

subtask is an important parameter of surgical studies. Un-

usual time spent on suturing tasks in surgery suggests

lack of skill.11

Tissue tear size area. The area of needle passage entry

and exit tears in the membrane, which measures tissue

trauma. This is one of the six items of global rating

scales, known as surgeon’s respect for tissue.12 Poor

trainees frequently damage tissue by inappropriate instru-

ment use; competent trainees handle tissue carefully but

occasionally cause inadvertent damage; clearly superior

surgeons consistently handle tissues appropriately, with

minimal damage.13,14

Penetration and exit angles. The angle between the

tissue surface and the needle tip direction at the point of

tissue contact. Most senior surgeons would agree that the

ideal penetration angle (see Fig. 4) is 908, which should

minimize trauma to tissue.10 Moreover, rolling the wrist

plays a major role in suturing and is related to this

parameter.

Tool motion. The distance that the stylus tip travels in

the entire task. In the Global Rating Scale, the poor

trainee makes many unnecessary or repetitive movements

while the superior trainee has a clear economy of move-

ment and maximum efficiency.15

Motion smoothness. We address smoothness by ana-

lyzing the oscillations during motion. A second order 2-

Hz cutoff high-pass Butterworth filter of the position data

retained only the oscillations, which we quantified by the

number of zero crossings and the integral of absolute

value, both normalized by the movement duration.

Ratio of distance traveled by needle relative to

stylus. The ratio of distance traveled by needle relative

Table 1. Participant Demographics

Characteristics Medical surgeons (n 5 6) Medical students (n 5 7) Medical staff (n 5 6) Nonmedical students (n 5 7)

Average age 43.2 25.4 33.9 25.8

Male gender 6/6 6/7 5/6 5/7

Right handed 6/6 7/7 6/6 7/7

Video game experience 0/6 1/7 1/6 2/7

Figure 3. Five different shapes of needles were used in this study.

A: 2/8 circle, (B) 3/8 circle, (C): 4/8 circle, (D) J-shaped, and (E)

compound double radius. [Color figure can be viewed in the online

issue, which is available at wileyonlinelibrary.com.]

Figure 4. The ideal penetration angle is 908. [Color figure can be

viewed in the online issue, which is available at wileyonlinelibrary.

com.]

Suturing Assessment in a Virtual Reality Simulator 481


to tool tip is a measure of the change of orientation and

hence the twisting of the wrist.

Grasp number. The mean of grasp/release steps per

suture, as a measure of fine control and work.

Data Analysis

The recorded data from each participant were ana-

lyzed afterward using MATLAB (v.R2006 a, The Math-

Works, Natick, MA, USA). We used Student’s t-test witha 5% significance level to compare between-group differ-

ences. All data were included in the analysis, with no

outliers discarded. To analyze performance, the suturing

task was first segmented to five key subtasks (see Fig. 5).

RESULTS

Motion Smoothness

Oscillations in stylus movements were isolated by

high-pass filtering of the position signal. By zero-crossing

count, medical students (MST) had significantly less os-

cillation than both medical staff (MSTF) and surgeons

(MS) (P < 0.036, Fig. 6A). In fact, both younger groups

had less oscillation (P < 0.041, Fig. 6B), suggesting age-

dependent tremor.

Time and Performance

We expected trained subjects to complete the experi-

ment significantly faster than the untrained group, to

require fewer grasps, and to save movement relative to

untrained subjects. However, this was not supported by

the data.

Mean times for the four groups (see Fig. 7) show no

significant difference, overall or in any of the five subtasks.

However, the novice group (medical staff and nonmedical

students) had significantly wider standard deviation than

surgeons, on all needles but the double-radius type.

Further, we found no statistically significant inter-

group difference in the grasp number for each pass, or in

the mean distance traveled by the stylus in any subtask.

Tissue Trauma

By contrast, in general surgeons and medical students

outperformed the novices in causing less damage to tissue

(see Fig. 8). Statistical significance was reached with the

4/8 needle for both entry (P < 0.019) and exit (P <0.033) tear sizes and for the J-shaped needle in exit tears

(P < 0.025). The trained subject group had significantly

less deviation in entry and exit tear sizes (P < 0.028),

except for the double- radius needle in the entry tear size.

Figure 5. Key suturing subtasks. A Grasp the needle; (B) position

for tissue entry; (C) insert and twist; (D) release and regrasp; and

(E) extraction. [Color figure can be viewed in the online issue,

which is available at wileyonlinelibrary.com.]

Figure 6. Normalized zero-crossing count of high-pass-filtered

holder tip position; a low number indicates smoother motion. A:

Comparison for the four groups of participant. B: Young versus

older subjects. (*P < 0.05). MS, medical surgeons; MST, medical

students; MSTF, medical staff; NMS, nonmedical students. [Color

figure can be viewed in the online issue, which is available at

wileyonlinelibrary.com.]

Figure 7. Mean completion time for the four groups of participants.



482 Kazemi et al.


Penetration and Exit Angles

Trained subjects (i.e., surgeons and medical students)

differed significantly from untrained subjects (staff and

nonmedical students) in manipulating a needle. The entry

angle was significantly larger for trained (see Fig. 9) with

all shapes (P < 0.009) but the unconventional double ra-

dius needle. Significant difference was detected neither

between the surgeons and medical students (P > 0.418)

nor between the staff and nonmedical students groups (P> 0.321), which suggests that the difference is due to the

training factor.

Effect of Needle Shape on Performance

To compare the effect of needle shape on performance,

we lumped data over groups for each of the five needles.

We analyzed the effect on performance for grasp number,

tear size, penetration and exit angles, smoothness, and tim-

ing, by taking the average scores for each shape. The results

first suggest that the needle shape does not affect motion

smoothness in virtual reality manipulation. Almost no dif-

ference was observed for any subtask or overall, across the

different needles for any of the groups (P > 0.532).

More difference was observed for time, in all of the

four subject groups. The 4/8 circle needle required the

most time to complete every subtask and overall suture,

while the nonconventional J-shaped needle required the

least. Similar results were obtained for the grasp number.

After normalizing by the time needed for needle 4/8

circle, a statistically significant difference in time was

observed between the 4/8 needle and both J-shaped and

3/8 needles (P < 0.040, Fig. 10).

Further, the 4/8 circle needle and compound double-

radius needle caused the largest tear size area, for all

subjects. The J-shaped needle seemed the easiest needle

to manipulate, with the least tear size area relative to the

other shapes.

Orientation Change

In three subtasks, the needle and needle holder travel

together: positioning the needle for needle entry, in-

serting and twisting the needle, and extracting it. No signifi-

cant interneedle differences in change of orientation

appeared in the positioning task (see Fig. 11). However, the

untrained subjects needed more changes of orientation than

the trained subjects in insertion and less in extraction.

DISCUSSION

Computer-based surgical simulation has been under de-

velopment for three decades.16 Playter and Raibert17 cre-

ated the first three-dimentsional virtual-reality simulator

intended to measure suturing skills in open surgery anasto-

mosis, the task of suturing together tube-like organs. A

PC-based laparoscopic cholecystectomy simulator, MIST

VR,18 has proved effective in improving laparoscopic in-

tracorporeal suturing skills.19 More recent work in neuro-

surgery20,21 and endoscopy22 is focused on highly realistic

Figure 8. Average entry tear size area (top) and exit tear size area

(bottom) for the four groups of study and different types of needles.



Figure 9. The angle between the needle tangent vector and the tis-

sue’s normal vector at the penetration and exit points of tissue (**P

< 0.01). The ideal path perpendicularly penetrates the tissue.





simulation by integrating visual realism and haptic (force)

feedback. Other groups examined the impact of surgical

simulation on improvement of psychomotor skills relevant

to the performance of laparoscopic surgery,23,24 Yamagu-

chi et al.25 reported the left hand performance difference

between experienced and novice laparoscopic surgeons,

while McDougall et al.26 found that simulations of all ba-

sic laparoscopic tasks could differentiate between subjects

with varying real-world expertise.

Our novel focus is immediate feedback and objective

measures of performance for trainees and for different

needle shapes, allowing comparison and improvement.

For each subtask, a performance database records time to

completion, distance traveled by tool tip, motion smooth-

ness, tear size area, errors in needle handling, orientation

change, and total grasp number for each suture pass. In

particular, errors in maneuvering the needle and the

extent of tissue damage, well quantified in the virtual

environment, are the important contribution of this simu-

lator, along with its ease in changing the magnification

level. In training surgeons for microvascular work, where

force feedback is minimal, visualization of the stitching

is more important than feeling the virtual membrane, so

haptic feedback has little to contribute.

This study assessed whether virtual suturing simula-

tion can distinguish expert and naive subjects in a stand-

ardized microsurgical suture placement task, and can

measure corresponding technical skills. Another important

issue was to examine whether the proposed performance

metrics are suitable for reflexes and technical skill.

The differences found in smoothness and oscillation

appear to stem from the age factor. Medical and nonmed-

ical students were younger than surgeons and staff, and

younger groups generally perform better in a dexterity

task.27 There is evidence that physiological tremor tends

to increase with age, and this is likely to have affected

smoothness during manipulation.28 However, micromani-

pulation skill in the virtual suture appeared not to depend

directly on the amount of oscillation. Despite a greater

oscillation, surgeons performed at least as well as the

younger medical students, by measures such as time,

shear size, and penetration angle.

The time to complete the task was smaller in the sur-

geons group, but the difference was not significant. In

previous studies, time to complete the suturing appeared

to be a sensitive measure of motor skill in microsur-

gery,29 but in our study, the completion time was similar

in all groups. One possible explanation is that the task

used in this study was simpler than actual suturing

including knot tying, which is more complex. However,

with respect to standard deviation of time, the untrained

group showed significantly larger deviation, in particular,

for subtasks 3 and 5. These are the most important

phases of the suturing task; entering and twisting the nee-

dle and finally extracting it from tissue. A smaller devia-

tion indicates better repeatability, which is important for

successful and reliable performance. The task simplicity

may have also contributed to the lack of statistically sig-

nificant differences in grasp number or traveled distance.

However, this study shows that our system can differ-

entiate well trained from untrained subjects, using the

penetration and exit angles, with trained subjects closer

to the orthogonal ideal angle. The results also confirmed

the relationship between the amount of tear size area and

angle error. Trained subjects, who passed the needle

through the tissue at an angle closer to the ideal, caused

lesser trauma than the untrained subjects. Our system was

sensitive enough to display this relationship. As expected,

the unconventional double-radius needle caused the larg-

est tear size on the entry side. However, the 4/8 circular

needle caused larger tear size in the exit.

Twisting the wrist is an important aspect of suturing.

The ratio of distance traveled by the needle relative to

Figure 11. Ratio of the distance traveled by needle relative to tool

tip. [Color figure can be viewed in the online issue, which is available


Figure 10. Effect of needle shape on task duration on trained and

untrained subjects. The needles are (from left to right) 2/8 C, 3/8 C,

4/8 C, J- shaped, and compound double radius. [Color figure can be

viewed in the online issue, which is available at wileyonlinelibrary.

com.]

484 Kazemi et al.


the tool showed significant differences in subtasks 3 and

5 between trained and untrained subjects. In subtask 3,

where inserting and twisting occurs, the untrained group

had a significantly larger ratio (more orientation change)

than the trained group. This may be due to a wrong

approach towards grasping the needle, and to a flatter

penetration angle, leading to more wrist effort for a suc-

cessful entry. However, it seems they were unaware of

the importance in the extracting phase of pulling out the

needle while following its trajectory, to minimize tear,

which requires more change of orientation. The untrained

group’s significantly smaller ratio indicates less effort to

turn the wrist, which likely resulted in their larger tear

size at the exit.

There are several notable features of our assessment

system. The membrane through which the needle had to

be passed was deliberately programmed to have no elas-

ticity. Unlike latex, which stretches before it tears, this

magnifies errors in needle handling. This gives immediate

and unambiguous feedback to the operator, with obvious

advantages to the trainee. The absence of resistance also

encourages more reliance on visual cues, which is impor-

tant in microsurgical skill acquisition. In addition, the

model allows the design of various virtual needle shapes.

In this study, three conventional-shaped needles (2/8, 3/8,

and 4/8) were mixed with two unconventional needle

shapes, unfamiliar to the microsurgeons. Our study

though does not address the grasping of the needle,

which in physical suturing can itself be a real challenge

to the novice surgeon. We replaced this hurdle by a sim-

ple button press, to minimize the complexity of the

assessment in this study.

In conclusion, although the task used in this study

was simpler than actual suturing, the simulator differenti-

ated well between experienced and inexperienced sub-

jects. The main parameters—motion smoothness, tissue

trauma, penetration angle, and the orientation change—

may be used in future to promote learning of microsurgi-

cal tasks. The comparison between trained and novice

groups showed that skill does transfer from real surgery

to the simulator, allowing useful assessment, and makes

it likely that transfer in the reverse direction is strong

enough for useful training. However, a systematic study

of needle grasping and manipulation subskills in trainees,

with and without time on the virtual system, is necessary

to confirm and quantify this.

REFERENCES

1. O’Connor HM, McGraw RC. Clinical skills training: Developingobjective assessment instrument. Med Educ 1997;31:359–363.

2. Darzi A, Smith S, Taffinder N. Assessing operative skill: Needs tobecome more objective. BMJ 1999;318:887–888.

3. MacRae H, Regehr G, Leadbetter W, Reznick RK. A comprehensiveexamination for senior surgical residents. Am J Surg 2000;179:190–193.

4. Ilie VL, Ilie VI, Ghetu N, Popescu S, Grosu O, Pieptu D. Assess-ment of the microsurgical skills: 30 minutes versus 2 weeks patency.Microsurgery 2007;27:451–454.

5. Kalu P, Atkins J, Baker D, Green C, Butler P. How do we assessmicrosurgical skill? Microsurgery 2005;25:25–29.

6. Moorthy K, Munz Y, Sarker SK, Darzi A. Objective assessment oftechnical skills in surgery. BMJ 2003;327:1032–1037.

7. Harden RM, Stevenson M, Downie WW, Wilson GM. Assessmentof clinical competence using objective structured examination. BMJ1975;1:447–451.

8. Satava RM, Jones SB. Current and future applications of virtual real-ity for medicine. Proc IEEE 1998;86:484–489.

9. Erel E, Aiyenibe B, Butler PE. Microsurgery simulators in virtualreality: Review. Microsurgery 2003;23:147–152.

10. O’Toole RV, Playter RR, Krummel TM, Blank WC, Cornelius NH,Roberts WR, Bell WJ, Raibert M. Measuring and developing sutur-ing technique with a virtual reality surgical simulator. J Am CollSurg 1999;189:114–127.

11. Johnston R, Bhoyrul S, Way L, Satava R. Assessing a virtualreality surgical skills simulator. Med Meets Virtual Real 1996;4:608–617.

12. Martin JA, Regehr G, Reznick RK, MacRae H, Murnaghan J, Hutch-inson C, Brown M. Objective structured assessment of technical skill(OSATS) for surgical residents. Br J Surg 1997;84:273–278.

13. Regehr G, MacRae H, Reznick RK, Szalay D. Comparing the psy-chometric properties of checklists and global rating scales for assess-ing performance on a osce-format examination. Acad Med1998;73:993–997.

14. Reznick RK, Regehr G, MacRae H, Martin J, McCulloch W. Testingtechnical skill via innovative bench station examination. Am J Surg1997;173:226–230.

15. Grober ED, Hamstra SJ, Wanzel KR, Reznick RK, Matsumoto ED,Sidhu RS, Jarvi KA. Validation of novel and objective measures ofmicrosurgical skill: Hand-motion analysis and stereoscopic visualacuity. Microsurgery 2003;23:317–322.

16. Rosen JM, Soltanian H, Redett RJ, Laub DR. Evolution of virtualreality [medicine]. IEEE Eng Med Biol 1996;15:16–21.

17. Playter R, Raibert M. A virtual surgery simulator using advancedhaptic feedback. Minim Invasive Ther Allied Technol 1997;6:117–121.

18. Wilson MS, Middlebrook A, Sutton C, Stone R, McCloy RF. MISTVR: A virtual reality trainer for laparoscopic surgery assesses per-formance. Ann R Coll Surg Engl 1997;79:403–404.

19. Kothari SN, Kaplan BJ, DeMaria EJ, Broderick TJ, Merrell RC.Training in laparoscopic suturing skills using a new computer-basedvirtual reality simulator (MIST-VR) provides results comparable tothose with an established pelvic trainer system. J Laparoendosc AdvSurg Tech A 2002;12:167–173.

20. Lemole GM Jr, Banerjee PP, Luciano C, Neckrysh S, Charbel FT.Virtual reality in neurosurgical education: Part-task ventriculostomysimulation with dynamic visual and haptic feedback. Neurosurgery2007;61:142–149.

21. Kockro RA, Stadie A, Schwandt E, Reisch R, Charalampaki C, NgI, Yeo TT, Hwang P, Serra L, Perneczky A. A collaborative virtualreality environment for neurosurgical planning and training. Neuro-surgery 2007;61:379–391.

22. Van der Meijden OAG, Schijven MP. The value of haptic feedbackin conventional and robot-assisted minimal invasive surgery and vir-tual reality training: A current review. Surg Endosc 2009;23:1180–1190.

23. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, RosenbergJ, Funch-Jensen P. Randomized clinical trial of virtual reality sim-ulation for laparoscopic skills training. Br J Surg 2004;91:146–150.

24. Aggarwal R, Grantcharov TP, Eriksen JR, Blirup D, Kristiansen VB,FunchJensen P, Darzi A. Evidence-based virtual reality training pro-gram for novice laparoscopic surgeons. Ann Surg 2006;244:310–314.



25. Yamaguchi S, Konishi K, Yasunaga T, Yoshida D, Kinjo N,Kobayashi K, Ieiri S, Okazaki K, Nakashima H, Tanoue K, MaeharaY, Hashizume M. Construct validity for eye-hand coordination skillon a virtual reality laparoscopic surgical simulator. Surg Endosc2007;21:2253–2257.

26. McDougall EM, Corica FA, Boker JR, Sala LG, Stoliar G, Borin JF,Chu FT, Clayman RV. Construct validity testing of a laparoscopicsurgical simulator. J Am Coll Surg 2006;202:779–787.

27. Scheueneman A, Pickleman J, Freeark R. Age, gender, lateral domi-nance, and prediction of operative skill among surgical residents.Surgery 1983;98:506–513.

28. Sturman MM, Vaillancourt DE, Corcos DM. Effects of aging on theregularity of physiological tremor. J Neurophysiol 2005;93:3064–3074.

29. Starkes JL, Payk I, Hodges NJ. Developing a standardized test forthe assessment of suturing skill in novice microsurgeons. Microsur-gery 1998;18:19–22.

486 Kazemi et al.


Documents

Assessing suturing techniques using a virtual reality surgical simulator