Do the laparoscopic skills of trainees deteriorate over time?

Prashant Sinha Æ Nancy J. Hogle Æ Dennis L. Fowler

Received: 12 December 2007 / Accepted: 24 March 2008 / Published online: 25 April 2008

� Springer Science+Business Media, LLC 2008

Abstract

Introduction Without ongoing practice, acquired motor

skills may deteriorate over time. The purpose of this study

is to document the level of retention of laparoscopic skills

over time.

Methods Thirty-three general-surgery PGY 1, 2, and 3

residents trained to established criteria and passed an exam

for each of seven technical skills (camera navigation,

instrument navigation, camera/instrument coordination,

grasping, lifting and grasping, cutting, and clip applying)

on a virtual simulator (LapSim� Surgical Science Ltd.,

Goteborg, Sweden). Six months later, the residents again

completed the exam for each of the seven skills. During the

6 months, the simulators were available, but additional

practice was not required. The retesting process consisted

of three attempts, the first of which was acclimatization.

The results of the subsequent two exams were compared

with baseline data.

Results At retest, the number of residents who passed clip

applying (7, 21%) and cutting tasks (18, 55%) was sig-

nificantly lower than for the other five tasks (p \ 0.05). In

failed tests, instrument wandering and tissue damage were

more common than increases in task time. Upper-level

residents were significantly more likely to pass than first-

year residents were (p \ 0.01). Time of day did not

influence passing rates.

Conclusion Six months after training to criteria, instru-

ment and tissue-handling skills deteriorated more than the

speed with which a task is completed. Evidence of skill

retention was present for some but not all tasks. Fine motor

skills, required to perform more difficult tasks, deteriorated

more than skills needed for easier tasks.

Keywords Simulator training � Virtual-reality training �Laparoscopic skills � Laparoscopic surgery � Test–retest �Skills retention

Surgical education in the future will necessarily require a

focus on quality as well as quantity. Hutter et al. [1]

expressed that, while the quality of life of the current

resident has improved, the opportunity for current surgical

trainees to develop the skills to become proficient surgeons

and surgeon-educators may be reduced. Unlike airline

pilots, there is no requirement for surgeons in training or

practice to pass formal technical evaluation prior to per-

formance in the operating room. For these reasons, surgical

educators have begun developing tools that offer alterna-

tive methods for surgeons and trainees to learn technical

skills. Two important components of these training systems

are the delivery of structured education (a curriculum) and

the development of methods for learning technical skills

outside the operating room. An important part of the latter

is the development of methods to assess technical skills.

The most advanced of these systems has been developed

for laparoscopic skills training and assessment. Torkington

et al. [2, 3] demonstrated that, with structured assessment,

virtual and real-world performance on simple tasks could

be improved using virtual simulator training. Peters et al.

[4] demonstrated that laparoscopic skills might be assessed

This work was presented at the meeting of SAGES, Las Vegas, NV,

April 2007.

P. Sinha � N. J. Hogle � D. L. Fowler

College of Physicians and Surgeons, Columbia University, 622

West 168th Street PH 12-Rm 126, New York, NY 10032, USA

D. L. Fowler (&)

Department of Surgery, 622 West 168th Street PH 12-Rm 126,

New York, NY 10032, USA

e-mail: dlf91@columbia.edu

123

Surg Endosc (2008) 22:2018–2025

DOI 10.1007/s00464-008-9929-5

reliably and with meaningful translation of those skills to

the operating room. Their findings highlight that significant

performance variations may exist at all levels and that

these variations could be time dependent, prompting a call

for longitudinal evaluation. Porte et al. [5] have demon-

strated that verbal expert training improves skill retention

in open knot tying when compared with computer-based

education. Stefanidis et al. [6] recently demonstrated that

ongoing training resulted in better laparoscopic suturing

and knot tying. Similarly, Moulton et al. [7] demonstrated

that a microvascular anastomosis technique that is taught in

a distributed manner over weekly sessions improved

retention when compared to a one-day course.

The issue of skill retention is, therefore, important in

relation to both the timing and methods with which trainees

learn and also in relation to its impact on patient care. By

performing a systematic assessment of skill retention for a

variety of laparoscopic tasks using a computer-based sim-

ulator, we evaluated which skills degrade over time, and

which components of performance deteriorate. For our

training and testing platform, we chose the LapSim� vir-

tual-reality laparoscopic simulator for its detailed motion

and tissue damage analysis, structured computer-based task

training, realistic immersional simulation, and documented

validity [8–10]. Our goal in this study was to use the

detailed feedback from this simulator to document the

retention of learned skills (or lack thereof) and to help us

understand which aspects of these skills (precision or

speed) are retained or lost.

Methods

Subjects

Thirty-three surgical residents from the first, second, and

third years of training were enrolled in the study. The

composition of residents included 18 first-year, 8 second-

year, and 7 third-year residents. All 33 residents completed

the study. They performed their baseline testing in the first

quarter of their training year and completed retesting

6 months later. Institutional review board (IRB) approval

was granted.

Simulator

The LapSim� laparoscopic simulator (Surgical Science

Ltd., Goteborg, Sweden) was used to train and test resi-

dents on seven tasks relevant to laparoscopic surgery. All

subjects trained until each task was passed during the first

quarter of the academic year. Passing score was determined

by an experienced laparoscopist. Six months later subjects

were retested. The retest consisted of one round on the

simulator for acclimatization followed immediately by two

formal testing rounds. For numerical scores, we compared

the mean of the two retest scores with the score at the time

the test was initially passed. During the intervening

6 months the simulator was made available for practice.

Tasks available on the simulator included camera nav-

igation (CN), instrument navigation (IN), camera/

instrument coordination (CO), grasping (GR), lifting and

grasping (LG), cutting (CU), and clip applying (CL).

Parameters that were continuously measured and recorded

by the computer for each task included some or all of the

following variables, depending on relevance to the specific

task: overall task pass/fail, time (in seconds) required to

complete a task, total angular path of travel (degrees

traveled around the trocar), total linear path of travel (in

mm, total length traveled by the instrument tip), tissue

damage (number of times nontargeted tissue touched),

tissue stretch (in mm, total distance that tissue is deformed

by instruments), clip misses (number of times clips are not

placed on tissue), badly placed clips (number of times clip

is not placed in the target area), dropped clips (number of

clips that are dropped and not placed), blood loss (in ml),

segments ripped (number of times tissue ripped instead of

cut), and segments dropped (number of cut tissue segments

dropped outside of target area). To pass a task, the resident

was required to achieve the established level of perfor-

mance for each parameter relevant to that task.

Conceptually, the goal was to enable the junior resident

to train to a skill level that was approximately two-thirds of

the level of performance of an experienced laparoscopic

surgeon. Hence, the passing score for each parameter was

determined by reducing by approximately one-third the

measured performance of a highly experienced laparosco-

pist. For example, if the average time for the experienced

laparoscopist to complete a task was 60 s, the passing time

for the test subjects was 80 s, about one-third longer.

Measured parameters for each task included time, misses,

and tissue damage, as well as task-specific parameters such

as blood loss and clip placement. Performance for each

parameter was recorded continuously by the simulator.

Failure to meet the established level of performance for one

or more parameter on any task resulted in failure of the

task. No weighting was given to any criterion. Passing the

task simply required passing each parameter of the task.

Statistical analysis

Chi-square analysis was used to determine differences in

passing rates between tasks. Failures in common task

components, linear path, damage, and time were correlated

to overall passing using linear regression analysis. Analysis

of variance (ANOVA) was used to determine differences

between task components in those passing and those

Surg Endosc (2008) 22:2018–2025 2019

123

failing. Finally, matched pairs analysis using Student’s T-

test was used to compare baseline and exam scores for each

task and task component. All statistical analysis was per-

formed using JMP� 2007 SAS Institute, Cary, NC, USA.

Results

All subjects completed testing. None of the subjects logged

any additional time on the simulator during the 6-month

follow-. The ACGME case logs of the residents were

reviewed for the entire testing period using basic and

advanced categories as defined by the ACGME. There

were no significant intralevel differences in numbers of

cases performed. First-year residents on average performed

9.6 basic cases during this period and 0.8 advanced cases.

Second-year residents on average performed 47 basic lap-

aroscopic cases and 4 advanced cases. Third-year residents

on average performed 40 basic laparoscopic cases and 16.7

advanced cases. Basic cases were more frequently per-

formed by upper-level residents than the first-year

residents, and the third-year residents additionally per-

formed more advanced cases than their juniors.

Skill retention was demonstrated by passing the repeat

exam. Passing rates were similar for CN, IN, and GR

(n = 26, 79%) and for CO and LG (n = 25, 76%). Passing

rates were significantly worse for CU (n = 18, 55%) and

CL (n = 7, 21%) than the other tasks. The cumulative

passing rate for all seven exams was significantly higher

for both second- and third-year residents when compared to

first-year residents, while individually, only the cutting task

demonstrated a significant interlevel difference, as shown

in Table 1. Failure of a task was frequently due to failure of

more than one task parameter. Table 2 summarizes the

pass/fail rates for each task and includes the parameters

common to all seven tasks: time, path length, angle length,

and tissue damage. In four cases of failure for the clip

application task, poorly placed clips solely accounted for

failure, however, for all of the other tasks, the failures (211/

215) were explained by failures of time, motion (angular

and/or linear) or tissue damage (excessive deformation

and/or excessive nontarget tissue manipulation). Multi-

variate analysis revealed parameter correlation to overall

task failure of 0.44 for time, 0.49 for path or angle length,

and 0.88 for tissue damage when all tasks were compared

together. Table 3 summarizes baseline to retesting scores

Table 1 Retest passing rates by

task and PGY levelNumber passing

on retest (%)

Significant

difference from

other tasks

PGY passing

rates

Significance for

interlevel

differences

Camera navigation 26 (79%) ns 1: 72.2% p = 0.15

2: 75%

3: 100%

Instrument navigation 26 (79%) ns 1: 72.2% p = 0.15

2: 75%

3: 100%

Coordination 25 (76%) ns 1: 66.7% p = 0.07

2: 100%

3: 71.4%

Grasping 26 (79%) ns 1: 66.7% p = 0.07

2: 100%

3: 85.7%

Lifting and grasping 25 (76%) ns 1: 66.7% p = 0.39

2: 87.5%

3: 85.7%

Cutting 18 (55%) p \ 0.05 1: 33.3% p = 0.02

2: 87.5%

3: 71.4%

Clip applying 7 (21%) p \ 0.05 1: 16.7% p = 0.77

2: 25%

3: 28.6%

Cumulative 153 (66%) – 1: 56% p \ 0.01

2: 79%

3: 78%

2020 Surg Endosc (2008) 22:2018–2025

123

for common components across the seven tasks using

matched pairs Student’s T-test along with significance.

Notably, there were no differences from baseline to retest

overall in the time taken to complete a task except for a

significantly longer time in the clip application time and a

significantly shorter time in instrument navigation. There

were increases at retest in the path or angular length taken

for all tasks. This reached significance for the CN, CO, CU,

and CL tasks. Finally, at retest, all tasks had increases in

tissue damage, reaching significance in all tasks except CN.

The scores at retest were then compared by segregating

those who passed and those who failed and comparing

those by analysis of variance. The results are tabulated in

Table 4. Baseline scores were very close to scores for those

who passed except in the path length taken for the CN task

where the passing path length was 6.02 cm versus 3.1 cm

at baseline, and 2.8 cm for those failing. Those differences

were not significant however (p = 0.08). The failing scores

explained the overall changes from baseline to retest in

Table 3 and are discussed in detail below.

Time for task completion

The time taken to complete some tasks on retesting dem-

onstrated significant differences when compared with

baseline times. However there was no consistency in the

Table 2 Failure analysis and

correlationsTime fail Path or angle fail Damage fail Overall fail

n (%) n (%) n (%) (%)

Camera navigation 12 (52) 14 (61) 3 (13) 23

Instrument navigation 0 (0) 0 (0) 23 (100) 23

Coordination 7 (28) 5 (20) 23 (92) 25

Grasping 6 (27) 5 (23) 19 (86) 22

Lifting and grasping 2 (7) 6 (21) 29 (100) 29

Cutting 3 (8) 6 (16) 31 (82) 38

Clip applying 26 (47) 32 (58) 49 (89) 55

R2 0.44 0.49 0.88 –

Table 3 T-test baseline to retest comparison

Time (s) Path length (mm) Angular path length

(deg)

Tissue damage

(n)

Maximum damage

(mm)

Base Retest Base Retest Base Retest Base Retest Base Retest

Camera navigation 16.5 18.7 3.1 3.8 112.5 175.3 0.06 0.14 0.35 2.2

p = 0.07 p = 0.48 p \ 0.01 p = 0.28 p = 0.16

Instrument navigation 12.2 (L) 9.9 (L) 0.63 (L) 0.57 (L) 154.6 (L) 140.1 (L) 0.27 0.98 0.48 10.7

12.3 (R) 9.7 (R) 0.6 (R) 0.55 (L) 125.1 (R) 116.9 (R) p \ 0.01 p = 0.02

p \ 0.01 p = 0.13 p [ 0.19

Coordination 33.5 33.9 1.1 (inst) 1.2 (inst) 327.7 (inst) 340.2 (inst) 0.55 1.7 0.8 12.1

p = 0.9 0.2 (cam) 0.3 (cam) 115.9 (cam) 165.2 (cam) p \ 0.001 p = 0.01

p \ 0.01 p = 0.04

Grasping 20.2 (L) 21.0 (L) 1.0 (L) 1.0 (L) 175.5 (L) 193.8 (L) 0.3 1.5 0.5 22.1

17.4 (R) 17.0 (R) 0.86 (R) 0.91 (R) 165.7 (R) 180.6 (R) p \ 0.0001 p \ 0.01

p [ 0.5 p [ 0.5 p [ 0.1

Lifting and grasping 56.1 51.3 1.6 (L) 1.6 (L) 324.0 (L) 339.9 (L) 1.8 3.4 1.6 22.3

p [ 0.1 1.6 (R) 1.6 (R) 315.3 (R) 337.4 (R) p \ 0.01 p \ 0.01

p [ 0.5 p [ 0.1

Cutting 48.2 45.7 0.3 (cut) 0.4 (cut) 66.0 (cut) 91.8 (cut) 0.1 1.3 0.2 21.3

p = 0.4 0.4 (grasp) 0.5 (grasp) 84.2 (grasp) 104.7 (grasp) p [ 0.01 p [ 0.01

p = 0.03 p = 0.01

Clip applying 56.3 84.6 0.9 (L) 1.5 (L) 147.5 (L) 275.2 (L) – – 46.2 83.6

p [ 0.001 0.9 (R) 1.7 (R) 142.7 (R) 305.8 (R) p [ 0.001

p [ 0.001 p [ 0.001

Bold values are significant p \ 0.05

Surg Endosc (2008) 22:2018–2025 2021

123

Ta

ble

4B

asel

ine

and

AN

OV

Ao

fre

test

sco

res

seg

men

ted

by

pas

san

dfa

il

Tim

eto

com

ple

te(s

)P

ath

len

gth

tak

en(c

m)

Dam

age

toti

ssu

ein

flic

ted

(mm

defl

ecti

on

)

Bas

elin

eP

assi

ng

Fai

lin

g;

pB

asel

ine

Pas

sin

gF

aili

ng

;p

Bas

elin

eP

assi

ng

Fai

lin

g;

p

Cam

era

nav

igat

ion

16

.5±

3.3

16

.5±

0.8

22

.9±

1.5

;3

.1±

1.2

6.0

2±

1.6

2.8

±0

.9;

0.0

61

±0

.24

0.0

67

±0

.04

0.2

6±

0.0

7;

p\

0.0

01

p=

0.0

8p

=0

.02

Inst

rum

ent

nav

igat

ion

12

.2±

3.2

(L)

11

.4±

0.4

(L)

8.2

±0

.7(L

)0

.63

±0

.16

(L)

0.6

2±

0.0

2(L

)0

.52

±0

.04

(L)

0.2

7±

0.4

50

.28

±0

.12

.3±

0.2

;

12

.3±

2.7

(R)

11

.4±

0.4

(R)

8.0

±0

.7(R

);0

.60

±0

.19

(R)

0.5

8±

0.0

2(R

)0

.52

±0

.04

(R);

p[

0.0

01

p\

0.0

01

(L)

p=

0.0

2(L

)

p\

0.0

01

(R)

p\

0.0

01

(R)

Co

ord

inat

ion

33

.5±

7.1

32

.4±

1.5

37

.9±

2.6

;1

.12

±0

.11

(in

st)

1.1

±0

.04

(in

st)

1.2

±0

.07

(in

st)

0.5

5±

0.6

20

.63

±0

.23

.4±

0.3

;

p=

0.0

70

.20

±0

.10

(cam

)0

.23

±0

.02

(cam

)0

.36

±0

.04

(cam

);p[

0.0

01

p=

0.1

0(i

nst

)

p=

0.0

08

(cam

)

Gra

spin

g2

0.2

±3

.6

(L)

19

.9±

1.0

(L)

23

.7±

1.8

(L)

1.0

±0

.19

(L)

0.9

9±

0.0

4(L

)1

.1±

0.0

7(L

)0

.30

±0

.47

0.3

±0

.12

.6±

0.2

;

17

.4±

4.8

(R)

16

.5±

0.7

(R)

19

.7±

1.3

(R)

0.8

6±

0.1

9(R

)0

.84

±0

.04

(R)

1.1

±0

.07

(R);

p[

0.0

01

p=

0.0

7(L

)p

=0

.22

(L)

p=

0.0

06

(R)

p=

0.0

03

(R)

Lif

tin

gan

dg

rasp

ing

56

.1±

9.3

55

.1±

2.0

47

.8±

3.1

;1

.56

±0

.27

(L)

1.6

±0

.06

(L)

1.6

±0

.09

(L)

1.8

5±

1.4

1.9

±0

.35

.2±

0.4

;

p=

0.0

61

.55

±0

.29

(R)

1.6

±0

.06

(R)

1.6

±0

.09

(R);

p[

0.0

01

p=

0.6

6(L

)

p=

0.5

1(R

)

Cu

ttin

g4

8.2

±7

.34

5.2

±1

.84

8.9

±2

.3;

0.3

5±

0.1

3(c

ut)

0.3

4±

0.0

3(c

ut)

0.4

9±

0.0

3(c

ut)

0.1

2±

0.3

30

.12

±0

.22

.1±

0.3

;

p=

0.2

10

.43

±0

.14

(gra

sp)

0.4

4±

0.0

2

(gra

sp)

0.5

5±

0.0

3(g

rasp

);p[

0.0

01

p[

0.0

01

(cu

t)p

=0

.00

4(g

rasp

)

Cli

pap

ply

ing

56

.3±

11

.55

7.3

±6

.58

9.0

±5

.8;

0.9

1±

0.1

6(L

)0

.92

±0

.2(L

)1

.6±

0.1

(L)

46

.2±

21

.04

7.1

±3

.69

0.0

±3

.2;

p[

0.0

01

0.8

5±

0.2

2(R

)0

.87

±0

.2(R

)1

.9±

0.2

(R);

p[

0.0

01

p[

0.0

01

(R)

p[

0.0

01

(L)

Bo

ldv

alu

esar

esi

gn

ifica

nt

p\

0.0

5

2022 Surg Endosc (2008) 22:2018–2025

123

manner in which mean times for those passing versus those

failing changed. For some tasks, such as CN and CL, the

mean time for those passing was significantly shorter than

for those failing on retesting, and the times correlated well

with whether the resident passed the task (CN R2 = 0.64,

CL R2 = 0.53). For IN, the mean passing times for those

passing the task were actually longer than for those failing,

and hence, did not correlate well at all (R2 = 0). Passing

times were not significantly different from failing times for

CO, GR, LG or CU. In multivariate analysis, achieving a

passing time was significantly correlated with passing the

task overall in five out of seven tasks; passing the LG and

IN tasks did not demonstrate a significant correlation with

achieving a passing time in those tasks. Overall, the mean

times did not demonstrate consistent and significant chan-

ges to explain passing from failing for all tasks.

Path length

Mean angular and linear path lengths were correlated with

each other, and most tasks resulted in average path lengths

that were significantly shorter for those passing than those

failing. CN was unique in that mean linear path lengths

were similar for passed and failed tasks, but significant

reductions in the mean angular path lengths were seen in

those passing. IN demonstrated nonsignificant increases in

mean angular and linear paths, and LG revealed a non-

significant decrease in path length for those passing. All

others (CO, GR, CU, and CL) demonstrated significantly

reduced mean angular and linear path lengths for those

passing. Failure on retesting was associated with signifi-

cantly longer mean path lengths in five out of seven tasks.

Correlation to overall passing ranged from 0 for IN to 0.59

for CL and was significant in all tasks except IN.

Tissue damage

Mean tissue damage for all tasks was significantly greater

for those residents who failed at the time of retesting when

compared with mean tissue damage for residents who

passed retesting. Two forms of tissue damage were mea-

sured: tissue damage (a summative number of the

incidences of touching nontargeted tissue) and maximum

damage (in millimeters, the amount of tissue deformation

of nontarget tissue). In all tasks except IN, the deformation

damage had higher correlation to task failure than simply a

count of nontarget tissue touching. When both types of

tissue damage parameters were combined, there was a very

high correlation to task failure. The composite tissue

damage parameter explained failures significantly for all

tasks with a correlation of 0.32 for CN, but 0.9 or higher

for all others. Table 3 summarizes matched-pair baseline to

retest differences in time, linear path length, and tissue

damage along with significance. Table 2 summarizes the

correlation of common task parameters to passing for all

tasks.

Other measures

Other measures were task specific, and the baseline to

retest changes are listed as follows.

Camera drift (CN only), measured in mm of linear

movement, documented the change from target-centered

focus and did not demonstrate significant change from

baseline, 0.56 to 0.63 (p = 0.2). Instrument misses in the

CO task measured the number of times an instrument

missed touching its target. There was a nearly significant

difference of 8.5 more misses on retesting (p = 0.059)

compared with no misses at baseline. In the LG task there

was also near significance in instrument misses: 9.1 more

misses on retest versus 0 (p = 0.056). In the GR task,

instrument misses in terms of target grasping did not show

any differences at retest.

Ripped segments (CU task only) measured the number

of tissue segments ripped during the task. There was a

highly significant difference at retest: 4.5 segments versus

1.8 at baseline (p = 0.018). Dropped segments (CU task

only) measured dropped tissue segments but did not dem-

onstrate any significant differences however in the CU

task; the maximum stretch-related damage was signifi-

cantly increased on retesting: 38.1 versus 13.7 mm

(p B 0.001).

Finally, components specific to the CL task were blood

loss measured (0.22 versus 0 cc), badly placed clips (0.52

versus 0), dropped clips (0.5 versus 0), and maximum

stretch damage (83.6 versus 46.2 mm). These components

were all increased on retesting with significance of

p \ 0.001.

Discussion

We have been able to quantify several key aspects of the

performance of surgical skills, namely retention, speed, and

precision, in this study. It is meaningful that 20% or more

of the subjects fail even simple tasks on retesting. Our

subjects appear to fail most often due to poor precision in

the performance of these tasks. Precision is manifested in

our study as efficiency in motion and the avoidance of

tissue damage. An increase in tissue damage was found to

be a particularly good surrogate for failing a task. Instru-

ment and camera wandering, as evidenced by increases in

either linear or angular motion or drift, were also signifi-

cantly correlated with task failure. However, the time taken

to complete a task had little correlation with passing the

task. Precision had a higher correlation to passing or failing

Surg Endosc (2008) 22:2018–2025 2023

123

a task than speed, and is likely a good surrogate marker for

the retention of skill.

Stefanidis et al. [6] demonstrated clearly that a 6-month

interval between testing and retesting is adequate to show

performance degradations. In their study they used the

Fundamentals of Laparoscopy (FLS) testing platform using

a suturing and tying model, one of the most difficult tasks

in laparoscopy. We find that skill retention varies signifi-

cantly by task complexity and somewhat by resident level

of training. Basic tasks such as camera and instrument

targeting, as well as grasping, are easier than complex tasks

such as cutting and clip applying. The advanced tasks

require finer movements and coordination of two hands in a

small working area. While we see some degradation of

basic task skills, the more advanced skills of cutting and

clip applying were poorly retained. This may be due to

faster loss of motor memory in finer, more complex tasks,

degradation of the cognitive aspects of more complex

tasks, or a combination of both. It is however evident that a

time-dependent degradation task performance may be

responsible for poor retention.

Both Porte and Stefanidis [3, 4] chose a group of sur-

gically naive medical students in their studies; however, we

chose to focus on surgical residents in training in order to

allow real-world influences to color the results. Surpris-

ingly, additional years of training in this study did not

impact the clip applying task despite a reasonable exposure

to clinical laparoscopy in the form of appendectomies and

cholecystectomies during the middle years of training.

Stefanidis [11] in an earlier study had also found similar

differences in rates of retention by residents between basic

laparoscopic skills (83%) and laparoscopic suturing (60%).

Our findings are in agreement with other studies that

demonstrate that skills, particularly more complex ones,

will degrade without ongoing structured practice. Impor-

tantly, ongoing mixed clinical exposure does not seem to

correct for this degradation of skill. We do not know

whether this exposes a weakness in clinical education, a

lack of transfer of skills between simulator training and the

operating room, or perhaps a difference in operator inde-

pendence and responsibility when performing a task in the

operating room versus in the simulator. This finding is

provocative though, and suggests that an interval of

6 months is too long between focused practices, resulting

in significant decline of simulator skill. Regularly sched-

uled intervals of structured skills training may be necessary

until the appropriate cognitive learning of a skill is trans-

lated into automatic skill performance with improved and

longer-term retention. We do not know from this study

whether improvements in skill on the simulator platform

will be translated to the operating room. It does show that

skills can decline significantly without practice, and that,

the more complex the skill, the sharper the decline.

If retention of skill requires ongoing practice and struc-

tured education, it may be necessary to allocate more

resources to this task. Live expert feedback has been dem-

onstrated to be a highly effective method to evaluate surgical

skills [5], however it is labor and time intensive. Given the

cost of labor and the current length of training, creating a

cost- and time-efficient skills training program poses a sig-

nificant challenge. Previous attempts to deconstruct surgical

skills into components have included both checklist rating

systems as well as global assessments and motion analysis

systems [5, 12–14]. Efforts have been made to capture effi-

ciency of movement and tissue handling with some success,

however as Porte has shown, a live instructor is able to

provide more detailed and focused feedback than a com-

puter. In a well-conducted randomized trial that

demonstrated positive translation from simulator to OR,

Seymour et al. [15] trained residents on the minimally

invasive surgical trainer-virtual reality (MIST-VR) system

with an expert surgeon present during the virtual training

session as well as real surgery. Our study, in contrast, did not

include a live expert during simulator training.

Global assessment scales rated by live experts can easily

capture many modes of failure, while computer systems are

still learning how to separate them. Motion analysis systems

such as the imperial college surgical assessment device

(ICASD) and advanced dundee endoscopic psychomotor

trainer (ADEPT) systems [14] do not understand tissue

damage yet, a very important mode of failure in task reten-

tion. We have just begun to understand some of the

challenges in using motion and tissue analysis in this study.

We have found that maintaining respect for tissue correlates

the most strongly and consistently with passing rates, fol-

lowed by path efficiency. The speed with which tasks were

completed did not correlate with passing a retest on the

laparoscopic virtual simulator. It is encouraging that com-

ponents of technical skills can be deconstructed using the

LapSim� system. This gives us hope that computerized

surgical skill education and assessment may be possible,

either with or without a live expert during training. This is an

area that will require ongoing research and development.

There are limitations to this study that preclude gener-

alization to clinical practice. While it is our belief that

practice on a regular basis is required for maintaining skill

performance on a simulator or in the operating room, we

cannot translate our simulator skill to operating room skill

as this study was not designed to test this. We also chose an

arbitrary scoring system based on an experienced laparos-

copist’s performance, and used our subjects as their own

baseline. While this provides the best control for this study,

it may prove difficult to replicate outside this institution

and does not easily allow comparison with other studies.

This study provides a series of important observations

that warrant further study, however. First, we did find that

2024 Surg Endosc (2008) 22:2018–2025

123

6 months is too long a period to go without practice. We

also learned that residents do not voluntarily practice on

simulators. Most importantly, we learned about the modes

of failure in the performance of a task. It remains to be seen

whether repeated practice on this virtual simulator

improves real-world performance or whether nonvirtual

simulator practice is required. We are hopeful that virtual

simulation can translate to the OR, as seen in papers from

Seymour and Torkington, but it may require the use of

expert trainers and more frequent training. Finally, failure-

mode analysis from this study may help to guide the

development of a virtual expert that can identify and cor-

rect performance as well as a live expert does on simulators

and in reality.

Conclusion

Laparoscopic surgical skills, even basic ones, are not

consistently retained without practice. This study demon-

strates that it is possible to understand which aspects of

laparoscopic skills are not retained. Using performance on

a simulator as an assessment tool for measuring parameters

specific to laparoscopic skills, we find that complex, fine

motor skills more commonly deteriorate than either time to

complete a task or less complex motor skills. We believe

that regularly scheduled, structured education and assess-

ment should and will become a mandatory part of surgical

training. In order to make the training and assessment

meaningful and translatable, expert systems whether live or

simulated may be required to guide training and assess-

ment. Sophisticated computer-based simulation currently

plays an integral role in the education and maintenance of

skills in airline pilots, and may eventually play a similar

role in the education and assessment of surgeons in training

and in practice.

References

1. Hutter MM et al (2006) The impact of the 80-hour resident

workweek on surgical residents and attending surgeons. Ann Surg

243(6):864–871; discussion 871–5

2. Torkington J et al (2001) The role of the basic surgical skills

course in the acquisition and retention of laparoscopic skill. Surg

Endosc 15(10):1071–1075

3. Torkington J et al (2001) Skill transfer from virtual reality to a

real laparoscopic task. Surg Endosc 15(10):1076–1079

4. Peters JH et al (2004) Development and validation of a com-

prehensive program of education and assessment of the basic

fundamentals of laparoscopic surgery. Surgery 135(1):21–27

5. Porte MC et al (2007) Verbal feedback from an expert is more

effective than self-accessed feedback about motion efficiency in

learning new surgical skills. Am J Surg 193(1):105–110

6. Stefanidis D et al (2006) Proficiency maintenance: impact of

ongoing simulator training on laparoscopic skill retention. J Am

Coll Surg 202(4):599–603

7. Moulton CA et al (2006) Teaching surgical skills: what kind of

practice makes perfect? a randomized, controlled trial. Ann Surg

244(3):400–409

8. Duffy A et al (2005) Construct validity for the LAPSIM lapa-

roscopic surgical simulator. Surg Endosc 19:401–405

9. Woodrum DT et al (2006) Construct validity of the LapSim

laparoscopic surgical simulator. Am J Surg 191(1):28–32

10. van Dongen KW et al (2007) Construct validity of the LapSim:

Can the LapSim virtual reality simulator distinguish between

novices and experts? Surg Endosc 21(8):1413–1417

11. Stefanidis D et al (2005) Skill retention following proficiency-

based laparoscopic simulator training. Surgery 138(2):165–170

12. Gumbs AA, Hogle NJ, Fowler DL (2007) Evaluation of resident

laparoscopic performance using global operative assessment of

laparoscopic skills. J Am Coll Surg 204(2):308–313

13. Reznick R et al (1997) Testing technical skill via an innovative

‘‘bench station’’ examination. Am J Surg 173(3):226–230

14. Moorthy K et al (2003) Objective assessment of technical skills

in surgery. BMJ 327(7422):1032–1037

15. Seymour NE et al (2002) Virtual reality training improves

operating room performance: results of a randomized, double-

blinded study. Ann Surg 236(4):458–463; discussion 463–4

16. Madan AK et al (2005) Participants’ opinions of laparoscopic

training devices after a basic laparoscopic training course. Am J

Surg 189(6):758–761

Surg Endosc (2008) 22:2018–2025 2025

123