Supporting Procedural and Perceptual Learning in Laparoscopic Surgery

1Lou,Y., 1Flinn, J. T., 1Ganapathy, S., 2Weyhrauch, P., 2Niehaus, J., 2Myers. B., 1Cao, C. G. L.

1Department of Biomedical, Industrial and Human Factors Engineering, Wright State University 2Charles River Analytics, Cambridge, MA

Expertise in surgical performance requires mastery of both technical skills such as suturing, and non-technical skills such as perceptual and procedural knowledge. “Refresher-training” after skill decay due to nonuse should consider the fact that non-technical skills often decay faster than technical skills. To support the re-learning of perceptual and procedural knowledge, this study examined the effectiveness of different design factors for digital training material. The factors considered included modality/fidelity of representation (illustration/cartoon vs. realistic/video images) and task difficulty (easy, medium, and difficult). Results suggest that low fidelity images are better for perceptual learning, and are equally effective as high fidelity images for procedural learning. The level of difficulty of the procedures did not affect performance in this study of novices, but may be an important factor with more experienced trainees. Time and error results indicate that refresher training in perceptual and procedural knowledge should begin with a procedural task to review surgical steps, followed by a perceptual task, to achieve greater efficiency and effectiveness.

INTRODUCTION

Despite the invention of laparoscopic surgery in the early 1900s, it was not widely used until the 1980s (Spaner, & Warnock, 1997). Now, almost 98% of cholecystectomies are laparoscopic (Wayand, 2004). Laparoscopic surgery has become more common according to the US Food and Drug Administration (FDA), and more than 2 million Americans each year experience laparoscopic surgery (Fuller, Scott, Ashar, & Corrado, 2003). Reductions in post-op pain, incisions, and chance of hemorrhage are all reasons why laparoscopic surgery is preferred over other alternatives. However, laparoscopic surgery skills require additional training over the skills needed to perform traditional open surgery. A great deal of past research has focused on the training of technical skills, such as suturing and knot-tying (Ritter, & Scott, 2007), resulting in standardized certification programs such as the Fundamentals of Laparoscopic Surgery (FLS) curriculum that is endorsed by the American College of Surgeon (ACS) and the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES). This paper concerns the non-technical aspect of surgical training, such as perceptual and procedural skills, as non-technical skills play an equally important role in determining surgical skill acquisition and retention (Yule, Flin, Patterson-Brown, Maran, & Rowley, 2006).

Despite the long training schedules and constant practice, skill decay is observed after periods of nonuse or non-practice (Arthur, Bennett, Stanush, & Mcnelly, 1998). According to the meta-analysis of Arthur et al. (1998), after more than 365 days of nonuse and non-practice, the average participant was performing at less than 92% of their performance level before the non-practice interval. The neuroscience of human memory indicates that there are three types of abilities (job knowledge, decision skills, and execution skills) that are located and controlled in different areas of the brain (Gabrieli, 1998). The

job knowledge category is based on the recall of domain specific information, such as job-related terms and rules; the decision category hinges on cognitive processing of the domain specific information such as trouble-shooting faulty equipment and decision-making (Allen, Secundo, Salas, & Morgan, 1983), and the execution category refers to both the perceptual and motor requirement of a task such as target acquisition and tracking (Fleishman, & Parker, 1962). A study by Wisher, Sabol, Ellis, and Ellis (1999) showed that the patterns of forgetting that occur in decision skills, procedural and perceptual-motor skills are different. For example, gross motor skills decayed after approximately 10 months while cognitive skills such as knowledge of procedures decayed within approximately 6 months.

In military medicine, loss of specialized skill and knowledge is a great concern for those military surgeons returning to general practice after deployment. It takes a huge amount of time for the personnel in the US military medical community (as in the civilian community) to learn knowledge and skills to perform lifesaving tasks. However, it is difficult for military surgeons to maintain their specialized knowledge and ability after a period of time performing combat casualty care in a military deployment cycle (Perez et al., 2013).

The ultimate goal of this research is to develop an effective refresher training system that is appropriate for surgeons with varying degrees of skill decay. The purpose of this study was to determine how well re-learning perceptual and procedural knowledge in laparoscopic surgery (i.e., identifying anatomy and knowing operational sequences) can be supported by digital material after skill decay. The fidelity and complexity of the digital re-learning material needed to support learning/re-learning was examined.

We hypothesized that simulation fidelity would have differential effects on the learning and retention of perceptual and procedure knowledge in laparoscopic surgery. That is, (1) realistic visual representation of the surgical site is more

important for learning and remembering perceptual knowledge than for procedural knowledge and (2) a less realistic-looking visual representation of the surgical site is sufficient for the purpose of learning and remembering procedural steps in surgery.

METHODS

The objective of this study was to evaluate how two

types of skills (perceptual and procedural) were affected by two types of information presentation (illustration representation vs. realistic representation), and three levels of task complexity (easy, medium, and difficult).

Based on a cognitive task analysis with experts, who were surgeons with over 20 years of experience, a set of tasks in the laparoscopic cholecystectomy procedure was identified (Grosdemouge, Weyhrauch, Niehaus, Schwaitzberg, & Cao, 2012). These tasks included perceptual skills and procedural skills. Perceptual tasks related to identifying anatomy, identification of points where incision had to be made, and angle of presentation of the image. Procedural tasks included identifying steps and sequences in the cholecystectomy procedure. These tasks were used to design the teaching material and the refresher material in this experiment. Participant

Twelve participants (6 male, 6 female) were recruited from the general public. Participants were novices who had no previous medical or surgical experience. The average age was 31.5 years. Apparatus

Teaching material was compiled based on the text Atlas of Minimally Invasive Surgery (Jones, Maithel, & Schneider, 2006). This material covered the basic anatomy and procedure for laparoscopic cholecystectomy and was prepared using Microsoft PowerPoint.

The images, videos, and questions presented during the experimental task were displayed using a custom-developed software program, which also recorded participants' responses automatically. This software, the Mobile Interactive Storyboard Tool (MIST), was developed in C++ using the Microsoft XNA framework and Direct X API (application programming interfaces). The MIST software allowed for the customizable arrangement of media and interactive elements required for the experiment. Task and Procedure

Initially, participants were shown a 10-minute PowerPoint presentation (teaching material) with a brief introduction of the relevant anatomy, the surgical steps and instrumentations used in a cholecystectomy procedure. At the end of the presentation, they were required to answer three basic questions based on what they have learned. If any one of the answers was incorrect, participants had to repeat the learning

process until they had all three correct answers. This ensured that participants had in fact learned the material presented.

To ensure that all trials are performed based on longer term memory rather than short-term memory, participants were asked to work on a Sudoku puzzle for 10 minutes following the training period. Following this, participants performed the experimental task (refresher task), which involved answering a series of questions about anatomy or steps in a laparoscopic cholecystectomy procedure, depending on the experimental condition (Table 1). The questions were presented at various points in the cholecystectomy procedure by the MIST program.

Participants responded to the questions using the mouse by clicking on specific areas in the scene, or selecting options in a multiple choice question. The program indicated whether they were correct or incorrect after they gave their response. If incorrect, participants were given a second chance to answer the question.

In this study, two representations of different fidelity levels (represented by two different modalities) were used to present the refresher material: illustrated and realistic. Figure 1 shows the screenshot of the illustrated condition and realistic condition. Three different levels of task difficulty (easy, medium and difficult), representing the difficulty of the surgical case, were used (Figure 2).

Six participants began with the perceptual condition first and the other 6 participants had the procedural condition first (randomly assigned). Within each condition, all combinations of the modalities and difficulty of the surgical case were presented in a random order. At the end of the experiment, participants were asked for subjective feedback on their preference of realism of the surgical scenes, and any comments on their choices during the experiment.

Table 1 Example Questions in Perceptual and Procedural Conditions

Perceptual Condition Procedural Condition

Locate the cystic artery by clicking the correct structure.

What is the next step in the cholecystectomy procedure?

A. Clip the Cystic Artery B. Clip the Cystic Duct C. Divide the Cystic Artery D. Divide the Cystic Duct

Click on the cystic artery to indicate where clips should be placed.

Choose how many clips are needed on the cystic artery?

A. 1 B. 2 C. 3 D. 4 E. 5

How will the clips be positioned on the cystic artery?

Figure 1. Sample screenshots of the Illustration Representation (top) and Realistic Representation (bottom) of a surgical step

in Laparoscopic Cholecystectomy.

Figure 2. Sample screenshots of the Easy (top), Medium

(middle), and Difficult (bottom) representations of Laparoscopic Cholecystectomy measures.

Time to task completion was determined as the response

time for answering the questions in each condition. Time for playing embedded videos was excluded from this measure. Error was the total number of incorrect responses for each trial.

RESULTS

Performance data in the perceptual and procedural conditions were separately analyzed using repeated

measurements 3-way ANOVA with order, fidelity and difficulty level as factors. Perceptual Knowledge

Error. Only fidelity (F=5.32, p=0.04) and task difficulty (F=11.22, p<0.01) showed statistically significant differences in terms of error in the perceptual condition. Order (F=4.09, p=0.07) didn’t show a significant difference. For fidelity, there were significantly more errors when the images were realistic (mean=5.44, SD=0.57) than when they were illustrations (mean=3.63, SD=0.57). For task difficulty, a post-hoc Tukey HSD test showed that only the easy (mean=3.48, SD=2.81) level can be differentiated from the difficult level (mean=4.54, SD=2.86). As shown in Figure 3, there was an increase in error in the case of difficult when compared to easy and medium. However, no significant difference exists between easy and medium, and between medium and difficult.

Figure 3. Error rate in the perceptual condition as a function of

difficulty level.

Time. Results indicate that order (F=9.81, p=0.01) and fidelity (F=8.76, p=0.01) had a significant effect on task completion time. In terms of order, participants who completed the perceptual condition first (mean=134.8, SD=55.8) were much slower than participants who completed the procedural condition first (mean=90.5, SD=53.8). In term of fidelity, participants spent longer time on the realistic pictures (mean=127.9, SD=54.8) than the illustrated pictures (mean=97.4, SD=55.4), as shown in Figure 4.

0

50

100

150

200

Illustration Realistic

Tim

e(se

cond

s)

Fidelity

Perceptual - TimePerceptual-Procedural

Procedural-Perceptual

Figure 4. Task completion time for the perceptual condition as a function of fidelity and order.

Procedural Knowledge

Error. In the error of procedural tasks, no significant main effect was found.

Time. For the procedural conditions, the interaction effect between fidelity and difficulty (F=4.44, p=0.03) was statistically significant. However, a post-hoc Tukey HSD test showed none of the conditions could be differentiated from one another in the experiment (Figure 5).

0

50

100

150

Easy Medium Difficult

Tim

e(se

cond

s)

Difficulty

Procedural - TimeIllustrationRealistic

Figure 5. Task completion time in the procedural condition as a function of difficulty and fidelity level.

DISCUSSION

In the perceptual condition as difficulty level was

increased, the number of errors also increased. This is not very surprising, as it is more difficult to identify the critical structures when the surgical field is bloodier and poorly dissected. Experts also mentioned this during the cognitive task analysis that knowledge and expertise play a bigger role in those cases that are complicated. There was no significant difference in procedural error amongst the difficulty levels in the procedural condition. This was also expected because procedural knowledge about a surgical operation should not be

affected by the perceptual difficulty of the case. However, in reality, the complicated cases may very well warrant a different approach that alters the sequence of surgical steps. In our study, we had limited our teaching material to a routine procedure for complete novices, so that variations in surgical procedure were not a consideration. A refresher training system may consider including variations in procedure to accommodate more experienced trainees. In the perceptual condition, participants who started in the perceptual condition first spent much longer than participants who performed the procedural condition first. The reason could be that participants who follow the order of “procedural-perceptual” had already performed 6 trials in the procedural condition before they entered the perceptual condition. They were thus familiar with the procedure of cholecystectomy, which also contained knowledge of the anatomy implicitly, if not explicitly. However, there was no effect of order in the procedural time. This suggests that training/re-training in procedural knowledge before perceptual knowledge would be much more efficient than training/re-training perceptual knowledge first before procedural knowledge. In fact, it may not be necessary for the perceptual condition after re-training in the procedural condition. When it comes to fidelity of training material, participants spent a longer time and have more errors in the realistic conditions compared to the illustrated conditions in the perceptual condition. There is no doubt that visualization of anatomical structures is much more discernible in the illustrated representation. There was no difference between time or errors of the illustration and realistic conditions in the procedural condition. A possible explanation is that procedural condition was asking participants what is the next step. As long as the participants knew what the next step was, there was no need for them to spend time on distinguishing those structures. Whereas in the perceptual condition, they needed to spend time to identify the structures carefully and click on the correct location in the picture. Therefore, illustrated representation of structures should be used in re-training perceptual knowledge. For re-training procedural knowledge, realism of the surgical site has no effect on relearning.

CONCLUSION

Based on this initial study, refresher training for

laparoscopic surgery should not decouple perceptual and procedural skills, especially for less experienced trainees. For an effective training system, procedural knowledge should be presented before perceptual knowledge. Also, the use of illustration vs. realistic image is important as it allows educators to quickly put together multiple scenarios using illustration which can also be used effectively for procedural tasks. This would reduce the time and resources required to create the training material. Potential future work includes using larger sample sizes to determine the interaction effects between the variables of interest; testing the current experimental design with surgeons who have been deployed and returned to identify skill decay parameters. We will also

examine performance differences on perceptual and procedural tasks in an interactive laparoscopic simulator.

REFERENCES

Allen, G., Secundo, M., Salas, E., & Morgan, B., Jr (1983). Evaluation of rate

parameters of the acquisition, decay, and reacquisition of complex cognitive skills (Technical Report ITR-82-27).

Arthur, W., Bennett, W., Stanush, P. L., & Mcnelly, T. L (1998). Factors that influence skill decay and retention: a quantitative review and analysis. Human performance,11(1), 57-101.

Fleishman, E.,& Parker, J.,Jr. (1962). Factors in the retention and relearning of perceptual motor skill. Journal of Experimental Psychology, 64, 205-226.

Fuller, J., Scott, W., Ashar, B., & Corrado, J. (2003). Laparoscopic trocar injuries: A report from a U.S. Food and Drug Administration (FDA) Center for Devices and Radiological Health (CDRH) Systematic Technology Assessment of Medical Products (STAMP) Committee. Retrieved from http://www.fda.gov.

Gabrieli, J. (1998). Cognitive neuroscience of human memory. Annual Review of Psychology, 49,87-115.

Grosdemouge, C., Weyhrauch, P., Niehaus, J., Schwaitzberg, S., & Cao, C. G. L. (2012). Design of training protocol for perceptual and technical skills in minimally invasive surgery. 2012 Proceedings of the ASME 11th Biennial Conference on Engineering Systems Design and Analysis (ESDA 2012), 855-860.

Jones, D., Maithel, S., Schneider,B.(2006). Biliary Surgery: Cholecystectomy, Atlas of Minimally Invasive Surgery (pp. 11 – 31). Woodbury, CT: Cine-Med.

Perez, R. S., Skinner, A., Weyrauch, P., Niehaus, J., Lathan, C., Schwaitzberg, S. D., Cao, C. G. L. (2013). Prevention of surgical skill decay. Military Medicine, Special Issue on Designing and Using Computer Simulations in Medical Education and Training, 178, 76-86.

Ritter, E.M., Scott DJ (2007). Design of a Proficiency-Based Skills Training Curriculum for the Fundamentals of Laparoscopic Surgery. Surgical Innovation, 14(2),107-112.

Spaner, S.J., Warnock, G.L. (1997). A brief history of endoscopy, laparoscopy, and laparoscopic surgery. Journal of Laparoendoscopic & Advanced Surgical Techniques, 7(6), 369-73.

Wayand,W. (2004). The history of Minimally Invasive Surgery. Business Briefing: Global Surgery.

Retrieved from http://www.touchbriefings.com/pdf/952/Wayand.pdf Wisher,R.A., Sabol,M.A., Ellis,J.&Ellis,K. (1999). Staying Sharp: retention of

military knowledge and skills.US Army Research Institute for Social and Behavioral Sciences. (Rep.No.ARI Special Report 39).

Yule, S., Flin, R., Patterson-Brown, S., Maran, N., & Rowley, D. (2006). Development of a rating system for surgeons’ non-technical skills. Medical Education, 40(11), 1098-1104.