ctiptafei

rwiaaTbs

cstpkpc

i

Peer Teaching and Computer-Assisted Learning: An EffectiveCombination for Surgical Skill Training?

David A. Rogers, M.D.,*,1 Glenn Regehr, Ph.D.,† Mark Gelula, Ph.D.,‡ Karen A. Yeh, M.D.,*Thomas R. Howdieshell, M.D.,* and Whitney Webb, M.D.*

*Department of Surgery, Medical College of Georgia, Augusta, Georgia 30912-4070; †Center for Surgical Research,University of Toronto, Toronto, Ontario, Canada M5G 2C4; and ‡Department of Medical Education,

University of Illinois at Chicago, Chicago, Illinois 60612-7309

Journal of Surgical Research 92, 53–55 (2000)doi:10.1006/jsre.2000.5844, available online at http://www.idealibrary.com on

ati

Submitted for public

Background. The surgical literature suggests thatollaborative learning using peers may be a valid wayo teach surgical skills and there is a growing interestn the use of computer-assisted learning for this pur-ose. Combining this evolving technology with thisype of teaching would theoretically offer a number ofdvantages including a reduction in the amount ofaculty time devoted to this task. In this study, wevaluate the efficacy of a type of collaborative learn-ng in a computer-assisted learning environment.

Materials and methods. We designed a prospective,andomized study comparing novice learners whoere allowed to work in pairs with those who worked

ndependently in a specially equipped computer-ssisted learning classroom. Both pretest and posttestssessments were performed by videotaping this skill.hree experts then evaluated the videotapes, in alinded fashion. Three different outcomes were as-essed.Results. Seventy-seven subjects were enrolled in and

ompleted the study. Comparison of the outcome mea-ures demonstrated no between group difference inhe average performance scores or posttest times. Theroportion of subjects who correctly tied a squarenot was significantly lower in the computer-assistedeer teaching group when compared with theomputer-assisted learning alone group (P 5 0.04).Conclusions. Collaborative learning in a computer-

assisted learning environment is not an effective com-bination for teaching surgical skills to novices.© 2000 Academic Press

Key Words: peer teaching; computer-assisted learn-ng; surgical skill training.

1 To whom correspondence should be addressed.

53

on October 26, 1999

INTRODUCTION

In the discussion of a surgical skills curriculum,Barnes et al. [1] describe a transition from a faculty-supervised experience to one of “self-directed learningby pairs.” The stated motivation for this shift was toreduce the time commitment of the faculty surgeons.This was also one of our primary motivations when webegan to evaluate a potential role for computer-assisted learning (CAL) in teaching surgical skills [2].Our experience with CAL suggested that there areadvantages to this teaching environment. Many learn-ers commented that they enjoyed the self-directed na-ture of this type of session. Further, the learner can seea digitized representation of an expert performing theentire motion of the skill. Despite these advantages,our research suggested that external feedback from acontent expert was a major component of effectiveskills teaching. The report by Barnes et al. [1] raisesthe possibility that peers might provide this importantexternal feedback but we were unable to find an in-stance where this idea had been tested for teachingsurgical psychomotor skills. In this study, our goal wasto examine the impact of peer teaching in a CAL envi-ronment on the acquisition of a basic surgical skill.

METHOD

Approval was obtained from the institutional review board at theMedical College of Georgia. Freshman medical students were en-rolled in the study and each was randomly assigned to either thecomputer-assisted learning alone (CALA) group or the computer-assisted peer teaching (CAPT) group. The subjects were not paid toparticipate in the study but were allowed to keep the knot tyingboard, manual, and rope when the session was complete. Both

groups used a CAL program that has been described previously [2].The subjects were placed in groups of 8 to 10. Subjects in the CALAgroup interacted individually with a single computer in a specially

0022-4804/00 $35.00Copyright © 2000 by Academic Press

All rights of reproduction in any form reserved.

att

gi

bcc0tt

swsasnt

d

ARC

equipped classroom. Subjects in the CAPT group were assigned topairs and each pair interacted with a single computer. At the begin-ning of the session, the subject was instructed to tie his or her “bestand fastest pair of two-handed square knots” and the performancewas videotaped. The subject was then supplied with a knot-tyingboard and tying rope and participated in a 45-min educational ses-sion. In the CALA group, the subjects were instructed not to interactwith one another. In the CAPT group, the pairs of subjects wereadvised that they might interact only with their partner. At the endof the session, the subject was given identical instructions and theposttest performance was videotaped. Finally, a survey was admin-istered to all subjects asking for their opinions about the session. Inthis survey the subjects were asked to rate the overall quality of thesession using a 5-point modified Likert scale.

The three surgical faculty members reviewed each videotapedperformance independently in a blinded fashion. He or she recordedwhether or not the knot was square, and, in cases where all three didnot agree, the majority opinion was recorded. Further, the quality ofthe knot tying was evaluated using a rating scale that explicitlyidentified all of the actions necessary for optimal performance. Thescale was created by dividing the entire skill into eight distinct steps.The faculty member then evaluated each step of the skill using a5-point scale. The sum of these numbers was termed a performancescore, with a maximum value of 32. Finally, the time for the task wasrecorded for each successful performance.

Data were entered in the Paradox 8.0 database (Corel Corp.) andanalyzed using Quattro Pro 8.0 (Corel Corp.) and SPSS 9.0 forWindows (SPSS, Inc). The interrater reliability of the pretest andposttest performance scores was evaluated using Cronbach’s a. Theverage performance scores were compared using a two-way mixed-ype ANOVA. The within-group comparison of the change in propor-ion of knots squared was performed using McNemar’s P. The

between-group comparison of this outcome was compared using x2 ofproportions. Times for successful completion of the test and theaverage global rating by the students were compared using theindependent samples Student t test. A P , 0.05 was consideredstatistically significant.

RESULTS

Seventy-seven subjects were enrolled in and com-pleted the study. Forty subjects were in the CAPTgroup and 37 were in the CALA group. Average age,gender distribution, and handedness were similar forboth groups. The reliability of the pretest and posttestperformance evaluation was 0.72 and 0.78, respec-tively. The ANOVA on the average performance scores(Table 1) evaluating the effect of the session and thegroup revealed a significant main effect of the session

TABLE 1

Performance Scoresa

Group Pretest Posttest

CAPT 2.1 (1.7) 15.1 (6.2)CALA 2.9 (2.3) 16.9 (6.2)

a Values shown are the average performance scores, with standardeviations in parentheses.

54 JOURNAL OF SURGICAL RESE

(F 5 324.78, P , 0.001) but no significant effect ofroup (F 5 2.96, P 5 0.09) and no group 3 sessionnteraction (F 5 0.386, P 5 0.536) suggesting that

oth groups improved to a similar extent on this out-ome. The proportion of knots squared (Table 2) in-reased significantly in both groups (McNemar P ,.001) and comparison of the groups demonstratedhat the CAPT group had a significantly lower propor-ion of knots squared in the posttest evaluation (x2 5

4.08, P 5 0.04). No comparison of time was made inthe pretest group because so few of the subjects couldsuccessfully complete the task. Comparison of the av-erage posttest time demonstrated no significant differ-ence between the CALA and CAPT groups (15.8 vs17.0 s, t 5 0.87, P 5 0.39). Thirty percent of theubjects in the CAPT group reported that interactingith a peer was an important positive aspect of this

ession and no student mentioned that this was a neg-tive aspect of the session. The average global ratingcore of the session by subjects in the CAPT group wasot different from that of the CALA group (3.95 vs 4.08,5 0.83, P 5 0.40).

DISCUSSION

Our results do not support the concept that surgicalskill can be acquired using this type of collaborativelearning in a CAL environment. In fact, we have dem-onstrated that this type of feedback of novices canactually produce worse outcomes when compared witha novice learner working independently. Our resultscan be explained by examining the theories of psy-chomotor skill acquisition and the unique attributes ofcollaborative learning.

It has been suggested that surgical psychomotor skillis acquired in three phases [3]. In the cognitive phase,the learner grasps the concept of the skill. Once theseconcepts are mastered, the learner then moves into theassociative phase where the skill is practiced. Finally,the learner moves to the autonomous phase where theskill is performed without cognitive input. During theassociative phase, the learner compares his or her ownperformance with that of the expert. The differencebetween these two performances is the error. Thelearner attains expertise when the error is minimized.The instructor plays an important role in both the

TABLE 2

Proportion Squareda

Group Pretest PosttestWithin-group

difference

CAPT 0.11 0.60 0.49*CALA 0.10 0.81 0.71*Between-group difference 0.01 0.21**

a Each value shown is the proportion of the group that tied a

H: VOL. 92, NO. 1, JULY 2000

s

quare knot.* Significantly different by McNemar test.

** Significantly different by x2 of proportions.

1

1

1

ND

cognitive and associative phases of motor skill acquisi-tion [3]. During the cognitive stage, the instructor pro-vides the conceptual framework for the task. Duringthe associative phase, the instructor has two functions.First the instructor must assess the skill and contrastit with the expert performance [4]. Second, the instruc-tor must communicate to the learner information aboutthe errors and prescribe steps to eliminate them [5].Our previous work demonstrated the importance offeedback from a content expert in obtaining the opti-mal acquisition of skill. The central question in thisstudy is: Can a peer fulfill this role in a collaborativeteaching environment?

Collaborative learning is an instructional methodthat minimizes the role of the teacher during the teach-ing session [6] and a potential role in medical educationhas been described [7]. In this instance, two truly equalpeers instructing each other is a specific subset of col-laborative learning called reciprocal peer teaching [6].There is evidence that this type of learning is suited tolearning some but not all tasks [8]. Our results suggestthat surgical skills training is not a task for which thistype of learning is well suited. The peer teachers mayhave been unable to sufficiently master the cognitiveaspects of the task to assess their partners’ perfor-mance. The limited information on peer assessment ofmedical psychomotor skill suggests that the ability toassess a performance is acquired with experience [9,10]. Further, even if peer instructors were able to as-sess the error in their partners’ task, they may nothave been able provide instructions for correcting thetask. Based on learner’s comments during our previouswork, we believed that working with a peer wouldproduce a more enjoyable learning experience but ourresults do not provide evidence that this was true.While a substantial number of the subjects in theCAPT group noted that learning in pairs was a positiveaspect of the session, this factor was not sufficient toproduce a higher overall rating of the session whencompared with the CALA group.

ROGERS ET AL.: PEER TEACHING A

We agree with others there will be a future forcomputer-assisted learning in surgical skills training[11] but believe that our results suggest that these

1

tools must be thoroughly evaluated before they areadopted [12].

Traditionally, surgical training has used more ad-vanced learners to teach beginning learners [13]. Theresults of our previous work [2] and this present studyindicate that the level of advancement of the teachermay be a critical element of skills instruction. If theteacher is more like a peer, then poor outcomes can beexpected.

REFERENCES

1. Barnes, R. W., Lang, N. P., and Whiteside, M. F. Halstediantechnique revisited: Innovations in teaching surgical skills.Ann. Surg. 210: 118, 1989.

2. Rogers, D. A., Regehr, G., Yeh, K. A., and Howdieshell, T. R.Computer-assisted learning versus a lecture and feedback sem-inar for teaching a basic surgical technical skill. Am. J. Surg.175: 508, 1998.

3. Kaufman, H. H., Wiegand, R. L., and Tunick, R. H. Teachingsurgeons to operate: Principles of psychomotor skills training.Acta Neurochir. 87: 1, 1987.

4. Peddie, B. K. What underlies the teaching of motor skills. Phys.Educ. 52: 119, 1995.

5. Schmidt, R. A. Motor Learning and Performance: From Princi-ples to Practice. Champaign, IL: Human Kinetics Book, 1991.

6. Cross, K. P., and Steadman, M. H. Classroom Research: Imple-menting the Scholarship of Teaching. San Francisco: Jossey–Bass, 1996.

7. Kleffner, J. H., and Dadian, T. Using collaborative learning indental education. J. Dent. Educ. 61: 66, 1997.

8. Wood, D., and O’Malley, C. Collaborative learning betweenpeers. Educ. Psych. Pract. 11: 4, 1996.

9. Calhoun, J. G., Woolliscroft, J. O., Hockman, E. M., Wolf, F. M.,and Davis, W. K. Evaluating medical student clinical skill per-formance: Relationships among self, peer and expert ratings.Proc. Annu. Conf. Res. Med. Educ. 23: 205, 1984.

0. Risucci, D. A., Tortolani, A. J., and Ward, R. J. Ratings ofsurgical residents by self, supervisors and peers. Surg. Gynecol.Obstet. 169: 519, 1989.

1. Souba, W. W. Reinventing the academic medical center.J. Surg. Res. 81: 113, 1999.

2. Reznick, R. K. Virtual reality surgical simulators: Feasible but

55COMPUTER-ASSISTED LEARNING

valid? J. Am. Coll. Surg. 189: 127, 1999.3. Nolen, W. A. The Making of a Surgeon. New York: Random

House, 1968.