Peer assisted learning in surgical skills laboratory training: A pilot study

  • Published on
    28-Feb-2017

  • View
    212

  • Download
    0

Embed Size (px)

Transcript

<ul><li><p>2012; 34: 957959</p><p>SHORT COMMUNICATION</p><p>Peer assisted learning in surgical skillslaboratory training: A pilot study</p><p>J.H. BEARD, P. OSULLIVAN, B.J.A. PALMER, M. QIU &amp; E.H. KIM</p><p>UCSF, USA</p><p>Abstract</p><p>Background: Peer assisted learning (PAL) has been rarely investigated in surgical skills training.</p><p>Aims: Describe feedback residents give peers on surgical skills with and without guidelines, determine the association between</p><p>feedback and actual performance, evaluate resident satisfaction with PAL.</p><p>Method: Participants evaluated peers using a validated knot-tying checklist and provided feedback on suturing without a</p><p>guideline. Feedback comments were coded by type and an expert scored performance of each participant. Residents completed a</p><p>satisfaction questionnaire.</p><p>Results: Comments were generally specific. Feedback was twice as likely with the use of a guideline. Specific feedback correlated</p><p>significantly with expert knot-tying score but not suturing score. Most participants felt peer feedback was helpful and were</p><p>motivated to practice surgical skills after PAL sessions.</p><p>Conclusions: Surgical residents can provide high quality specific feedback to peers on surgical skills using performance</p><p>guidelines. Further exploration of effective PAL methodology in surgical skills laboratory training is needed.</p><p>Introduction</p><p>Informal peer teaching is at the core of general surgical</p><p>education in the United States. Inside and out of the operating</p><p>room, senior residents educate and shape the development of</p><p>juniors through role-modeling, direct instruction, and feed-</p><p>back. Although peer assisted learning (PAL) is emerging as a</p><p>valuable tool in medical education, there has been little</p><p>investigation of the formal use of PAL in the development of</p><p>surgical skills.</p><p>Peer teachers are thought to be cognitively congruent</p><p>with their students and thus theoretically better than experts at</p><p>understanding the knowledge and skill level of their trainees</p><p>(Lockspeiser et al. 2008). This similarity in educational</p><p>experience and technical skill levels between student and</p><p>teacher can enhance motivation and achievement through</p><p>modeling of attainable learning objectives (Ross &amp; Cameron</p><p>2007). In addition, PAL promotes the development of the tutor</p><p>as a teacher, a lifelong skill required of academic physicians.</p><p>Finally, PAL can solidify the knowledge and skill of the tutor</p><p>through the instruction of others in a supported, socially</p><p>stimulating learning environment (Gibson &amp; Campbell 2000).</p><p>A recent randomized comparison of resident versus peer</p><p>medical student teaching during an operating room introduc-</p><p>tion course demonstrated better technical performance by</p><p>students in the peer-teaching group (Graziano 2011). Evans</p><p>and colleagues found that peer assessment scores reflect</p><p>expert trainer scores more accurately that surgeon self-</p><p>assessment in dental extraction (Evans et al. 2007). In addition</p><p>to being accurate, the authors suggest that peer assessment</p><p>may encourage honest reflection on skills, increased</p><p>collaboration, and teamwork. Although the use of guidelines</p><p>in PAL has been advocated in the literature, there are no</p><p>definitive studies on the issue (Ross &amp; Cameron 2007).</p><p>We piloted a PAL program in our surgical skills lab. The</p><p>aims of this study were: describe the nature of feedback</p><p>residents give each other on basic surgical skills with and</p><p>without guidelines, determine an association between peer</p><p>feedback an actual performance, and evaluate resident satis-</p><p>faction with PAL.</p><p>Methods</p><p>We conducted this descriptive study at the Surgical Skills Lab</p><p>of the University of California, San Francisco (UCSF). The</p><p>UCSF Committee on Human Research certified the study as</p><p>exempt from review. Thirty-three general surgery residents in</p><p>their first year of training participated in the knot-tying session,</p><p>while twenty residents participated in the suturing session.</p><p>During the sessions, residents either completed a series of two</p><p>handed knots or a simple suturing exercise and received peer</p><p>feedback upon completion of the task. Participants were</p><p>instructed on the use of a previously validated Knot-tying</p><p>Checklist for feedback in the knot-tying session (Kim 2011).</p><p>There were no guidelines given for the suturing session. PAL</p><p>sessions were videotaped. Care was taken to ensure anonym-</p><p>ity of the participants.</p><p>Researchers transcribed and coded each participant com-</p><p>ment as global positive, specific positive, global negative and</p><p>specific negative following a previously employed methodol-</p><p>ogy (Kruidering et al. 2009). Global comments were defined as</p><p>Correspondence: Jessica H. Beard, 513 Parnassus Ave, S-321, San Francisco, CA 94143-0470, USA. Tel: 713.818.1870; fax: 415.502.1259;</p><p>email: jessica.beard@ucsfmedctr.org</p><p>ISSN 0142159X print/ISSN 1466187X online/12/110957959 2012 Informa UK Ltd. 957DOI: 10.3109/0142159X.2012.706340</p><p>Med</p><p> Tea</p><p>ch D</p><p>ownl</p><p>oade</p><p>d fr</p><p>om in</p><p>form</p><p>ahea</p><p>lthca</p><p>re.c</p><p>om b</p><p>y SU</p><p>NY</p><p> Sta</p><p>te U</p><p>nive</p><p>rsity</p><p> of </p><p>New</p><p> Yor</p><p>k at</p><p> Sto</p><p>ny B</p><p>rook</p><p> on </p><p>10/2</p><p>7/14</p><p>For </p><p>pers</p><p>onal</p><p> use</p><p> onl</p><p>y.</p></li><li><p>feedback related to overall impression of the activity, while</p><p>specific comments included feedback on particular technical</p><p>aspects of performance. An attending surgeon scored resident</p><p>performance using the validated Knot-tying Checklist and a</p><p>separate suturing checklist. Residents completed an anony-</p><p>mous satisfaction questionnaire.</p><p>We used SPSS Version 17.0 for data analysis. Differences</p><p>between the mean number of each type of peer comment in</p><p>the knot-tying and suturing categories were identified using a</p><p>t-test. We calculated Pearson correlations to determine the</p><p>relationship between peer feedback and expert scores.</p><p>Results</p><p>Most peer feedback was either specific positive (34%) or</p><p>specific negative (41%) but rarely global positive (23%) or</p><p>global negative (2%). Your [knot-tying] distance was perfect</p><p>represents an example of a specific positive comment while</p><p>You didnt gather the suture is an example of a specific</p><p>negative comment.</p><p>Table 1 illustrates the mean number of types of comments</p><p>in the knot-tying and suturing PAL sessions. On average, peer</p><p>teachers made 7.18 comments (95% CI: 5.558.81) during the</p><p>knot-tying sessions using a teaching guide and 3.25 comments</p><p>(95% CI: 2.533.97) during the suturing sessions, which were</p><p>unguided. The amount of feedback in knot-tying sessions</p><p>using a guideline more than doubled when compared to</p><p>suturing sessions without teaching guidelines (p5 0.001).Specific positive and negative comments were also more</p><p>common than global comments in the knot-tying group.</p><p>In the knot-tying group, expert score was positively</p><p>associated with the proportion of specific positive feedback</p><p>(r 0.48, p 0.003) and negatively associated with theproportion of specific negative feedback (r0.35,p 0.028). There was no significant association identifiedbetween expert suturing score and peer feedback (positive</p><p>comments: r0.03, p 0.46; negative comments: r0.08,p 0.38).</p><p>Most participants (88.5%) felt the PAL sessions were helpful</p><p>in pointing out technical abilities while 65.7% of residents felt</p><p>more confident with their technical skills upon completion of</p><p>the exercises. Residents (80.0%) felt motivated to practice their</p><p>surgical skills at home after the PAL session, and 71.4% noted</p><p>an increased sense of peer camaraderie. 42.8% of participants</p><p>stated that the PAL session engendered a feeling of anxiety to</p><p>perform well.</p><p>Discussion</p><p>In this pilot study, we created a picture of peer feedback in the</p><p>context of our surgical skills training laboratory. We found that</p><p>residents generally provided specific feedback to peers in PAL</p><p>sessions. Participants rarely made global negative comments,</p><p>which would likely be of little use in peer instruction and may</p><p>serve to create a hostile learning environment. Although the</p><p>correlations are modest, our results show that with the</p><p>assistance of a guideline on knot-tying tasks, peer teachers</p><p>did indeed provide accurate and specific feedback.</p><p>The PAL sessions provided a unique opportunity for</p><p>residents to assess their skills in the larger context of their</p><p>peers. We found that PAL may even stimulate independent</p><p>learning and foster motivation to practice skills outside of the</p><p>lab. Despite a potentially competitive learning environment,</p><p>the majority of PAL participants endorsed a sentiment of</p><p>resident camaraderie after the sessions, a feeling that could</p><p>translate to stronger team relationships in the realm of patient</p><p>care. Finally, nearly half of the residents identified feelings of</p><p>stress associated with the PAL activity, which can aid in</p><p>memory formation and reinforce learning at low levels</p><p>(Friedlander et al. 2011). Higher levels of anxiety may be</p><p>detrimental to learning, and further investigation into stress</p><p>effects of peer learning is warranted.</p><p>Our findings contrast with the sentinel study on peer</p><p>teaching in surgery by Rogers et al. (2000). In this study,</p><p>addition of peer collaboration to a computer-based knot-tying</p><p>curriculum had a negative impact on first year medical</p><p>students ability to tie knots. Differences in design and</p><p>outcome measurement may explain the divergent results of</p><p>our studies. While the endpoint of the Rogers study was</p><p>performance, we focused on the quality and accuracy of peer</p><p>feedback. In addition, Rogers et al grouped peers in pairs,</p><p>which may increase the risk of faulty interpretation of</p><p>instructions on knot-tying. Our peer groups included three to</p><p>five participants to ensure appropriate group communication.</p><p>Finally, Rogers et al evaluated the performances of first year</p><p>medical students after a single 45minute session. Our partic-</p><p>ipants were first year residents with more practical experience</p><p>with basic surgical skills than novice medical students.</p><p>Therefore PAL may be more productive at more advanced</p><p>stages of technical skills acquisition.</p><p>This study has limitations. We compared two different</p><p>surgical skills in the PAL sessions, using a guideline for the PAL</p><p>exercise on knot-tying and no guideline for the suturing</p><p>session. Participants may have been more comfortable giving</p><p>feedback in the knot-tying PAL session owing to an inherent</p><p>greater complexity of suturing. Thus peer teacher facility with</p><p>each skill may have influenced the number and type of</p><p>feedback comments, making it difficult to extrapolate the</p><p>effect of the guideline itself. In addition, the suture scoring</p><p>system used by the expert in our study has not been validated</p><p>in this population of surgical residents, which may have further</p><p>influenced the relationship between guideline use and expert</p><p>scoring.</p><p>Table 1. Comment types in knot-tying and suturing PAL activities.</p><p>Mean number of comments (95% CI)</p><p>Comment typeKnot-tying/guide</p><p>usedSuturing/no guide</p><p>used p-value</p><p>Global positive 1.16 (0.78, 1.58) 0.85 (0.40, 1.30) 0.20</p><p>Specific positive 3.24 (2.13, 4.35) 0.90 (0.39, 1.41) 0.002</p><p>Global negative 0.09 (0.09, 0.27) 0.15 (0.02, 0.32) 0.66Specific negative 2.67 (1.79, 3.55) 1.35 (0.92, 1.78) 0.008</p><p>Total comments 7.18 (5.55, 8.81) 3.25 (2.53, 3.97) 50.001</p><p>J.H. Beard et al.</p><p>958</p><p>Med</p><p> Tea</p><p>ch D</p><p>ownl</p><p>oade</p><p>d fr</p><p>om in</p><p>form</p><p>ahea</p><p>lthca</p><p>re.c</p><p>om b</p><p>y SU</p><p>NY</p><p> Sta</p><p>te U</p><p>nive</p><p>rsity</p><p> of </p><p>New</p><p> Yor</p><p>k at</p><p> Sto</p><p>ny B</p><p>rook</p><p> on </p><p>10/2</p><p>7/14</p><p>For </p><p>pers</p><p>onal</p><p> use</p><p> onl</p><p>y.</p></li><li><p>Conclusions</p><p>Surgical residents can provide high quality specific feedback to</p><p>peers on surgical skills using performance guidelines. Given</p><p>the satisfaction with PAL and quality of peer feedback, we</p><p>recommend developing guidelines for peer assessment of</p><p>surgical skills along with further exploration of effective peer</p><p>teaching methodology in surgical skills laboratory training.</p><p>Declaration of interest: The authors report no declarations</p><p>of interest.</p><p>Notes on contributors</p><p>JESSICA H. BEARD, MD, MPH is a general surgery resident at UCSF.</p><p>PATRICIA OSULLIVAN, EdD is Professor of Medicine and Director of</p><p>Research and Development in Medical Education at UCSF.</p><p>BARNARD PALMER, MD, MEd completed general surgery residency at</p><p>UCSF-East Bay program and is an Endocrine Surgery fellow at Mount Sinai</p><p>School of Medicine.</p><p>MARY QIU is a medical student at UCSF.</p><p>EDWARD H KIM, MD is Assistant Professor of Surgery and Director of the</p><p>Surgical Skills Lab at UCSF.</p><p>References</p><p>Evans AW, Leeson RM, Petrie A. 2007. Reliability of peer and self-</p><p>assessment scores compared with trainers scores following third molar</p><p>surgery. Med Educ 41(9):866872.</p><p>Friedlander MJ, Andrews L, Armstrong EG, Aschenbrenner C, Kass JS,</p><p>Ogden P, Schwartzstein R, Viggiano TR. 2011. What can medical</p><p>education learn from the neurobiology of learning? Acad Med: J Assoc</p><p>Am Med Coll 86(4):415420.</p><p>Gibson DR, Campbell RM. 2000. The role of cooperative learning in the</p><p>training of junior hospital doctors: A study of paediatric senior house</p><p>officers. Med Teach 22(3):297300.</p><p>Graziano SC. 2011. Randomized surgical training for medical students:</p><p>Resident vs. peer-led teaching. Am J Obstet Gynecol 204(6):541542.</p><p>Kim EH. 2011. Validation of the UCSF knot-tying efficiency checklist,</p><p>Unpublished manuscript.</p><p>Kruidering M, Chou C, OSullivan P. 2009. Teaching feedback to first year</p><p>medical students: Long term skill retention and accuracy of student self-</p><p>assessment. J Gen Internal Med 24:721726.</p><p>Lockspeiser TM, OSullivan P, Teherani A, Muller J. 2008. Understanding</p><p>the experience of being taught by peers: The value of social and</p><p>cognitive congruence. Adv Health Sci Educ, Theory Practice</p><p>13(3):361372.</p><p>Rogers DA, Regehr G, Gelula M, Yeh KA, Howdieshell TR, Webb W. 2000.</p><p>Peer teaching and computer-assisted learning: An effective combination</p><p>for surgical skill training? J Surg Res 92(1):5355.</p><p>Ross MT, Cameron HS. 2007. Peer assisted learning: A planning and</p><p>implementation framework: AMEE Guide no. 30. Med Teach 29(6):</p><p>527545.</p><p>Peer assisted learning in surgical training</p><p>959</p><p>Med</p><p> Tea</p><p>ch D</p><p>ownl</p><p>oade</p><p>d fr</p><p>om in</p><p>form</p><p>ahea</p><p>lthca</p><p>re.c</p><p>om b</p><p>y SU</p><p>NY</p><p> Sta</p><p>te U</p><p>nive</p><p>rsity</p><p> of </p><p>New</p><p> Yor</p><p>k at</p><p> Sto</p><p>ny B</p><p>rook</p><p> on </p><p>10/2</p><p>7/14</p><p>For </p><p>pers</p><p>onal</p><p> use</p><p> onl</p><p>y.</p></li></ul>