Peer assisted learning in surgical skills laboratory training: A pilot study

  • Published on
    28-Feb-2017

  • View
    212

  • Download
    0

Transcript

  • 2012; 34: 957959

    SHORT COMMUNICATION

    Peer assisted learning in surgical skillslaboratory training: A pilot study

    J.H. BEARD, P. OSULLIVAN, B.J.A. PALMER, M. QIU & E.H. KIM

    UCSF, USA

    Abstract

    Background: Peer assisted learning (PAL) has been rarely investigated in surgical skills training.

    Aims: Describe feedback residents give peers on surgical skills with and without guidelines, determine the association between

    feedback and actual performance, evaluate resident satisfaction with PAL.

    Method: Participants evaluated peers using a validated knot-tying checklist and provided feedback on suturing without a

    guideline. Feedback comments were coded by type and an expert scored performance of each participant. Residents completed a

    satisfaction questionnaire.

    Results: Comments were generally specific. Feedback was twice as likely with the use of a guideline. Specific feedback correlated

    significantly with expert knot-tying score but not suturing score. Most participants felt peer feedback was helpful and were

    motivated to practice surgical skills after PAL sessions.

    Conclusions: Surgical residents can provide high quality specific feedback to peers on surgical skills using performance

    guidelines. Further exploration of effective PAL methodology in surgical skills laboratory training is needed.

    Introduction

    Informal peer teaching is at the core of general surgical

    education in the United States. Inside and out of the operating

    room, senior residents educate and shape the development of

    juniors through role-modeling, direct instruction, and feed-

    back. Although peer assisted learning (PAL) is emerging as a

    valuable tool in medical education, there has been little

    investigation of the formal use of PAL in the development of

    surgical skills.

    Peer teachers are thought to be cognitively congruent

    with their students and thus theoretically better than experts at

    understanding the knowledge and skill level of their trainees

    (Lockspeiser et al. 2008). This similarity in educational

    experience and technical skill levels between student and

    teacher can enhance motivation and achievement through

    modeling of attainable learning objectives (Ross & Cameron

    2007). In addition, PAL promotes the development of the tutor

    as a teacher, a lifelong skill required of academic physicians.

    Finally, PAL can solidify the knowledge and skill of the tutor

    through the instruction of others in a supported, socially

    stimulating learning environment (Gibson & Campbell 2000).

    A recent randomized comparison of resident versus peer

    medical student teaching during an operating room introduc-

    tion course demonstrated better technical performance by

    students in the peer-teaching group (Graziano 2011). Evans

    and colleagues found that peer assessment scores reflect

    expert trainer scores more accurately that surgeon self-

    assessment in dental extraction (Evans et al. 2007). In addition

    to being accurate, the authors suggest that peer assessment

    may encourage honest reflection on skills, increased

    collaboration, and teamwork. Although the use of guidelines

    in PAL has been advocated in the literature, there are no

    definitive studies on the issue (Ross & Cameron 2007).

    We piloted a PAL program in our surgical skills lab. The

    aims of this study were: describe the nature of feedback

    residents give each other on basic surgical skills with and

    without guidelines, determine an association between peer

    feedback an actual performance, and evaluate resident satis-

    faction with PAL.

    Methods

    We conducted this descriptive study at the Surgical Skills Lab

    of the University of California, San Francisco (UCSF). The

    UCSF Committee on Human Research certified the study as

    exempt from review. Thirty-three general surgery residents in

    their first year of training participated in the knot-tying session,

    while twenty residents participated in the suturing session.

    During the sessions, residents either completed a series of two

    handed knots or a simple suturing exercise and received peer

    feedback upon completion of the task. Participants were

    instructed on the use of a previously validated Knot-tying

    Checklist for feedback in the knot-tying session (Kim 2011).

    There were no guidelines given for the suturing session. PAL

    sessions were videotaped. Care was taken to ensure anonym-

    ity of the participants.

    Researchers transcribed and coded each participant com-

    ment as global positive, specific positive, global negative and

    specific negative following a previously employed methodol-

    ogy (Kruidering et al. 2009). Global comments were defined as

    Correspondence: Jessica H. Beard, 513 Parnassus Ave, S-321, San Francisco, CA 94143-0470, USA. Tel: 713.818.1870; fax: 415.502.1259;

    email: jessica.beard@ucsfmedctr.org

    ISSN 0142159X print/ISSN 1466187X online/12/110957959 2012 Informa UK Ltd. 957DOI: 10.3109/0142159X.2012.706340

    Med

    Tea

    ch D

    ownl

    oade

    d fr

    om in

    form

    ahea

    lthca

    re.c

    om b

    y SU

    NY

    Sta

    te U

    nive

    rsity

    of

    New

    Yor

    k at

    Sto

    ny B

    rook

    on

    10/2

    7/14

    For

    pers

    onal

    use

    onl

    y.

  • feedback related to overall impression of the activity, while

    specific comments included feedback on particular technical

    aspects of performance. An attending surgeon scored resident

    performance using the validated Knot-tying Checklist and a

    separate suturing checklist. Residents completed an anony-

    mous satisfaction questionnaire.

    We used SPSS Version 17.0 for data analysis. Differences

    between the mean number of each type of peer comment in

    the knot-tying and suturing categories were identified using a

    t-test. We calculated Pearson correlations to determine the

    relationship between peer feedback and expert scores.

    Results

    Most peer feedback was either specific positive (34%) or

    specific negative (41%) but rarely global positive (23%) or

    global negative (2%). Your [knot-tying] distance was perfect

    represents an example of a specific positive comment while

    You didnt gather the suture is an example of a specific

    negative comment.

    Table 1 illustrates the mean number of types of comments

    in the knot-tying and suturing PAL sessions. On average, peer

    teachers made 7.18 comments (95% CI: 5.558.81) during the

    knot-tying sessions using a teaching guide and 3.25 comments

    (95% CI: 2.533.97) during the suturing sessions, which were

    unguided. The amount of feedback in knot-tying sessions

    using a guideline more than doubled when compared to

    suturing sessions without teaching guidelines (p5 0.001).Specific positive and negative comments were also more

    common than global comments in the knot-tying group.

    In the knot-tying group, expert score was positively

    associated with the proportion of specific positive feedback

    (r 0.48, p 0.003) and negatively associated with theproportion of specific negative feedback (r0.35,p 0.028). There was no significant association identifiedbetween expert suturing score and peer feedback (positive

    comments: r0.03, p 0.46; negative comments: r0.08,p 0.38).

    Most participants (88.5%) felt the PAL sessions were helpful

    in pointing out technical abilities while 65.7% of residents felt

    more confident with their technical skills upon completion of

    the exercises. Residents (80.0%) felt motivated to practice their

    surgical skills at home after the PAL session, and 71.4% noted

    an increased sense of peer camaraderie. 42.8% of participants

    stated that the PAL session engendered a feeling of anxiety to

    perform well.

    Discussion

    In this pilot study, we created a picture of peer feedback in the

    context of our surgical skills training laboratory. We found that

    residents generally provided specific feedback to peers in PAL

    sessions. Participants rarely made global negative comments,

    which would likely be of little use in peer instruction and may

    serve to create a hostile learning environment. Although the

    correlations are modest, our results show that with the

    assistance of a guideline on knot-tying tasks, peer teachers

    did indeed provide accurate and specific feedback.

    The PAL sessions provided a unique opportunity for

    residents to assess their skills in the larger context of their

    peers. We found that PAL may even stimulate independent

    learning and foster motivation to practice skills outside of the

    lab. Despite a potentially competitive learning environment,

    the majority of PAL participants endorsed a sentiment of

    resident camaraderie after the sessions, a feeling that could

    translate to stronger team relationships in the realm of patient

    care. Finally, nearly half of the residents identified feelings of

    stress associated with the PAL activity, which can aid in

    memory formation and reinforce learning at low levels

    (Friedlander et al. 2011). Higher levels of anxiety may be

    detrimental to learning, and further investigation into stress

    effects of peer learning is warranted.

    Our findings contrast with the sentinel study on peer

    teaching in surgery by Rogers et al. (2000). In this study,

    addition of peer collaboration to a computer-based knot-tying

    curriculum had a negative impact on first year medical

    students ability to tie knots. Differences in design and

    outcome measurement may explain the divergent results of

    our studies. While the endpoint of the Rogers study was

    performance, we focused on the quality and accuracy of peer

    feedback. In addition, Rogers et al grouped peers in pairs,

    which may increase the risk of faulty interpretation of

    instructions on knot-tying. Our peer groups included three to

    five participants to ensure appropriate group communication.

    Finally, Rogers et al evaluated the performances of first year

    medical students after a single 45minute session. Our partic-

    ipants were first year residents with more practical experience

    with basic surgical skills than novice medical students.

    Therefore PAL may be more productive at more advanced

    stages of technical skills acquisition.

    This study has limitations. We compared two different

    surgical skills in the PAL sessions, using a guideline for the PAL

    exercise on knot-tying and no guideline for the suturing

    session. Participants may have been more comfortable giving

    feedback in the knot-tying PAL session owing to an inherent

    greater complexity of suturing. Thus peer teacher facility with

    each skill may have influenced the number and type of

    feedback comments, making it difficult to extrapolate the

    effect of the guideline itself. In addition, the suture scoring

    system used by the expert in our study has not been validated

    in this population of surgical residents, which may have further

    influenced the relationship between guideline use and expert

    scoring.

    Table 1. Comment types in knot-tying and suturing PAL activities.

    Mean number of comments (95% CI)

    Comment typeKnot-tying/guide

    usedSuturing/no guide

    used p-value

    Global positive 1.16 (0.78, 1.58) 0.85 (0.40, 1.30) 0.20

    Specific positive 3.24 (2.13, 4.35) 0.90 (0.39, 1.41) 0.002

    Global negative 0.09 (0.09, 0.27) 0.15 (0.02, 0.32) 0.66Specific negative 2.67 (1.79, 3.55) 1.35 (0.92, 1.78) 0.008

    Total comments 7.18 (5.55, 8.81) 3.25 (2.53, 3.97) 50.001

    J.H. Beard et al.

    958

    Med

    Tea

    ch D

    ownl

    oade

    d fr

    om in

    form

    ahea

    lthca

    re.c

    om b

    y SU

    NY

    Sta

    te U

    nive

    rsity

    of

    New

    Yor

    k at

    Sto

    ny B

    rook

    on

    10/2

    7/14

    For

    pers

    onal

    use

    onl

    y.

  • Conclusions

    Surgical residents can provide high quality specific feedback to

    peers on surgical skills using performance guidelines. Given

    the satisfaction with PAL and quality of peer feedback, we

    recommend developing guidelines for peer assessment of

    surgical skills along with further exploration of effective peer

    teaching methodology in surgical skills laboratory training.

    Declaration of interest: The authors report no declarations

    of interest.

    Notes on contributors

    JESSICA H. BEARD, MD, MPH is a general surgery resident at UCSF.

    PATRICIA OSULLIVAN, EdD is Professor of Medicine and Director of

    Research and Development in Medical Education at UCSF.

    BARNARD PALMER, MD, MEd completed general surgery residency at

    UCSF-East Bay program and is an Endocrine Surgery fellow at Mount Sinai

    School of Medicine.

    MARY QIU is a medical student at UCSF.

    EDWARD H KIM, MD is Assistant Professor of Surgery and Director of the

    Surgical Skills Lab at UCSF.

    References

    Evans AW, Leeson RM, Petrie A. 2007. Reliability of peer and self-

    assessment scores compared with trainers scores following third molar

    surgery. Med Educ 41(9):866872.

    Friedlander MJ, Andrews L, Armstrong EG, Aschenbrenner C, Kass JS,

    Ogden P, Schwartzstein R, Viggiano TR. 2011. What can medical

    education learn from the neurobiology of learning? Acad Med: J Assoc

    Am Med Coll 86(4):415420.

    Gibson DR, Campbell RM. 2000. The role of cooperative learning in the

    training of junior hospital doctors: A study of paediatric senior house

    officers. Med Teach 22(3):297300.

    Graziano SC. 2011. Randomized surgical training for medical students:

    Resident vs. peer-led teaching. Am J Obstet Gynecol 204(6):541542.

    Kim EH. 2011. Validation of the UCSF knot-tying efficiency checklist,

    Unpublished manuscript.

    Kruidering M, Chou C, OSullivan P. 2009. Teaching feedback to first year

    medical students: Long term skill retention and accuracy of student self-

    assessment. J Gen Internal Med 24:721726.

    Lockspeiser TM, OSullivan P, Teherani A, Muller J. 2008. Understanding

    the experience of being taught by peers: The value of social and

    cognitive congruence. Adv Health Sci Educ, Theory Practice

    13(3):361372.

    Rogers DA, Regehr G, Gelula M, Yeh KA, Howdieshell TR, Webb W. 2000.

    Peer teaching and computer-assisted learning: An effective combination

    for surgical skill training? J Surg Res 92(1):5355.

    Ross MT, Cameron HS. 2007. Peer assisted learning: A planning and

    implementation framework: AMEE Guide no. 30. Med Teach 29(6):

    527545.

    Peer assisted learning in surgical training

    959

    Med

    Tea

    ch D

    ownl

    oade

    d fr

    om in

    form

    ahea

    lthca

    re.c

    om b

    y SU

    NY

    Sta

    te U

    nive

    rsity

    of

    New

    Yor

    k at

    Sto

    ny B

    rook

    on

    10/2

    7/14

    For

    pers

    onal

    use

    onl

    y.

Recommended

View more >