Upload
eh
View
218
Download
0
Embed Size (px)
Citation preview
2012; 34: 957–959
SHORT COMMUNICATION
Peer assisted learning in surgical skillslaboratory training: A pilot study
J.H. BEARD, P. O’SULLIVAN, B.J.A. PALMER, M. QIU & E.H. KIM
UCSF, USA
Abstract
Background: Peer assisted learning (PAL) has been rarely investigated in surgical skills training.
Aims: Describe feedback residents give peers on surgical skills with and without guidelines, determine the association between
feedback and actual performance, evaluate resident satisfaction with PAL.
Method: Participants evaluated peers using a validated knot-tying checklist and provided feedback on suturing without a
guideline. Feedback comments were coded by type and an expert scored performance of each participant. Residents completed a
satisfaction questionnaire.
Results: Comments were generally specific. Feedback was twice as likely with the use of a guideline. Specific feedback correlated
significantly with expert knot-tying score but not suturing score. Most participants felt peer feedback was helpful and were
motivated to practice surgical skills after PAL sessions.
Conclusions: Surgical residents can provide high quality specific feedback to peers on surgical skills using performance
guidelines. Further exploration of effective PAL methodology in surgical skills laboratory training is needed.
Introduction
Informal peer teaching is at the core of general surgical
education in the United States. Inside and out of the operating
room, senior residents educate and shape the development of
juniors through role-modeling, direct instruction, and feed-
back. Although peer assisted learning (PAL) is emerging as a
valuable tool in medical education, there has been little
investigation of the formal use of PAL in the development of
surgical skills.
Peer teachers are thought to be ‘‘cognitively congruent’’
with their students and thus theoretically better than experts at
understanding the knowledge and skill level of their trainees
(Lockspeiser et al. 2008). This similarity in educational
experience and technical skill levels between student and
teacher can enhance motivation and achievement through
modeling of attainable learning objectives (Ross & Cameron
2007). In addition, PAL promotes the development of the tutor
as a teacher, a lifelong skill required of academic physicians.
Finally, PAL can solidify the knowledge and skill of the tutor
through the instruction of others in a supported, socially
stimulating learning environment (Gibson & Campbell 2000).
A recent randomized comparison of resident versus peer
medical student teaching during an operating room introduc-
tion course demonstrated better technical performance by
students in the peer-teaching group (Graziano 2011). Evans
and colleagues found that peer assessment scores reflect
expert trainer scores more accurately that surgeon self-
assessment in dental extraction (Evans et al. 2007). In addition
to being accurate, the authors suggest that peer assessment
may encourage honest reflection on skills, increased
collaboration, and teamwork. Although the use of guidelines
in PAL has been advocated in the literature, there are no
definitive studies on the issue (Ross & Cameron 2007).
We piloted a PAL program in our surgical skills lab. The
aims of this study were: describe the nature of feedback
residents give each other on basic surgical skills with and
without guidelines, determine an association between peer
feedback an actual performance, and evaluate resident satis-
faction with PAL.
Methods
We conducted this descriptive study at the Surgical Skills Lab
of the University of California, San Francisco (UCSF). The
UCSF Committee on Human Research certified the study as
exempt from review. Thirty-three general surgery residents in
their first year of training participated in the knot-tying session,
while twenty residents participated in the suturing session.
During the sessions, residents either completed a series of two
handed knots or a simple suturing exercise and received peer
feedback upon completion of the task. Participants were
instructed on the use of a previously validated Knot-tying
Checklist for feedback in the knot-tying session (Kim 2011).
There were no guidelines given for the suturing session. PAL
sessions were videotaped. Care was taken to ensure anonym-
ity of the participants.
Researchers transcribed and coded each participant com-
ment as global positive, specific positive, global negative and
specific negative following a previously employed methodol-
ogy (Kruidering et al. 2009). Global comments were defined as
Correspondence: Jessica H. Beard, 513 Parnassus Ave, S-321, San Francisco, CA 94143-0470, USA. Tel: 713.818.1870; fax: 415.502.1259;
email: [email protected]
ISSN 0142–159X print/ISSN 1466–187X online/12/110957–959 � 2012 Informa UK Ltd. 957DOI: 10.3109/0142159X.2012.706340
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y SU
NY
Sta
te U
nive
rsity
of
New
Yor
k at
Sto
ny B
rook
on
10/2
7/14
For
pers
onal
use
onl
y.
feedback related to overall impression of the activity, while
specific comments included feedback on particular technical
aspects of performance. An attending surgeon scored resident
performance using the validated Knot-tying Checklist and a
separate suturing checklist. Residents completed an anony-
mous satisfaction questionnaire.
We used SPSS Version 17.0 for data analysis. Differences
between the mean number of each type of peer comment in
the knot-tying and suturing categories were identified using a
t-test. We calculated Pearson correlations to determine the
relationship between peer feedback and expert scores.
Results
Most peer feedback was either specific positive (34%) or
specific negative (41%) but rarely global positive (23%) or
global negative (2%). ‘‘Your [knot-tying] distance was perfect’’
represents an example of a specific positive comment while
‘‘You didn’t gather the suture’’ is an example of a specific
negative comment.
Table 1 illustrates the mean number of types of comments
in the knot-tying and suturing PAL sessions. On average, peer
teachers made 7.18 comments (95% CI: 5.55–8.81) during the
knot-tying sessions using a teaching guide and 3.25 comments
(95% CI: 2.53–3.97) during the suturing sessions, which were
unguided. The amount of feedback in knot-tying sessions
using a guideline more than doubled when compared to
suturing sessions without teaching guidelines (p5 0.001).
Specific positive and negative comments were also more
common than global comments in the knot-tying group.
In the knot-tying group, expert score was positively
associated with the proportion of specific positive feedback
(r¼ 0.48, p¼ 0.003) and negatively associated with the
proportion of specific negative feedback (r¼�0.35,
p¼ 0.028). There was no significant association identified
between expert suturing score and peer feedback (positive
comments: r¼�0.03, p¼ 0.46; negative comments: r¼�0.08,
p¼ 0.38).
Most participants (88.5%) felt the PAL sessions were helpful
in pointing out technical abilities while 65.7% of residents felt
more confident with their technical skills upon completion of
the exercises. Residents (80.0%) felt motivated to practice their
surgical skills at home after the PAL session, and 71.4% noted
an increased sense of peer camaraderie. 42.8% of participants
stated that the PAL session engendered a feeling of anxiety to
perform well.
Discussion
In this pilot study, we created a picture of peer feedback in the
context of our surgical skills training laboratory. We found that
residents generally provided specific feedback to peers in PAL
sessions. Participants rarely made global negative comments,
which would likely be of little use in peer instruction and may
serve to create a hostile learning environment. Although the
correlations are modest, our results show that with the
assistance of a guideline on knot-tying tasks, peer teachers
did indeed provide accurate and specific feedback.
The PAL sessions provided a unique opportunity for
residents to assess their skills in the larger context of their
peers. We found that PAL may even stimulate independent
learning and foster motivation to practice skills outside of the
lab. Despite a potentially competitive learning environment,
the majority of PAL participants endorsed a sentiment of
resident camaraderie after the sessions, a feeling that could
translate to stronger team relationships in the realm of patient
care. Finally, nearly half of the residents identified feelings of
stress associated with the PAL activity, which can aid in
memory formation and reinforce learning at low levels
(Friedlander et al. 2011). Higher levels of anxiety may be
detrimental to learning, and further investigation into stress
effects of peer learning is warranted.
Our findings contrast with the sentinel study on peer
teaching in surgery by Rogers et al. (2000). In this study,
addition of peer collaboration to a computer-based knot-tying
curriculum had a negative impact on first year medical
students’ ability to tie knots. Differences in design and
outcome measurement may explain the divergent results of
our studies. While the endpoint of the Rogers study was
performance, we focused on the quality and accuracy of peer
feedback. In addition, Rogers et al grouped peers in pairs,
which may increase the risk of faulty interpretation of
instructions on knot-tying. Our peer groups included three to
five participants to ensure appropriate group communication.
Finally, Rogers et al evaluated the performances of first year
medical students after a single 45 minute session. Our partic-
ipants were first year residents with more practical experience
with basic surgical skills than novice medical students.
Therefore PAL may be more productive at more advanced
stages of technical skills acquisition.
This study has limitations. We compared two different
surgical skills in the PAL sessions, using a guideline for the PAL
exercise on knot-tying and no guideline for the suturing
session. Participants may have been more comfortable giving
feedback in the knot-tying PAL session owing to an inherent
greater complexity of suturing. Thus peer teacher facility with
each skill may have influenced the number and type of
feedback comments, making it difficult to extrapolate the
effect of the guideline itself. In addition, the suture scoring
system used by the expert in our study has not been validated
in this population of surgical residents, which may have further
influenced the relationship between guideline use and expert
scoring.
Table 1. Comment types in knot-tying and suturing PAL activities.
Mean number of comments (95% CI)
Comment typeKnot-tying/guide
usedSuturing/no guide
used p-value
Global positive 1.16 (0.78, 1.58) 0.85 (0.40, 1.30) 0.20
Specific positive 3.24 (2.13, 4.35) 0.90 (0.39, 1.41) 0.002
Global negative 0.09 (�0.09, 0.27) 0.15 (�0.02, 0.32) 0.66
Specific negative 2.67 (1.79, 3.55) 1.35 (0.92, 1.78) 0.008
Total comments 7.18 (5.55, 8.81) 3.25 (2.53, 3.97) 50.001
J.H. Beard et al.
958
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y SU
NY
Sta
te U
nive
rsity
of
New
Yor
k at
Sto
ny B
rook
on
10/2
7/14
For
pers
onal
use
onl
y.
Conclusions
Surgical residents can provide high quality specific feedback to
peers on surgical skills using performance guidelines. Given
the satisfaction with PAL and quality of peer feedback, we
recommend developing guidelines for peer assessment of
surgical skills along with further exploration of effective peer
teaching methodology in surgical skills laboratory training.
Declaration of interest: The authors report no declarations
of interest.
Notes on contributors
JESSICA H. BEARD, MD, MPH is a general surgery resident at UCSF.
PATRICIA O’SULLIVAN, EdD is Professor of Medicine and Director of
Research and Development in Medical Education at UCSF.
BARNARD PALMER, MD, MEd completed general surgery residency at
UCSF-East Bay program and is an Endocrine Surgery fellow at Mount Sinai
School of Medicine.
MARY QIU is a medical student at UCSF.
EDWARD H KIM, MD is Assistant Professor of Surgery and Director of the
Surgical Skills Lab at UCSF.
References
Evans AW, Leeson RM, Petrie A. 2007. Reliability of peer and self-
assessment scores compared with trainers’ scores following third molar
surgery. Med Educ 41(9):866–872.
Friedlander MJ, Andrews L, Armstrong EG, Aschenbrenner C, Kass JS,
Ogden P, Schwartzstein R, Viggiano TR. 2011. What can medical
education learn from the neurobiology of learning? Acad Med: J Assoc
Am Med Coll 86(4):415–420.
Gibson DR, Campbell RM. 2000. The role of cooperative learning in the
training of junior hospital doctors: A study of paediatric senior house
officers. Med Teach 22(3):297–300.
Graziano SC. 2011. Randomized surgical training for medical students:
Resident vs. peer-led teaching. Am J Obstet Gynecol 204(6):541–542.
Kim EH. 2011. Validation of the UCSF knot-tying efficiency checklist,
Unpublished manuscript.
Kruidering M, Chou C, O’Sullivan P. 2009. Teaching feedback to first year
medical students: Long term skill retention and accuracy of student self-
assessment. J Gen Internal Med 24:721–726.
Lockspeiser TM, O’Sullivan P, Teherani A, Muller J. 2008. Understanding
the experience of being taught by peers: The value of social and
cognitive congruence. Adv Health Sci Educ, Theory Practice
13(3):361–372.
Rogers DA, Regehr G, Gelula M, Yeh KA, Howdieshell TR, Webb W. 2000.
Peer teaching and computer-assisted learning: An effective combination
for surgical skill training? J Surg Res 92(1):53–55.
Ross MT, Cameron HS. 2007. Peer assisted learning: A planning and
implementation framework: AMEE Guide no. 30. Med Teach 29(6):
527–545.
Peer assisted learning in surgical training
959
Med
Tea
ch D
ownl
oade
d fr
om in
form
ahea
lthca
re.c
om b
y SU
NY
Sta
te U
nive
rsity
of
New
Yor
k at
Sto
ny B
rook
on
10/2
7/14
For
pers
onal
use
onl
y.