Evaluating the Briefing, Intraoperative Teaching,

Debriefing (BID) method for surgical skills teaching:

a pilot study

Evaluating the Briefing, Intraoperative Teaching,

Debriefing (BID) method for surgical skills teaching:

a pilot study

Daniel Hoppe MD MEdOleg Safir MD FRCS(C) MEdAdam Dubrowski PhD

What is BID?What is BID?

Journal of the American College of Surgeons (2009)

Step Task

Briefing Set learning objectives

Intra-operative Teaching

Teaching during the encounter

DebriefingReflection

RulesReinforceCorrect

Why the BID?Why the BID?

WHAT LEARNERS WANT*

Instructor “game-plan”

Involvement in the learning process

Ongoing and direct feedback

WHAT ACTUALLY HAPPENS**

“Opportunistic” teaching

Passive transfer of information

Limited feedback given

*Vikis et al. (2008), Cox and Swanson (2001), Snyder et al (2012)**Scallon et al. (1992), Irby (1995), Roberts et al. (2008), Jefree and Clarke (2010)

Objective

✦ To evaluate whether the BID method is an effective framework for surgical skills teaching

✦ Learner reactions, satisfaction

MethodsMethods

First year orthopaedic residents (n=12)

Allocated to control group (n=6)

Allocated to BID group (n=6)

PRE-TEST (OSATS)

Learned Control Skill(Central Line)

Learned Control Skill(Central Line)

BID Method Training Session

First year orthopaedic residents (n=12)

Allocated to control group (n=6)

Allocated to BID group (n=6)

PRE-TEST (OSATS)

Learned Control Skill(Central Line)

Learned Control Skill(Central Line)

BID Method Training Session

✦ 60 minute session

✦ Interactive - presentation, discussion, role-play

✦ Given to both instructor and BID group separately

✦ Learners trained to create synergistic learning effect*

BID Method Training Session

*Aagaard et al. (2004), Manyon et al. (2003), Vikis et al. (2008)

Learned Control Skill(Central Line)

Learned Control Skill(Central Line)

BID Method Training Session

Learned Test Skill 1(Bone Wedge)

Learned Test Skill 2(Bone Plating)

Learned Test Skill 1(Bone Wedge)

Learned Test Skill 2(Bone Plating)

POST-TEST (OSATS)

Outcome Measures✦ Change in individual OSATS scores -

checklist and GRS

Outcome Measures

Rizzolo (2006), National League for Nursing (2007)

✦ Focus group data

✦ Satisfaction Questionnaire, Educational Practices Questionnaire*

Standardization of Instructors

✦ Used a number of measures to try and ensure approximate instructor equivalency

1. Perception of Preparedness Questionnaire - no differences

2. “Assessing Teaching Skills” assignment (adapted from Ramsden et al. 1995)

3. “Control Skill” (Central Line Insertion) - taught to each group prior to BID training

ResultsResults

Assessing Instructor Equivalency

✦ Perception of Preparedness Questionnaires similar between both instructors

✦ also had similar teaching ideologies/experience-no formal educational training

-no familiarity with educational theory

-preferred small-group learning to lectures

-feedback important (but neither knew framework for delivering feedback)

-use questions to confirm understanding

OSATS Pre-test Data✦ no significant differences in baseline pre-test

scores between groups for checklist and GRS scores

Skill GRS Checklist

Central Line Insertion

P=0.326 P=0.102

Bone Wedge P=0.478 P=0.536

Bone Plating P=0.927 P=0.263

Control Skill: Central Line Insertion

PROPORTIONAL CHANGE

84.93

55.96

0

20

40

60

80

100

Pro

po

rtio

nal

Incr

ease

Control Group BID Group

P=0.129Pre-Test

Post-Test

Control 46 96

BID 53 81

Global Rating Scale

After BID trainingAfter BID training

Test Skill 1: Bone Wedge Cut

PROPORTIONAL CHANGE

13.8

82.2

0

20

40

60

80

100

Pro

po

rtio

nal

Incr

ease

Control Group BID Group

P=0.049Pre-Test

Post-Test

Control 59 69

BID 66 94

Global Rating Scale

Test Skill 2: Bone Plating

PROPORTIONAL CHANGE

23.7

69

0

20

40

60

80

Pro

po

rtio

nal

Incr

ease

Control Group BID Group

P=0.011Pre-Test

Post-Test

Control 42 56

BID 43 82

Global Rating Scale

Checklist Scores

✦ No significant difference in improvement in checklist scores for any skills

Secondary Outcome Measures

Secondary Outcome Measures

Questionnaire Data✦ After first skill (both groups

untrained)Control

Group (%)BID Group

(%)Satisfaction

with Teaching Method

85 85

Objectives/Information

83 77

Active Learning/Facult

y Interaction79 83

Feedback 85 72

Questionnaire Data✦ After last skill

Control Group (%)

BID Group (%)

Satisfaction with Teaching

Method77 90

Objectives/Information

74 84

Active Learning/Facult

y Interaction72 88

Feedback 69 81

Focus Group Data

✦ Easy to use and remember

✦ For majority, first time setting objectives

✦ Formal time for feedback appreciated

✦ Less time wasted during sim sessions

Conclusions

✦ training both the learners and instructors in the BID method produced significant improvement in GRS scores for specific surgical skills

✦ trend towards higher learner satisfaction

✦ BID provides an easy-to-implement framework for surgical simulation teaching

Limitations

✦ Small sample size

✦ Choice of skills - limited

✦ Retention/transfer not tested

✦ In vivo vs. in vitro effects?

Future Directions

✦ these results are promising enough to continue with a larger-scale study

✦ more residents, more skills, more instructors

✦ simulation vs. OR

✦ role in Competency-Based Education?

Quotes from BID Group

“These sessions seemed a bit like a study group”

“I can see this working in the OR but it would need to have staff buy-in”

“This may not work when I’m on call and it is very busy”

“I think using this approach improved the flow of teaching”

“Having an objective to work on focused me on a specific task”

“I really liked the debriefing part, and I wish this would happen in real life!”

Thank youThank you

Acknowledgments:Thanks to Lisa Satterthwaite, Dr. Ranil

Sonnandara, Carween Hui, Polina Mironova, Dr. Hesham Abdelbary, Dr. Michael Drexler,

and the staff of the Mount Sinai Surgical Skills Centre

Acknowledgments:Thanks to Lisa Satterthwaite, Dr. Ranil

Sonnandara, Carween Hui, Polina Mironova, Dr. Hesham Abdelbary, Dr. Michael Drexler,

and the staff of the Mount Sinai Surgical Skills Centre

Step Task Script

Briefing Set learning objectives

“What would you like to focus on?”

Intra-operative Teaching

Teaching during the encounter

Focus on stated objectives

DebriefingReflection

RulesReinforceCorrect

“How did you think you did?”

“You did well at...”“Next time, do...”