Creating a Culture of Safety: Training Doctor of Nursing Practice Students and

Medical Residents in Team-Based Quality Improvement

Valentina Brashers MD, FACP, FNAP ; University of Virginia

Beth Quatrara, DNP, RN, CMSRN, ACNS-BC ; University of Virginia

Marianne Baernholdt, Ph.D., M.P.H., RN, FAAN ; Virginia Commonwealth University Christine Kennedy, RN, PhD, FAAN ; University of Virginia

Meg Norling; University of Virginia

Supported by HRSA ANE Grant

#D09HP26942

Presentation Goals After having heard this presentation, participants will be better able to:

• Discuss the background for creating an interprofessional quality improvement and patient safety training in graduate nursing and medical education

• Identify the components of a comprehensive interprofessional quality improvement program for graduate nursing and medical trainees

• Describe the graduate nursing student and medical resident learner outcomes of an interprofessional quality improvement training program

What is at stake if we don’t change? • Poor Outcomes and Errors

• Prevention • Diagnostic • Treatment • Transitions in Care

• Costs • Attrition of healthcare providers

• Turnover • Suicide

• Patient suffering

Makary, MA & Daniel, D (2016) BMJ 353:i2139

Mrs. Bailey One day observation of a patient on an acute cardiology –

0700: examined by the resident → new med ordered

0724: assessment by day shift nurse → discussed day’s care plan

0804: examined by the attending → indicated new tests needed

1120 – 1900: 43 individual care provider visits: - RN 17 - MD 2 - CNS 3, - PharmD 1 - PCA 18 - CM 2 - SW 1 How many different messages did this patient receive today?

Creating a Culture of Safety

• 2014-2017, three cohorts of BSN-DNP students from acute care, primary care, and psych/mental health + 3 cohorts of internal medicine residents

• Three year curriculum in interprofessional quality improvement and teamwork

• Interprofessional leadership by faculty experts in teamwork and quality improvement from nursing, medicine, pharmacy, social work, business, public health, & healthcare administration

Responsibility & Roles

Values & Ethics

Teamwork

Communication

©

ASPIRE framework for IPE competencies:

Creating a Culture of Safety • 7 two-hour sessions with medical residents

on quality improvement skills for Cohort 1 (now have a total of 11 QI workshops for subsequent cohorts)

• 12 ninety-minute IP Brownbag Seminars for Cohort I (now have 18 Brownbag Seminars)

• 7 three-hour Project Development Workshops for Cohort 1 (now have 10 Project Development Workshops)

• 4 Interprofessional Simulations

• 6 nationally – recognized expert IPE Guest Speakers (Reeves, Brandt, Zierler, Haidet, Cox, Hall)

DNP student participation

Year

1st-year cohort

Returning students

(2nd-year)

Returning students

(3rd-year)

Total number of students in IPE

PSQI curriculum

2014-15 (Year 1)

7 0 0 7

2015-16 (Year 2)

9 7 0 16

2016-17 (Year 3)

14 9 7 30

24 medical residents per year

Outcomes Data (the following results for Cohort 1 only, DNP =7, Resident = 24)

• Session evaluations • TeamSTEPPS Teamwork Attitudes Questionnaire

(TAQ) AHRQ TeamSTEPPS: https://www.ahrq.gov/teamstepps/instructor/reference/teamattitude.html

• Quality Assessment & Improvement Curriculum (QAIC) Assessment Tool Oyler, Univ. of Chicago:

• https://static-content.springer.com/esm/art%3A10.1007%2Fs11606-010-1547-y/MediaObjects/11606_2010_1547_MOESM4_ESM.pdf

• Quality Improvement Project Assessment Tool (QIPAT) Leenstra et al. J Gen Intern Med 2007; 22:1330-1334; http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2219765/

0

1

2

3

4

5

6

7

HumanFactors &SystemsThinking

ParientSafety and

ErrorDisclosure

RCA & ErrorResponse

Variation,Run Charts

PainManagement

Chronic CareModel

HealthcarePaymentModels

DNP Student Average Evaluation Score DNP/Resident Workshops:

(Strongly Disagree- Strongly Agree Scale 1-7)

Sampling of DNP student comments about learning alongside medical residents

• “Highlighting how MDs struggle w/ responsibilities they have. Letting go of past negative encounters to form new teams.”

• “Understanding MD side, how to apply my knowledge & communicate more effectively about my patient to my team.”

• “Thinking at systems level really contributes to teamwork- helps everyone see bigger picture. Interacted with residents to see how care delivery system works.”

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Human Factors& SystemsThinking

Parient Safetyand ErrorDisclosure

RCA & ErrorResponse

Variation, RunCharts

PainManagement

Chronic CareModel

HealthcarePaymentModels

Resident Average Evaluation Score DNP/Resident Workshops: (Disagree-Agree Scale 1-5)

Sampling of Resident comments about learning alongside DNP students

• “Glad to have multidisciplinary session with RNs involved.”

• “It was nice to work with nurses and practice an integrated disclosure.”

• “Good practice identifying errors and working with nurses.”

• “Exposure to some of the errors that nurses look for that physicians may pass over.”

• “Seeing how nurses view patients differently.”

0

1

2

3

4

5

6

7

DNP Student Average Evaluation Score Brownbag Seminars:

(Strongly Disagree- Strongly Agree Scale 1-7)

Sampling of DNP student teamwork-related comments from Brownbag Seminars

• “Made me to look back at my past experiences & identify areas that I could improve to facilitate in building collaborative teams”

• “You gain more from working together. Communication is essential for team success.”

• “It helped support the idea that collaborative teams are able to accomplish more together for patient safety.”

• “How we affect each other based on our actions & responses. How all people want to feel valued and job perceptions.”

0

1

2

3

4

5

6

7

Project aimsand methods

Outcomesmeasures

Workingthroughbarriers

Qualitativemethods

Discussionand

Conclusions

Projectpresentation

Creating aposter

DNP Student Average Evaluation Score Project Development Sessions:

(Strongly Disagree- Strongly Agree Scale 1-7)

Sampling of DNP student comments from Project Development Workshops

• “Just keep more of the sessions coming!”

• “I found the content very helpful for my DNP project. I wish to follow up with presenter.”

• “More great speakers like today!”

• “Presenting the projects to one another was great!”

• “So helpful to hear others presentations & learn from each other! Very beneficial!”

• “Can you guys come to my work and keep these lessons/sessions going?!”

0

1

2

3

4

5

6

7

Room of Errors Team-Based RapidResponse

Rounding with Heart InterprofessionalHandoffs

DNP Student Average Evaluation Score Interprofessional Simulation Sessions:

(Strongly Disagree- Strongly Agree Scale 1-7)

Sampling of DNP student comments from Simulation Sessions

• Room of Errors: • “ “This definitely opened my eyes to what different members of the care team

identify as errors. It makes it evident how important a multi-disciplinary approach is to patient safety.”

• Team-based Rapid Response: • “I would like to redo these sessions next year but I want to practice leading the

code.”

• Rounding with Heart • “Yes because it helped support the idea that collaborative teams are able to

accomplish more together for patient safety, whole team can see care from the patient’s perspective.”

• Handoffs: • “Highlighted priorities in communication and how easy it can be to leave out

information. This was great to begin the transition to thinking as a provider.”

DNP student pre/post TAQ

• 5 domains: Team Structure, Leadership, Situation Monitoring, Mutual Support, Communication

• Pretest score averages for almost all questions were very high – self-selected group of teamwork oriented nurses making statistically significant improvement unlikely for this small n.

• Posttest TAQ total score improved by 10 points (but p=0.053) • No significant improvements in individual item postest scores except: “It is

important to monitor the emotional and physical status of other team members.” o Interesting to note that one of the most highly rated Brownbag Seminars

was “Relational Factors in Teamwork” which focused heavily on emotional issues – students commented on the insights they gained from this session

Resident pre/post TAQ Pretest (Mean)

Posttest (Mean)

p

Individuals can be taught how to scan the environment for important situational cues.

3.89 4.06 0.015

Monitoring patients provides an important contribution to effective team performance

4.07 4.19 0.030

It is important to monitor the emotional and physical status of other team members.

4.28 4.40 0.038

Team members who monitor their emotional and physical status on the job are more effective.

4.31 4.43 0.020

DNP student pre/post QAIC Pretest (Mean)

Posttest (Mean)

p

Writing a clear problem statement 2.56 3.86 0.022 Applying the best professional knowledge 2.78 3.43 0.231 Using measurement to improve skills 2.89 3.57 0.172 Studying the Process 2.78 3.71 0.045 Making changes in a system 2.67 3.43 0.094 Identifying whether a change leads to an improvement in your skills

2.78 3.71 0.017

Using small cycles of change 2.44 3.57 0.018 Identifying best practices and comparing these to your local practice/skills

2.89 3.71 0.008

Implementing a structured plan to test a change 2.44 3.86 <0.001 Using the PDSA model as a systematic framework for trial and learning

1.33 3.86 <0.001

Identifying how data is linked to specific processes 2.33 3.71 0.003 Building your next improvement upon success or failure 2.67 3.71 0.038 Total score (summed values for each variable) 30.56 44.14 <0.001

RESIDENT pre/post QAIC Pretest (Mean)

Posttest (Mean)

p

Writing a clear problem statement 2.86 3.49 <0.001 Applying the best professional knowledge 2.78 3.38 <0.001 Using measurement to improve skills 2.61 3.29 <0.001 Studying the Process 2.64 3.14 <0.001 Making changes in a system 2.23 2.90 <0.001 Identifying whether a change leads to an improvement in your skills

2.69 3.20 <0.001

Using small cycles of change 2.49 3.12 <0.001 Identifying best practices and comparing these to your local practice/skills

2.70 3.15 <0.001

Implementing a structured plan to test a change 2.48 3.12 <0.001 Using the PDSA model as a systematic framework for trial and learning

2.17 2.97 <0.001

Identifying how data is linked to specific processes 2.38 3.07 <0.001 Building your next improvement upon success or failure 2.63 3.27 <0.001 Total (summed scores for each variable) 30.68 37.91 <0.001

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

Implement/Evaluate

Proposed Interention

Potential Intervention

Choice of QI Project

Root Cause Analysis

Identify Stakeholders

Define Problem

OSCE Project Proposal

DNP student QIPAT OSCE Scores vs Project Proposal Scores (Scale 1-5)

p<0.001

p<0.001

p=0.006

p<0.001

p=0.002

p=0.002

p=0.008

New DNP Creating a Culture of Safety

Interprofessional Patient Safety & Quality Improvement Scholar

2014-2017

Questions ?