Quality and safety educational training programmes

and how we evaluate them

Conflicts of interest

• Ekaterine Rukhadze – none• Rob Bethune – none• Jane Runnacles – none• Jo-Inge Myhre – none• Jessica Perlo - none

twitter - #d4

#quality2014

Quality and Safety should be included in the Curriculum of

Medical Universities

Eka Rukhadze MD, PhDNino Butskhrikidze MD

David Tvildiani Medical University; Medical Alliance for Quality and Safety

International Forum on QUALITY&SAFETY in HEALTHCARE, Paris 2014

Health Care is Hazardous

Leape LL. Presentation at the Public’s health: a matter of trust symposium, 2002. Harvard University School of Public Health

Incidence of Medical Errors

Leape LL. Errors in medicine. Clinica Chimica Acta2009;404:2-5. MacDermaid LJ. First, do no harm: medical error in Canada. 2005

Status of patient safety in Georgia

• Lack of knowledge • Seriousness of the situation is not

acknowledged• Unavailability of objective data

Goals and Objectives

• To improve the knowledge in Quality and Patient Safety

• To introduce an evidenced based approach to quality and safety

• To create the course curriculum

Course Design

• David Tvildiani Medical University• Target Audience:

– 5-th year students – PhD students

• 16 academic hours (8 seminars)• Testing (pre- and post-tests)• Course evaluation

Educational Recourses for Course Curriculum

• Online Courses in Patient Safety (PS 100- PS 106) www.ihi.org– Medical Quality/Safety. CHOP Seminar In Salzburg, 2013– R.M. Wachter. Understanding Patient Safety, The

McGraw-Hill, 2008– Gary Cook. Introduction to patient Safety; www.bmj.org– Imran Qureshi. Quality and safety in healthcare;

www.bmj.org– P.R. Scholtes et all, The Team Handbook, 2010

Content of the Course

• Introduction to Patient Safety • The System Reasons of Medical Errors – Blunt End• The Individual Reasons of Medical Errors – Sharp End• Types of Medical Errors• Response on Errors• Strategies of Improvement

Results

Pre-test (percentage of correct answers)

Post-test (percentage of correct answers)

Improvement0%

10%20%30%40%50%60%70%80%90%

100%

28%

51%

23%

Students -2013

Results

Pre-test (percentage of correct answers)

Post-test (percentage of correct answers)

Improvement0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

37%

64%

27%

PhD students -2014

Comparison of TEST RESULTS

0%

20%

40%

60%

80%

100%

28%

51%

23%

Students -2013

0%30%60%90%

37%64%

27%

PhD students -2014

Results

47%

35%

13%

2%3%

Course evaluation by Students - 2013

very goodgoodfairpoorvery poorno response

Results

62%

36%

20%

Course evaluation by PhD students - 2014

very goodgoodfairpoorvery poorno response

Comparison of EVALUATION

62%36%

20%

PhD students - 2014

very goodgoodfairpoorvery poorno response47%

35%

13%

2% 3%

Students - 2013

very goodgoodfairpoorvery poorno response

Results Suggestions about particular topics – PhD 2014

0%10%20%30%40%50%60%70%80%90%

100%100%

80%

40%

20%

40%

20%

60%

80%

60%

100%

information is enough

should be more detailed

Summary• Improvement was achieved in both cases (23%/27%)• The course was more interesting for PhD students (very good: 62%/47%)• Course evaluation shoes more interest in practical topics comparing

theoretical issues– For 100 % of attendees the information of Introduction part was

sufficient– For 80 % of attendees the information about System Reasons of Errors

was sufficient– For 100 % of attendees the information about Strategies of

Improvement was insufficient– For 60 % of attendees the information about the Individual Reasons of

Medical Errors was insufficient– For 60 % of attendees the information how to Respond on Errors was

insufficient– For 80 % of attendees the information about the Types of Errors was

insufficient

Question?

Should Quality and Safety be included in the Curriculum of

Medical Universities?

Answer

Yes and it is not sufficient!

Future Plans

Training and Educational System for all level of

Medical Facilities

Already done• Foundation of

association “Medical Alliance for Quality and Safety” - MAQS

• Support from the Ministry of Health, Labour, and Social Affairs of Georgia

Thank YOU!

D4: Bridging the gap between undergraduate and postgraduate education

Jo-Inge Myhre, MD and Jessica Perlo, MPH

19th Annual International Forum on Quality and Safety in Healthcare

This presenter has nothing to disclose

April 10, 2014

Open School

Meet Dan

Took Courses & Created a Chapter

IHI Open School working group

Motivated and passionate volunteers

Weekly meetings

28

Leveraged Faculty

Partnership with institutional leadership, secured a mandate

Georgetown Center for Patient Safety

Georgetown Masters in Health System Administration

Georgetown School of Medicine

- Remove barriers- Buy faculty time- Encourage learners to participate

Recruited Interprofessional Members

Focused on Institutional Priorities

Engaged students/trainees in projects that were central to the strategic plan of their health care organization

For them, this meant:– Resident handoffs– Central line blood stream infections– Hospital readmissions– DVT prophylaxis improvement– Post discharge communication with community primary care

physicians– Hand hygiene– Central line air embolism prevention– Private partnership with an industry partner

31

Built the Case for Resident Involvement

System dysfunction is never more evident than when one is in training.– Because of the unfortunate nature of our training system,

trainees are often blamed for system errors

Because of this front line view, there is a tremendous will for change among trainees.

They are tremendously agile in their thought processes and are not attached to an ingrained status quo.

They rarely have the opportunity to work in an interprofessional manner.

Practicum example: CLABSITeam structure: – Health system administration student: project manager, Daniel

Bitman, BS– Physician champion: medicine resident, Daniel Alyeshmerni, MD– Nursing champion: Elizabeth Giunta, RN– Medical student: Orlando Sabbag, MSIII Peter Aleksandrov, MSIII– Nursing student: Lindsay Gingras

Barriers: time, focus, maintaining momentum

Results: – On vascular surgery unit, CLABSI rate

~ 3.2/1000 device days to 0 CLABSI rate for over one year

Continued Professional Growth

Presented work at conferences

Quality Improvement Chief Resident, DC VA

VA Quality Scholar Fellowship

IHI Improvement Advisor Training

Cardiology Fellowship, UMI

Faculty Advisor to UMI Chapter

Dan’s Experience(Beginning Prelicensure Learner)

Novice

(Advanced Prelicensure Learner)

Advanced Beginner

(Beginning Postlicensure Learner)

Competent

(Advanced Postlicensure Learner)

Proficient Expert

Student

Resident/Trainee/Junior Doctor

Faculty

QI Educator

OS Courses

OS Practicum

IHI IA, VA Quality Scholar

* Adapted from Ogrinc G, et al. A framework for teaching medical students and residents about practice-based learning and improvement. Acad Med. 2003; 78(7): 748-756

Actual State(Beginning Prelicensure Learner)

Novice

(Advanced Prelicensure Learner)

Advanced Beginner

(Beginning Postlicensure Learner)

Competent

(Advanced Postlicensure Learner)

Proficient Expert

Student

Resident/Trainee/Junior Doctor

Faculty

QI Educator

OS Courses

OS Courses

OS Courses

* Adapted from Ogrinc G, et al. A framework for teaching medical students and residents about practice-based learning and improvement. Acad Med. 2003; 78(7): 748-756

Early Postlicensure Barriers

Junior Doctor/Residents’ busy schedules

Not enough mentors who feel comfortable providing guidance

Lack of interest among trainees or belief that QI/PS is unimportant

Trainees’ transient presence on certain units or rotations

Lack of time to teach basic foundational principles of quality and safety

Lack of infrastructure (data managers, statisticians)

Lack of support from residency leadership regarding perceived value of these activities

Graduate Training Success Factors

1. Health system culture embraces the idea that residents and junior doctors are critical to quality and safety.

2. Engaged, capable faculty are willing to mentor.

3. Training projects are aligned with quality and safety institutional goals.

4. Early student exposure to QI concepts can create champions and a pathway for application once they enter the delivery system

5. Ongoing, experiential learning opportunities allow deep practice.

“Advance health care improvement and patient safety competencies in the next generation of health

professionals worldwide.”

IHI Open School Mission

The IHI Open School

Curriculum Content

Experiential Learning

Community Networks

• 23 online courses developed by world-renowned experts in the following topics:• Improvement Capability• Patient Safety• Person- and Family-Centered Care• Triple Aim for Populations• Quality, Cost, and Value• Leadership

• Mobile App for iPhone and iPad

IHI Open School Courses

Certificates

Certificate of Completion

30 contact hours available for nurses, physicians, and pharmacists

200,000+ students, residents, and professionals638 Chapters in 67 countries167 Chapters (26%)

are located in hospitals or health systems

Community

Allied Health Pro-fessions

Business

Dentistry

Engineering

Health Sci-ence & Ad-

ministrationLawMedicine

Nursing

Occupa-tional & Physical Therapy

Pharmacy

Physician AssistantSocial Work

Learner-driven quality improvement projects

Within local clinical setting

Opportunity to apply gained knowledge

Project Examples:– Reducing wait times – Improving hand hygiene compliance rates – Improving medication processes and implement

checklists

Quality Improvement Practicum (QI201)

Combining QI&PS with Leadership Training and EBM

Jo Inge Myhre, MD

Teaching assistant ”KLoK”

University of Oslo Medical School

Aim of KLoK

Through KLoK you’ll aquire knowledge and skills in EBM, leadership and quality improvement. This will aid you in your future professional role as an individual as well as a member/leader of teams.

Course overview

1. sem.: Introduction to patient safety (lecture)

6. sem.: Leadership and patient safety (seminar)

7. sem.: One week course in EBM (with exam)

10. sem.: EBM, Leadership and QI, Lectures, seminars and individual assignments during rotations in both hospitals and primary care

– Critical analysis of scientific publication and or guideline– Patient satisfaction– ”The patient’s journey”

11. semester: – Lectures and seminars– Group based assignment (QI Project proposal)

12. semester: – ”Survival week”– Student-BEST – Interprofessional simulation day– OSCE

Our experience

It’s hard to teach one subject without the others

Making it as clinical as possible is crucial

Invite students in the process

Create mechanism for continuous evaluation of the course

QUESTIONS?Email openschool@ihi.org

Follow us on Twitter@IHIOpenSchool

Like us on Facebook Download our App

• What’s happening in your organisation at the moment in terms of quality improvement and safety training programmes

Plan- do- evaluate- act cycle

Future curricula should be evaluated based on whether learner’s attitudes, knowledge and skills improve,

especially when improvements are ‘associated with intermediate clinical gains’

• Effectiveness of Teaching Quality Improvement to Clinicians: a systematic review (Boonyasai et al, 2007)

• Quality Improvement in medical education: current state and future directions (Wong et al, 2012)

Kirkpatrick Evaluation Model

4 Results

3 Behaviour

2 Learning

1 Reaction

Kirkpatrick Evaluation Model

4 Results

3 Behaviour

2 Learning

1 Reaction

Results for the organisation: quality, efficiency, productivity

Change in behaviour (knowledge/skills applied on the job)

Increase in knowledge, skills or attitudes?

Satisfaction- Valuable, Relevant?

• Pre- and post- programme: QI attitudes, knowledge & skills (Likert scale rating)

• Benefit to learners• Benefit to organisation• 2 year follow-up questionnaire

Challenges

Lack of tools to assess QI

knowledge & skills (level 2)

PLUS control group

Difficulties measuring behaviour

change (level 3)

Demonstrating benefits to

patients/organisation (level 4)

Questionnaires

• Up to level 2 – attitudes, self reported knowledge and skills

• Not so good at measuring behaviour, values and competencies

‘Validating a questionnaire’

• Modified Delphi technique - adapt

• Then trial it with participations – adapt

• Then sit down with them while they are filling it out and find out what they are thinking - adapt

…but to do behaviours, values and competencies (level 3)you need interviews

• Probably the best way• Expensive and takes time – long follow up is

best

• What are you going to take away from this talk?

The Road Ahead