Embed Size (px)
Citation preview
1
Nzhde Agazaryan, PhD, DABR
Professor of Radiation Oncology Professor of Physics and Biology in Medicine
Chief of Clinical Medical Physics UCLA Health System Quality Officer
The UCLA Experience with RO-ILS:
Developing a Culture of Safety, Processes, and Metrics
Quality and Safety Oversight
Committee
(QSOC)
Chaired by
Quality Officers
Quality and Safety Oversight
Committee
(QSOC)
Chaired by
Quality Officers
Peer Review
Prospective Chart Round
Peer Review
Prospective Chart Round
Weekly Continuous
Quality Improvement
Committee
(CQI)
Weekly Continuous
Quality Improvement
Committee
(CQI)
Event Reporting
and
Response System
Event Reporting
and
Response System
Incident Review
Committee
Incident Review
Committee
Laboratory Safety
Laboratory Safety
Lessons Learned
Committee
(M&M)
Lessons Learned
Committee
(M&M)
Quality and Safety Organization
Programs, Systems and Processes
Various Committees
Nzhde Agazaryan, Ph.D., DABR
Professor of Clinical Radiation Oncology Professor of Physics and Biology in Medicine
Chief of Clinical Medical Physics FPG Quality Officer Co-Chair, Quality Oversight Committee
Phillip Beron, M.D.
Associate Professor of Radiation Oncology
FPG Quality Officer Co-Chair, Quality Oversight Committee
Physicist and Physician Quality Officers
2
Event Reporting
and
Response System
Event Reporting
and
Response System
RO-ILS and SOFI
RO-ILS and SOFI Incident Reporting Policy
Continuous Quality
Improvement
Committee
(CQI)
Continuous Quality
Improvement
Committee
(CQI)
Two Weekly Meetings
12-16 attendees, 1.5 hours total
2 Administrative Assistants
• Review Incidents
• Assign Champions
• Review Follow-Ups
• Review QI Projects
• Review A3 Projects
• Review APEX Accreditation Progress
3
RO-ILS – Incident Review Committee Agenda (Weekly)
RO-ILS – Instructions to Incident Champions
RO-ILS – Instructions to Incident Champions
4
Quality Working Group Meeting Agenda (Weekly)
Dashboard Metrics (10) – Dr. Agazaryan
Monthly Incident Review (5) – Dr. Beron
Physics Billing Updates (10) – Phil Chow, MS
CT Sim Ordering Go-Live Updates (10) – Walter Chin, RN
APEx Updates (5) – Dr. Agazaryan
File Server Updates (10) – Kathy Rose
Quality and Safety Oversight Committee (QSOC) Meeting Agenda (Monthly)
5
UCLA Health Quality Council Presentation (Yearly)
# Hashtags RO-ILS Milestones
0
20
40
60
80
100
120
140
160
Paper RO-ILS
# Incidents
Number of Incidents Per Year
6
Current Sites: • Flagship Sites: UCLA - Westwood / Santa Monica • Professional Services: Northridge Hospital, Los Alamitos Hospital, 21C – El
Segundo, Orange County Sites (Anaheim, Fountain Valley), Good Samaritan Hospital
Planned: • UCLA – Santa Clarita, JV – RT-Hyperthermia, JV – Proton Center
Current Sites: • Flagship Sites: UCLA - Westwood / Santa Monica • Professional Services: Northridge Hospital, Los Alamitos Hospital, 21C – El
Segundo, Orange County Sites (Anaheim, Fountain Valley), Good Samaritan Hospital
Planned: • UCLA – Santa Clarita, JV – RT-Hyperthermia, JV – Proton Center
RO-ILS Reporting from Multiple Sites
7
19
8
13
6
14
11
6
8 9
12
6
10
2
7
15 14
24
10
15
17
15
8
12
24
12 12
15 16
15
13 12
0
5
10
15
20
25
30
Jan
Feb
Mar
Ap
r
May
Jun
e
July
Au
g
Sep
t
Oct
No
v
De
c
ROILS Event Reporting Analysis 2015 - 2017
2015 2016 2017
0
2
4
6
8
10
12
Number ofIncidents
Number ofIncidents Closed
Number ofIncidents Open
A3 ProjectsInitiated
A3 ProjectsCompleted
Quality DashboardMonthly Incidents (June 2017)
12 9 3 2 0
12
9
3
2
0
QUALITY DASHBOARD MONTHLY INCIDENTS (JUNE 2017)
0
50
100
150
200
250
300
350
400
450
Number ofIncidents
Number ofIncidents Closed
Number ofIncidents Open
A3 Projects Total A3 Projects InProgress
A3 ProjectsCompleted
Quality DashboardAll Incidents To Date
448 288 160 37 25 12
448
288
160
37 25 12
QUALITY DASHBOARD ALL INCIDENTS TO DATE
RO-ILS Incidents and A3 Projects
8
0 20 40 60 80 100 120 140 160 180 200
Total
Open
Closed
Number of Open vs. Closed ROILS Events 2016 and 2017 YTD
2017 2016
0%
10%
20%
30%
40%
50%
60%
70%
80%
Dec-16 Jan-17 Feb-17 Mar-17 Apr-17 May-17
ROILS EVENTS REVIEW PROCESS 6 Month Running Totals - % Closed
Incident Review and Quality Improvement Process
Rapid
Review
Process
24 hours
No Reportable Incidents
Since 2013
9
Incident Review and Quality Improvement Process
Rapid
Review
Process
24 hours
No Reportable Incidents
Since 2013
More Efficient and Quantitative Processes
A3 Initiatives
ROI-LS Helped
Discover Unanticipated Failure Modes
Communicate Best Practices
Make Processes Efficient
Optimize Continuous Quality Improvement Process
Implement Practice Improvements
10
Quality Improvement and A3 Jackets
Quality and Safety Administrative Support
→
Kathy Rose Executive Assistant to the Chair Educational Programs Administrator
←
Julia Melkonian Administrative Analyst
Each Workflow Step Requires People with Expertise
Patient Assessment
Imaging for Planning
Treatment Planning
PreTX Review
On TX Quality Review
Treatment Delivery
Post TX
Review
MD RTT
MD
Dosimetry
Dosimetry
MD
Physics
Physics
MD
Dosimetry
RTT
Physics
MD
Dosimetry
RTT
MD
Dosimetry
Physics
Physics
MD
11
Technology at UCLA
3D Conformal 3D Conformal IMRT/VMAT IMRT/VMAT SRS/SBRT SRS/SBRT
Gantry mounted KV IGRT Gantry mounted KV IGRT
MRIgRT MRIgRT
TBI TBI
MVCT MVCT
IORT IORT Eye plaque Eye plaque
Ceiling/Floor-Mounted kV IGRT Ceiling/Floor-Mounted kV IGRT
Adaptive planning Adaptive planning Knowledge based planning Knowledge based planning
Optical surface tracking Optical surface tracking 4DCT 4DCT US Real Time HDR US Real Time HDR
Value Based Growth
12
34
0
10
20
30
40
50
60
70
80
90
2014 2015 2016
Fractionated vs SBRT Patients Treated with Viewray
Fractionated
SBRT
# Hashtags
Primary Incident Category in RO-ILS
0
10
20
30
40
50
60
70
13
Brachytherapy - Secondary Category of Incidents
0
1
2
3
4
5
6
7
Fall Insurance Schedule Eye plaque IT HDR Catheter
Physician - Secondary Category of Incidents
0
1
2
3
4
5
6
7
8
9
Policy Procedure Scheduling Communication Incorrect Orders
Therapist - Secondary Category of Incidents
0
1
2
3
4
5
6
7
8
9
14
IT - Secondary Category of Incidents
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Server Install Support Epic Aria
0
5
10
15
20
25
30
35
40
Axi
s Ti
tle
Shift
ExacTrac
Timeout
Planning
Bolus
Robotics
Multiple sites
ViewRay
Worksheet
TBI
2nd check
Orders
Naming
Medication
Equipment
Education
TLD
Partial treatment
Mechanical
Support
Aria
CT simulation
Patient experience
Insurance
Communication
Clutter
Procedure policy
Physics
Inpatient
Eyeplaque
Channel
Careconnect
Scheduling
15
287 Suggestions for improvement
Documentation
Communication
Time Out Processes
Policy and Procedure Updates and Changes
Initiation of A3 projects
0
5
10
15
20
25
30
35
A3 Projects Total A3 Projects In Progress A3 Projects Completed
34
22
12
QUALITY DASHBOARD A3 PROJECTS
A3 PROJECT
Clinical Treatment Planning (CTP) Directives
Completeness Project
Project Lead James Lamb, Ph.D. Assistant Professor of Radiation Oncology
M O V E R S
16
CTP Completeness Project
Introduction:
• Verifying consistency of the patient’s treatment with the Clinical Treatment Plan (CTP)
document is an essential part of the treatment quality control performed by medical
physicists, radiation therapists, and other staff members. It has been anecdotally
observed that the CTP form is in error (incorrectly filled out or incomplete) at the time
of treatment plan approval. CTP form errors hinder quality control checks and increase
the probability of a treatment error. We attempted to quantify the frequency of CTP
form errors at the time of plan approval and make recommendations on how to reduce
the error rate.
Methods:
• We asked all physicists and dosimetrists to report instances of CTP form errors at the
time of second check. Data was collected for several quarters.
CTP Completeness Project
Introduction:
• Verifying consistency of the patient’s treatment with the Clinical Treatment Plan (CTP)
document is an essential part of the treatment quality control performed by medical
physicists, radiation therapists, and other staff members. It has been anecdotally
observed that the CTP form is in error (incorrectly filled out or incomplete) at the time
of treatment plan approval. CTP form errors hinder quality control checks and increase
the probability of a treatment error. We attempted to quantify the frequency of CTP
form errors at the time of plan approval and make recommendations on how to reduce
the error rate.
Methods:
• We asked all physicists and dosimetrists to report instances of CTP form errors at the
time of second check. Data was collected for several quarters.
CTP Completeness Project
Recommended Actions:
• The CTP form should be created in ARIA at the time of simulation.
• The attending physician should fill out the CTP at the time of contour approval to the
extent possible, with the understanding that the dose prescription, dose volume
constraints, technique, imaging, and other components of the treatment plan, may
change during the planning process.
• The attending physician should review the CTP with the planner at the time of plan
approval and correct the form if necessary.
17
CTP Completeness Project
Recommended Actions:
• The CTP form should be created in ARIA at the time of simulation.
• The attending physician should fill out the CTP at the time of contour approval to the
extent possible, with the understanding that the dose prescription, dose volume
constraints, technique, imaging, and other components of the treatment plan, may
change during the planning process.
• The attending physician should review the CTP with the planner at the time of plan
approval and correct the form if necessary.
23
10
5
1 2
0
5
10
15
20
25
Q2 2016 Q3 2016 Q4 2016 Q1 2017 Q2 2017
CTP A3 Project Data
Pattern Analysis (similar incidents) with the Electronic Incident Learning System
18
PROJECT
Timeliness of Patient Treatments
Project Leads Phillip Chow, M.S., Medical Physicist Nzhde Agazaryan, Ph.D., Professor
M O V E R S
Fishbone Diagram of Patient Care Pathway
5. Treatment delivery
3. Treatment planning 5.1 Prepare room / setup 7. Treatment Completion
1. Consultation 3.1 Patient data import 5.2 Patient identification 7.1 Physician final patient evaluation
1.1 Referral 3.2 Normal organ and target volume delineation 5.3 Patient setup / localization 7.2 Therapist chart completion
1.2 Medical history 3.3 Dosimetric objectives 5.4 Time-out: confirm patient/site/settings 7.3 Physicist final chart review & dose summary
1.3 Patient consult 3.4 Treatment technique design/optimization 5.5 Deliver treatment 7.4 Physician end-of-treatment summary
1.4 Treatment recommendation 3.5 Physician review / approval of treatment plan 5.6 Complete session, document delivery 7.5 Follow-up plan
2.5 Transfer data to dosimetry 4.6 QA tests / measurements and analysis 6.5 Weekly physicist chart review 8.3 Lab tests / imaging studies
2.4 Summarize setup information 4.5 Mount filters / blocks 6.4 Weekly physician management 8.2 Primary/referring physician follow-up
2.3 CT simulation – positioning, patient data acquisition 4.4 Independent check of parameters and setup instructions 6.3 Weekly nurse management 8.1 Follow-up radiation oncologist evaluations
2.2 Physician directive - objectives 4.3 Device/filter manufacture 6.2 Daily therapist observation 8. Follow-Up
2.1 Patient consent 4.2 Treatment parameters exported 6.1 Daily receptionist observation
2. Simulation 4.1 Treatment plan documentation 6. On-Treatment Management
4. Preparation
Fishbone Diagram of Patient Care Pathway
5. Treatment delivery
3. Treatment planning 5.1 Prepare room / setup 7. Treatment Completion
1. Consultation 3.1 Patient data import 5.2 Patient identification 7.1 Physician final patient evaluation
1.1 Referral 3.2 Normal organ and target volume delineation 5.3 Patient setup / localization 7.2 Therapist chart completion
1.2 Medical history 3.3 Dosimetric objectives 5.4 Time-out: confirm patient/site/settings 7.3 Physicist final chart review & dose summary
1.3 Patient consult 3.4 Treatment technique design/optimization 5.5 Deliver treatment 7.4 Physician end-of-treatment summary
1.4 Treatment recommendation 3.5 Physician review / approval of treatment plan 5.6 Complete session, document delivery 7.5 Follow-up plan
2.5 Transfer data to dosimetry 4.6 QA tests / measurements and analysis 6.5 Weekly physicist chart review 8.3 Lab tests / imaging studies
2.4 Summarize setup information 4.5 Mount filters / blocks 6.4 Weekly physician management 8.2 Primary/referring physician follow-up
2.3 CT simulation – positioning, patient data acquisition 4.4 Independent check of parameters and setup instructions 6.3 Weekly nurse management 8.1 Follow-up radiation oncologist evaluations
2.2 Physician directive - objectives 4.3 Device/filter manufacture 6.2 Daily therapist observation 8. Follow-Up
2.1 Patient consent 4.2 Treatment parameters exported 6.1 Daily receptionist observation
2. Simulation 4.1 Treatment plan documentation 6. On-Treatment Management
4. Preparation
19
Database for Tracking Treatment Planning Process
MOC Project
Database for Tracking Treatment Planning
Process MOC Project
ID Status
1 Patient Consulted
2 Simulation Pending
3 Simulation Complete
4 Pending Insurance
5 CT Imported; waiting for MR
6 MR Imported; waiting for CT
7 Contours Needed
8 Contours ready for review
9 Rad Onc Contour approval needed
10 Neuro Contour approval needed
11 Contours Approved
12 Plan Approval Needed
13 Rad Onc Plan approval needed
ID Status
14 Neurosurery Plan approval needed
15 Plan approved
16 QA Pending
17 Treatment to be Scheduled
18 Planning In-progress
19 On Hold
20 On Treatment
21 Treatment Complete
22 Complete; Not Treated
23 Second Check Needed
24 Authorization Pending
Patient Simulation to Contours Approved Intervention, Improvement, Sustainability and Continuous Improvement
20
Patient Simulation to Contours Approved Intervention, Improvement, Sustainability and Continuous Improvement
DASHBOARD
• Text
DASHBOARD
• Text
21
Patient Simulation to Contours Approved Intervention, Improvement, Sustainability and Continuous Improvement
Irradiated Area
Markers
Not A Reportable Incident
Incorrect Vertebral Body Alignment
M O V E R S
A flashlight and spine can be successfully fused without
alerting the therapist of any error
22
Automated Patient Identification and Localization
Error Detection Using X-Ray Setup Images
…. can a computer
differentiate two patients
from their x-rays?
If a computer algorithm can differentiate these two men from their pictures….
Automated Patient Identification and Localization
Error Detection Using X-Ray Setup Images
Histogram of similarity measures of correctly and
incorrectly matched cranial radiotherapy patients
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
-1.5
-1.4
5
-1.4
-1.3
5
-1.3
-1.2
5
-1.2
-1.1
5
-1.1
-1.0
5 -1
-0.9
5
-0.9
-0.8
5
-0.8
-0.7
5
-0.7
-0.6
5
-0.6
-0.5
5
-0.5
-0.4
5
-0.4
-0.3
5
-0.3
-0.2
5
-0.2
-0.1
5
-0.1
-0.0
5 0
No
rmali
zed
Fre
qu
en
cy
Similarity Measure
Cranial Patient ID
Correct Patient
Incorrect Patient
23
Histogram of similarity measures of spinal cases
localized to the correct and incorrect vertebral body
0
0.05
0.1
0.15
0.2
0.25
0.3
0.35
-1.5
-1.4
5
-1.4
-1.3
5
-1.3
-1.2
5
-1.2
-1.1
5
-1.1
-1.0
5 -1
-0.9
5
-0.9
-0.8
5
-0.8
-0.7
5
-0.7
-0.6
5
-0.6
-0.5
5
-0.5
-0.4
5
-0.4
-0.3
5
-0.3
-0.2
5
-0.2
-0.1
5
-0.1
-0.0
5 0
No
rmali
zed
Fre
qu
en
cy
Similarity Measure
Vertebral Body Localizations
Correct Localization
Incorrect Localization
Example of spinal treatments localized to the correct and incorrect vertebral body
L/R Radiographs
L/R DRRs
The method will likely result in overall practice improvement
Secondary Category Of Incidents
0
5
10
15
20
25
30
24
Plan Name Standardization
Prospective Chart Rounds
RO-ILS Submission Resulted in Chart Rounds Schedule Change
MOVERS 0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Prior to February 2015 February - March 2015 Currently
0%
68% 75%
Percent Patients Reviewed in Chart Rounds Prior to Treatment
A3 PROJECT
CT Simulation Ordering Accuracy
M O V E R S
25
Problem Statement
Over the last calendar year there have been
38 near misses, wrong set ups, ambiguities in SIM orders
reported in RO-ILS
Kaizen (改善) is the Japanese word for "continual improvement“ or "change for better“
Kaizen events are short duration improvement projects with a specific aim for improvement
26
Opportunities Opportunity Impact Frequency
1. Diagnosis Change- no code HIGH HIGH
4. Rushing HIGH HIGH
5. Not able to update form HIGH HIGH
6. Incorrect Info HIGH HIGH
10. Order is unclear HIGH HIGH
11. Authorization not cleared HIGH HIGH
13. Multiple sites on one order HIGH HIGH
14. Clarity on intentions HIGH HIGH
21. CT Order Form incorrect HIGH HIGH
22. MD not available HIGH HIGH
3. CT Form not mirror image HIGH MID
15. Level Loading Machines HIGH LOW
2. Multiple sites on one order LOW HIGH
12. Patients being scheduled when MD not available LOW HIGH
16. IV contrast not requested on RT order LOW HIGH
17. Labs not ordered LOW HIGH
18. Missed pregnancy test LOW HIGH
19. Consents granularity LOW HIGH
7. AVS not editable if done by attending LOW LOW
20. Time-outs for inpatient LOW LOW
Future State (July) – 2 Step Process Scheduling and Setup Separated Went from 38 Questions to 16 Questions
CURRENT ORDER FORM
1. Is a separate simulation verification visit requested,
or will the patient be treated the day of simulation
verification? {RAD ONC SIM VISIT_AMB_UCLA:29021}
2. Goal Start Time: {RAD ONC goal start:28613}
3. Is a ViewRay Simulation being requested? {YES
NO:23722::"no"}
4. Schedule CT Sim on (date): ***
5. Schedule CT Sim at: {RAD ONC Location_UCLA:28744::"UCLA
WW”}
6. Need Consent signed at time of Sim: {YES NO:23722::"no"}
7. Clinical Trial: {YES NO:23722::"no"}
8. MRI requested: {YES NO:23722::"no"} (diagnostic)
9. Last Labs: Lab Results
10. IV Contrast: {RAD ONC Contrast:28610}
11. Oral Contrast: {YES NO:23722::"no"}
12. Clinical Site and Treatment Type (Free Text)
13. Type of Treatment: {Treatment Type:24373}
14. Treatment Site: ***
15. Length of Treatment: *** Fractions
3 P's
16. Pacemaker: {Pacemaker clearance:28741}
17. Pregnancy: Is the patient a female age 55 or under? {Pregnancy:28742}
18. Previous Radiation: {Previous radiation:28745}
Patient Set-Up and SIM Instructions
19. Type of Simulation: {Sim type:28674}
20. Increased Time Required for Simulation: {Additional time
options:28859::"None"}
21. Patient Positioning: {Simulation Positioning:24831}
22. Immobilization: {Immobilization:24379}
23. Mouthpiece to be made by: {RAD ONC
mouthpiece_UCLA:28612::"None"}
24. Wire: {RAD ONC Wire:28739::"None"}
25. Bolus: {YES NO:23722::"no"}
26. Additional Devices: {Simulation Devices:20322::"None"}
27. Special Markers: {RAD ONC Marker:28740::"None"}
28. Physician Check Set-up: {RAD ONC Physician
Check_UCLA:28746}
29. Place Iso: {RAD ONC Place Iso_ucla:28747}
ADDITIONAL INFO
30. Patient Instructions: {RAD ONC Patient Instructions_UCLA:28748}
31. Referrals Requested: {Rad Onc EBRT Referral_ucla:28614}
32. Additional Comments:
27
Nzhde Agazaryan, PhD, DABR
Professor of Clinical Radiation Oncology Professor of Physics and Biology in Medicine
Chief of Clinical Medical Physics UCLA Health System Quality Officer
The UCLA Experience with RO-ILS:
Developing a Culture of Safety, Processes, and Metrics
