Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
Joseph Cafazzo, PhD PEng
Monday, June 28, 2010
“Modern Times”, United Artists 1936
Monday, June 28, 2010
“Science Finds, Industry Applies, Man Conforms”
Slogan from the 1933 Chicago World’s Fair
Monday, June 28, 2010
"People Propose, Science Studies, Technology Conforms"
Don Norman’s person-centered motto for the 21st century.
"Things that make us smart." Addison-Wesley, 1993.
Monday, June 28, 2010
Monday, June 28, 2010
Monday, June 28, 2010
Monday, June 28, 2010
Poor Design
• Images -
• c prompt
• ballot
• friendly fire
Monday, June 28, 2010
“We have a serious training problem that needs to be corrected”
“We need to know how our equipment works: when the battery is changed, it defaults to its own location. We’ve got to make sure our people understand this”
Senior Air Force Official
Monday, June 28, 2010
Monday, June 28, 2010
“Recount”, HBO 2008
Monday, June 28, 2010
Monday, June 28, 2010
Monday, June 28, 2010
You are not alone
Radiation Oncologyand
Patient Safety
Monday, June 28, 2010
How do we prevent error?
Hierarchy of Effectiveness1. Forcing functions & constraints
2. Automation/computerization
3. Simplification/standardization
4. Reminders, checklists, double checks
5. Rules & policies
6. Training & education
tech focus
people focus
Monday, June 28, 2010
Human Factors in Healthcare
• IOM 1999: 44, 000 to 98, 000 US deaths annually due to medical error
• Baker, Norton 2004: 9, 000 to 23, 000 deaths annually due to preventable adverse events
Monday, June 28, 2010
User Centered Design
Analyze userrequirements
Design & Prototype
Evaluate
Monday, June 28, 2010
Design & Prototype
Monday, June 28, 2010
Evaluate• Expert/Heuristic Analysis • Visibility
• Consistency
• Efficiency
• Flexibility
• Autonomy
Monday, June 28, 2010
Evaluation Criteria• Efficiency & Speed
• Task completion time
• Task accuracy & error frequency
• Number of requests for help
• Number of attempts to correct errors
• Impact & Severity of error (patient safety)
• Workload/Difficulty scale (1-7)
• User feedback & preferences
Monday, June 28, 2010
Evaluate
• Usability Testing • Representative users
• Realistic scenarios
• Think-aloud protocol
• Record video & audio
• Qualitative & quantitative
Monday, June 28, 2010
Monday, June 28, 2010
Evaluation of a Functional Prototype
Monday, June 28, 2010
Improving Patient Safety during Radiation Therapy through Human Factors
Alvita ChanPrinciple Investigators: Dr. Mohammad Islam, Dr. Joe Cafazzo
Co-Investigators: Dr. Tony Easty, Dr. David Jaffray
Monday, June 28, 2010
Systematic Literature Review• A systematic review on existing literature was conducted to
understand sources of errors• 7 distinctive studies were found• 40-50% errors occur during treatment delivery• Most frequent errors:
– Incorrect patient position (e.g. wrong site, wrong SSD, wrong couch height)
– Incorrect treatment accessories (i.e. Shielding blocks, bolus)
Yeung TK et al. Radiotherapy and Oncology. 2005, 74:283-291Marks LB et al. International Journal of Radiation Oncology, Biology and Physics. 2007, 69:1579-1586
Monday, June 28, 2010
Yeung TK et al. Radiotherapy and Oncology. 2005, 74:283-291Yeung TK et al. Radiotherapy and Oncology. 2005, 74:283-291
40.4%
Monday, June 28, 2010
Work Environment Analysis• Identified issues and developed
recommendations regarding treatment/control room setup
– Ambient Noise/Interruptions– Furniture/computer features
and placement– Storage– Lighting– Temperature and humidity
Monday, June 28, 2010
Monday, June 28, 2010
Workflow Analysis• Mapped out entire workflow of the treatment delivery
process using the Unified Modeling Language (UML) 2.0 Activity Diagrams
* From 24 observations, 6 for each treatment team
Monday, June 28, 2010
Monday, June 28, 2010
Heuristic Evaluation• Systematic inspection of current user interfaces according to a list of
guidelines/heuristics – E.g. Visibility of system state, minimize memory load, informative
feedback, flexibility and efficiency, prevent errors– Identified 75 usability problems
0
10
20
30
40
High Medium Low Positive
3737
2018
Usability Problems by Severity Ratings
Num
ber o
f Pro
blem
s Id
entif
ied
Severity Ratings
Zhang et al. Journal of Biomedical Informatics. 2003, 36: 23-30
Monday, June 28, 2010
Workflow Analysis • Identified areas that can be improved:
– Heavy reliance on policy• Checking process during patient setup
Hierarchy of effectiveness in preventing errors
1. Forcing functions and constraints
2. Automation / Computerization
3. Simplification / Standardization
4. Checks, double checks, checklists
5. Policy and trainingLeast effective
Institute for Safe Medication Practices. ISMP Medication Safety Alert. 1999, 4 (11)
Hierarchy of Effectiveness
Monday, June 28, 2010
How can the process be improved?
• Focused on checking process during patient setup
• Redesigned components of the user interface:
– To lessen the reliance on policy and procedure– To improve efficiency of the checking process– To display information based on the users’ needs
• Design process: Expert input + 2 focus groups
12
Monday, June 28, 2010
Monday, June 28, 2010
Monday, June 28, 2010
Redesigned InterfaceDRR is displayed visibly to improve
checking compliance
Patient picture is displayed visibly for easy identification
Monday, June 28, 2010
Redesigned Interface
Forcing function/checklist is used to improve checking compliance
New messages and outstanding items are
highlighted
Statuses of approval dates and images are
clearly displayed
DOB given by patient is checked by system
Both SSD and depths are displayed
Notes that required certain actions to be
performed are highlighted
Monday, June 28, 2010
Redesign of Existing System
Current interface:14 mouse clicks
Redesigned interface:4 mouse clicks
Improved efficiency
17
Monday, June 28, 2010
Experimental Evaluation - Usability Testing
• 16 radiation therapy students from PMH were recruited to participate
• An actor played the role of a second therapist• Performed patient setup tasks using both the current and
redesigned interfaces
18
Monday, June 28, 2010
Experimental Design
• 2 interfaces x 4 scenarios
• 3 out of the 4 scenarios were designed with a high potential for common errors to occur:
– Overlooking an important note– Shifting couch incorrectly – Overlooking a change in
approval dates
19
Monday, June 28, 2010
Rate of Errors
P<0.04
P<0.01
20
Monday, June 28, 2010
Task Time
P<0.02
21
Monday, June 28, 2010
User Satisfaction
• Post-test Questionnaire:
22
Monday, June 28, 2010
User Satisfaction
Attributes Current Redesigned Significance
Better at showing the information needed
0% 94% Yes (P<0.01)
Better at drawing attention to important items
0% 100% Yes (P<0.01)
Better for catching an important message, a change in approval date, or an unapproved image
0% 100% Yes (P<0.01)
Enable treatment to be delivered more safely
13% 81% Yes (P<0.01)
Easier to use 25% 56% No
More efficient 19% 63% No
Preferred system 25% 56% No
23
Monday, June 28, 2010
Limitations
• Students vs. experienced therapists
• Mannequins vs. real patients
• Redesigned interface was only a prototype, not all features were functional
• Only the first iteration of the design cycle
24
Monday, June 28, 2010
Acknowledgements
Other: - Catherine Dupuis- Mary Stewart-Hardy- Varuna Prakash- Mark Fan- Radiation therapists at
PMH- Radiation therapist
students
Project team:- Dr. Joe Cafazzo- Dr. Mohammad Islam- Dr. Tony Easty- Dr. David Jaffray- Tara Rosewall
Funding agencies:- NSERC, NPSF, AAPM, CPSI
26
Monday, June 28, 2010
Summary
•Humans are fallible. We will err.
•We can reduce frequency and consequences of errors by designing systems within our limitations
•Solutions should aim towards the top of the Hierarchy of Effectiveness
Monday, June 28, 2010
Monday, June 28, 2010
healthcarehumanfactors.com
Joseph Cafazzo, PhD PEng
Monday, June 28, 2010