Measuring the cost of quality - Lab Quality Confab€¦ · Measuring the cost of quality ......

Measuring the cost of quality

How to Collect the Right Data and Use it to Demonstrate the Full and Positive Impact of Quality Improvement Projects

Gaurav Sharma, MD, FCAP Douglas C. Wood, ASQ CQE, SSBB

November 4, 2015 Lab Quality Confab

Objectives

•! See how a non conformance management system as required by ISO 15189 can be used to quickly identify, quantify, and resolve workflow issues – even those originating outside the laboratory.

•! Understand how cost of quality can be used to estimate ROI of improvement projects.

•! Understand how to create a compelling business case for building a defect management system.

© 2015 College of American Pathologists. All rights reserved. 2

Work People Processes

Work People Processes

“VISIBLE” “INVISIBLE”

Work People Processes

“VISIBLE” “INVISIBLE”

People must be well trained and proficient

Processes must be well designed and measurable

Work People Processes

People must be well trained and proficient

Processes must be well designed and measurable

Ineffective Management

Effective Management

A Quality Journey at Fairlane Medical Center

© 2015 College of American Pathologists. All rights reserved. 7

Ambulatory Clinics and the largest free standing ER in US

24/7 on-site laboratory >40,000 phlebotomy in OPL

Order Process in 2013

Provider orders a

test in EMR

Patient is collected in the clinic

Specimen is received in the lab for testing Patient is

collected in the

outpatient lab

Or

Order Process in 2014

Provider orders a

test in EMR

Patient is collected in the clinic

Specimen is received in the lab for testing Patient is

collected in the

outpatient lab

CLINIC COLLECT ORDER

LAB COLLECT ORDER

What happens when CC walks into the lab?

Provider orders a

test in EMR

Patient is collected in the clinic

Patient arrives in OPL

CLINIC COLLECT ORDER

What happens when CC walks into the lab?

Provider orders a

test in EMR

Patient is collected in the clinic

Patient is now

waiting

CLINIC COLLECT ORDER

What happens when CC walks into the lab?

Provider orders a

test in EMR

Patient is collected in the clinic

Specimen is received in the lab for testing

Patient is collected in the outpatient lab

CLINIC COLLECT ORDER

Patient experience

Clinic supplies the defect

Laboratory cannot accept defect or performs defective work

Clinic receives defective service

This is the lab’s fault,

they should fix it!

This is the clinic’s fault,

they should fix it!

This is the computer’s

fault, somebody

should fix it!

A clinic collect was received in

the lab •! Why?

The provider selected the

incorrect order type

•! Why?

Was not aware of the difference between clinic vs lab collect

•! Why?

The provider was trained but

the training was not

recently done

•! Why?

Rollout was phased and

training occurred along with other

tasks

LEAN Principles Liker: The Toyota Way, 2004

1. Create a long term philosophy 2. Continuous process ‘Flow’ 3. Use ‘Pull’ rather than ‘Push’ 4. Level the workload 5. Build a culture to fix the problem 6. Standardize tasks 7. Use visual controls 8. Use tested technology 9. Grow leaders 10. Develop people 11. Respect partners and suppliers 12. Go and See 13. Make decisions by consensus 14. Become a learning organization

* applies to the problem of clinic collect vs lab collect

•! Specific to content, sequence, timing, location and expected outcome Activity

•! Direct and specific •! Feedback mechanism exists Connection

•! Service should be pre defined •! No loops or forks Pathway

•! By PDCA cycles and by those who do the work Improvements

Rules of Standard Work Henry Ford Production System LEAN Training

* apply to the problem of clinic collect vs lab collect

•! Excess transportation •! Excess inventory •! Excess motion •! Excess waiting •!Over-production •!Over-processing •!Defects

Types of Waste Liker: The Toyota Way, 2004

HFPS LEAN Training

•! Excess transportation •! Excess inventory •! Excess motion •! Excess waiting: Patients waited •!Over-production •!Over-processing: Lab and clinic did rework •!Defects: Miss a test / order incorrect test

Types of Waste Liker: The Toyota Way, 2004

HFPS LEAN Training

Customer Supplier Team

Describe the problem

Describe current condition

Identify root cause

Develop and implement an action plan

Review and monitor performance

Describe the problem •! Training/Practice Gap !!Waste

Describe current condition •! Gather baseline data using deviation

management system

Address root cause •! Focused re-training

Develop and implement an action plan •! Labs will track defects, relay back to

clinics, EMR retraining continues

Review and monitor performance •! Use deviation management system

Customer Supplier Team: CC vs LC

Dr. Drucker once said……

“What’s measured improves”

How do we track defects? How do we gauge efficacy?

Conformity

Non Conformity

Did the work occur as expected?

Yes

No

Deviation Management System ISO 15189 accreditation requirements

4.9.1 Laboratory management shall have a policy and procedure to be implemented when it detects that any aspect of its examinations does not conform with its own procedures or the agreed upon requirements of its quality management system or the requesting clinician. These shall ensure that

a) personnel responsible for problem resolution are designated, b) the actions to be taken are defined, c) the medical significance of the nonconforming examinations is considered and where

appropriate, the requesting clinician informed, d) examinations are halted and reports withheld as necessary, e) corrective action is taken immediately, f) the results of nonconforming examinations already released are recalled or

appropriately identified, if necessary, g) the responsibility for authorization of the resumption of examinations is defined, and h) each episode of nonconformity is documented and recorded, with these records being

reviewed at regular specified intervals by laboratory management to detect trends and initiate preventive action.

NOTE Nonconforming examinations or activities occur in many different areas and can be identified in many different ways, including clinician complaints, quality control indications, instrument calibrations, checking of consumable materials, staff comments, reporting and certificate checking, laboratory management reviews, and internal and external audits.

The Plan

RECORD (Paper)

Clinic collect vs

Lab collect defect

RESOLVE by calling

clinic

RECORD (Excel)

All defects from the clinics

ASSESS ? High Impact

? Frequent ? 1 Person

COLLABORATE Customer Supplier Meetings

RETRAIN and

MONITOR

Stop

YES NO

Paper Form

Excel File

4.9 Tracking

99% Reduction in Defective Orders

99% reduction

Success lies in the last 2 steps…

RECORD (Paper)

Clinic collect vs

Lab collect defect

RESOLVE by calling

clinic

RECORD (Excel)

All defects from the clinics

ASSESS ? High Impact

? Frequent ? 1 Person

MEET Customer Supplier Meetings

RETRAIN and

MONITOR

Stop

YES NO

What they say about our deviation management system….

LAB STAFF: I have more time to

focus on testing, rather than re-

working unclear orders…..

CLINIC STAFF:

I have fewer interruptions and

focus on my work…

MANAGERS: A systematic focus and discipline of problem solving

makes my life easy…

PHLEBOTOMISTS

Orders are clear and we can focus on accurate draws…

99% Reduction in Defective Orders

99% reduction

What is the “cost of quality” for this example?

•! Laboratory expenses can be initially divided into two areas.

–!Expenses of running the lab without errors, retesting, or any mistakes.

–!Expenses that involve mistakes, attempting to prevent mistakes, and dealing with the consequences of mistakes.

© 2015 College of American Pathologists. All rights reserved. 34

The expenses beyond ‘no errors’ can be defined as:

•! Prevention expenses are from proactive measures to improve quality and prevent errors throughout the laboratory.

•! Appraisal expenses are from inspection and maintenance.

•! Internal Failure expenses occur when something has gone wrong with the process.

•! External Failure expenses occur when results have left the laboratory and failed to meet the requirements of customers for whatever reason.

© 2015 College of American Pathologists. All rights reserved. 35

The expenses beyond ‘no errors’ can be redefined as:

•! Investment expenses are from proactive measures to improve quality and prevent errors throughout the laboratory.

•! Monitoring expenses are from inspection and maintenance.

•! Rework expenses occur when something has gone wrong with the process.

•! Downstream consequence expenses occur when results have left the laboratory and failed to meet the requirements of customers for whatever reason.

© 2015 College of American Pathologists. All rights reserved. 36

Proportions of quality costs

(or downstream consequences)

(or Rework)

(or Monitoring)

(or Investments)

Applying the model – quality costs associated with defective lab orders

•! Investment Costs: –!Set up tracking system

–!Make a case for retraining at clinics

–!Clinic RN managers meetings

–!Clinic providers meetings

•!Monitoring Costs: –! Lab Supervisor tracks defects

–!Analyze data

© 2015 College of American Pathologists. All rights reserved. 38

Calculation details

© 2015 College of American Pathologists. All rights reserved. 39

Applying the model – quality costs associated with defective lab orders (continued)

•!Rework Costs: –! Lab assistant time

–! Lab Med tech

–!Clinic Nurse

–!Clinic Physician

•!Downstream consequence costs: –! (intangible)

© 2015 College of American Pathologists. All rights reserved. 40

Calculation details (continued)

© 2015 College of American Pathologists. All rights reserved. 41

!! 42 minutes of rework per defect !! Cost of $20.34 per defect

Chart of relative annual quality costs

Potential downstream consequences

•!Lower satisfaction scores- patients not returning

•!Physician complaints

•!High risk to patient safety

© 2015 College of American Pathologists. All rights reserved.

Example: quantifying the financial impact of patient waiting

Patient Waiting

Patient Satisfaction

Patient Intent to Return

Anderson (2007) Zoller (2000)

Added 15 min drives 30-40% reduction in satisfaction

30% reduction in satisfaction results in 1% loss in patient returns/ year

Cost, improvement and timing

© 2015 College of American Pathologists. All rights reserved. 45

Change in Defect Management System

Initial Defect Management

System

Evidence of Order Defect – Not Precise

Defect Management System – Iteration 2

Investment Costs: 1.! Refine the system to

track this specific order defect

2.! Train internal lab people to capture the defects

3.! Build the case to train providers to stop making errors – lab collect vs. clinic collect

4.! Train providers to stop making errors

“Before” Costs: 1.! Tracking general

defects relating to lab orders

Added Monitoring Costs: 1.! Detailed

tracking of lab order defects

Cost, improvement and timing

© 2015 College of American Pathologists. All rights reserved. 47

Summary of annual quality costs

© 2015 College of American Pathologists. All rights reserved. 48

Investment of $3,175 results in a savings of $29,462

Project return on investment

© 2015 College of American Pathologists. All rights reserved. 49

Savings of $9.30 for every $1 spent

Another business case: recall our quality cost model

(or downstream consequences)

(or Rework)

(or Monitoring)

(or Investments)

Building a business case

Proactive Reactive

Costs 1. Estimate the cost of defects: incorrect blood draw orders, 300 times per month; assume that each defect costs the organization $100 to fix

Building a business case

Proactive Reactive

Costs 2. Extrapolate those costs to all similar defects in the chosen flow: with 5 clinics, and including other incorrect lab orders the count increases to 700 times per month

Building a business case

Proactive Reactive

Costs Rework cost calculation: 700 x $100

Rework: $70,000

Building a business case

Proactive Reactive

Costs 3. Estimate the cost to set up a defect management system: Labor investments to create a robust defect collection method total $10,000

Rework: $70,000

Investment: $10,000

Business case ROI

Proactive Reactive

Costs

$70000 / $10000 or a 7.0 : 1 return on investment

Rework: $70,000

Investment: $10,000

The value of cost of quality measures

“A properly understood and managed quality cost system will aid organizations in realizing costs savings while avoiding some of the serious pitfalls that can accompany cost cutting: decreases in product or service quality, increased customer dissatisfaction, added rework costs, or simple shifts in costs from one area to another.”

From Douglas C. Wood. The Executive Guide to Understanding and Implementing Quality Cost Programs. ASQ Quality Press, 2007

© 2015 College of American Pathologists. All rights reserved. 56

Key takeaways

•! Variation is inevitable !! having a ISO 15189 compliant deviation management system lets you detect, capture and reduce it in an effective data-driven manner.

•! Divide laboratory expenses into ‘running the lab without mistakes’ and ‘rework/ consequence expenses’, the latter is ‘waste’.

•! Use the information that you have on defects, rework costs and consequence costs to justify investments in knowledge or infrastructure.

© 2015 College of American Pathologists. All rights reserved.

References

•! Anderson, R., Camacho, F., & Balkrishnan, R. (2007). Willing to wait?: The influence of patient wait time on satisfaction with primary care. BMC Health Services Research, 7(31). doi:10.1186/1472-6963-7-31

•! Zoller, J., Lackland, D., & Silverstein, M. (2000). Predicting Patient Intent to Return from Satisfaction Scores. Journal of Ambulatory Care Management, 24(1), 44-50.

•! Gouveia, A. (Ed.). (n.d.). Browse Salaries by Job Category. Retrieved October 4, 2015, from http://swz.salary.com/SalaryWizard/LayoutScripts/Swzl_SelectCategory.aspx International Business Machines Corporation

© 2015 College of American Pathologists. All rights reserved.

Source: performancemanagementcompanyblog.com