ACR Education Department Presentationstatic.crowdwisdomhq.com/acr/contents/3960/3960_Quality_part1.pdf · medicine, we need to review some of the terms and basic definitions. The

1

What is it? Why should I care?

QUALITY

2009 Valerie P. Jackson Fellowship RecipientsJennifer Kohr, MDChristine Cinelli, MD

ACR Education Department Presentation

Presentation by the VPJ Fellowship recipients of 2009.

1

2

The faculty members listed below have indicated that they have no relevant financial relationships or conflicts of interest related to the presentation of their material.

Jennifer Kohr, MDChristine Cinelli, MD

Disclosure Statement:

.

2

3

Quality

• As a noun it is neutral and technically refers to“a degree or grade of excellence or worth”E.g. “The quality of students has risen.” or

“An executive of low quality.”

• As an adjective “being of high quality”E.g. “This is a quality presentation.”

Let’s start with the definition itself. What is Quality?It can be a nebulous term, but starting with the basic definition from Webster’s dictionary:

3

4

Quality

From a business perspective,

“The most important but perhaps the most amorphous value attribute for any business is

quality.”

Steele and Schomer, JACR 2009

From a business perspective quality is very important.[Steele JR and Schomer DF Continuous Quality Improvement Programs Provide New Opportunities to Drive Value Innovation Initiatives in Hospital-Based Radiology Practices. JACR 2009;6:491-499.]

4

5

ACR Definition of Quality

Bruce Hillman, MD, FACR, et al, in their article, “The Future Quality and Safety of Medical Imaging,”emphasized the ACR definition of quality. This definition was based on the IOM guidelines, but applied specifically to radiology.

5

6

Objectives• State the background of quality and its integration into medicine

• Differentiate QC, QA, CQI, PQI, TCM

• Explain how systems must take unavoidable human error into account

• Describe how quality is an integral part of lifelong learning

• List processes in radiology that are possible areas of quality improvement

• Explain the difference between PBLI and SBP and the importance of each

• Give specific examples of possible PQI projects

• Suggest intervention for problems and ways to re‐measure

• Develop several metrics to measure for improvement

• Communicate to others why quality matters

Above is the list of objectives for this presentation. For those persons wishing to obtain credit and/or a completion certificate for this activity, there will be an online quiz following this presentation.

6

7

Lesson Outline

I. History

II. Terms and definitions

III. Requirements

IV. Examples

The content of this presentation is as follows:1. HistoryWe’ll begin with an IOM report entitled, Quality Chasm, and then provide a brief history of Quality improvement beginning in Business Management.

8

II. Terms & Definitions

• Quality Control (QC)

• Quality Assurance (QA)

• Practice Quality Improvement (PQI)

• Continuous Quality Improvement (CQI)

• Total Quality Management (TQM)

• Definition of Quality of Care (by ACR)

2. We’ll define some of the important terms like Quality Control, Quality Assurance, Practice Quality Improvement, Continuous Quality Improvement, Total Quality Management, and review the ACR definition of quality of care.

9

III. Requirements

• ACGME (RRC) Requirements

• ABMS (MOC) Requirements

3. Next we’ll list the Accreditation Council for Graduate Medical Education(ACGME) Residency Review Committees (RRC) and the American Board of Radiology (ABR)

Maintenance of Certification (MOC) requirements.

10

• Quality Assurance

• Improvement Problems

• Continuous Quality Improvement

IV. Examples

4. And finally, we’ll provide examples of quality assurance/quality improvement problems and solutions to help you prepare your own quality improvement project.

11

What does this mean to you?

The requirements of the ABR are intended to implement a systematic approach forimproving the practice of radiology

Bottom-line: Requirements to document improved quality are here. The most recent additions to the competencies are directed at Practice Quality Improvement (PQI) and performing a project to improve your practice.These requirements to improve quality have the potential to…..

11

12

What does this mean to you?

• Make us better radiologists

• Improve efficiency of health care

• Improve “quality”

• Improve the bottom line

• Strengthen the field of radiology

We can chose to see this requirement as another annoying bit of bureaucracy or we see this requirement as an opportunity to make things better.Also, the survival (or at least thriving) of our specialty depends upon improving the quality of our practice.

12

13

Present: Hot topic

It is a hot topic! -The theme for RSNA 2009 was Quality Counts, and Gary Becker, MD highlighted its role in the future of radiology.-The current health care debate targets implementation of quality measures with the goal of improving care and decreasing costs. -It seems that quality can help us maintain strength in our field and help us proceed into the future as successful radiologists.

13

14

Quality

I. Historya. Background

Let’s start with a brief look at the background and the history of the concept of quality.

14

15

Past: Back to Business

• Born in business and manufacturing by engineers and statisticians in the 1920‐30’s

• W.A Shewhart, a statistical engineer at Bell Labs, focused on recurring manufacturing errors– Goal: reduce errors

The concept of quality was born in the 1920s and 30s in the business and manufacturing field by engineers and statisticians.

Shewhart, one of the most well known pioneers in quality in the US, worked at Bell Labs as a statistical engineer and his goal was to reduce errors by codifying incidence of specific recurring errors so as to prevent their recurrence.

15

16


W. Edwards Deming, a statistician, developed his expertise in “quality issues” during WWII while trying to improve efficiency or “kill ratio”of war material.

W. Edward Deming was a statistician who worked for the US Government during WWII. He worked to improve “kill ratio” (efficiency) of war materials.

16

17


• After WWII, American business regarded Deming’s quality management topics as unnecessary.

• At the same time, a colleague, J.M. Juran, added customer centric approach to quality.

• Their combined approach was embraced enthusiastically in post‐WWII Japan.

• Deming’s concepts were not widely recognized in the US until 1982!

Despite the work he produced in WWII, the idea of quality did not become popular in the US. Deming traveled to Japan, which was economically booming after WWII and his quality ideas were well received. He and J.M. Juran worked together to further the role of quality in business. This combined approach and the concept of quality were not widely recognized in the US business and manufacturing fields until 1982. Deming was recognized through his 1982 book “Out of the Crisis” (referring to the economic recession). The US felt significant competition from international competitors (especially Japan) and began to embrace Deming’s ideas to implement quality concepts in business and manufacturing.

17

18

Quality Moves into Medicine

• 1982: Deming’s quality concepts accepted in US Business

• 1991: JCAHO’s Principles and Organization and Management Effectiveness

• 2005: JCAHO initiative evolved into National Committee for quality Assurance –introducing “Pay for performance”

Quality concepts came to be a part of medical accreditation (driven by reimbursement) and implemented by JCAHO and later the ACGME, beginning in the 1990s, but it did not really take hold until the early 2000s.

18

19

Quality

II. Definitions

Before we transition into discussing the integration of quality and many of Deming’s concepts into medicine, we need to review some of the terms and basic definitions. The following slides will present many of the quality terms that were born in business and manufacturing.We will provide radiology examples for these definitions to make them clearer and more applicable.

19

20

Definitions

Quality Control, Quality Assurance, & Total Quality Management:

1. Quality Control: The regular, intermittent, technical testing of medical equipment and the evaluation of image quality. • Preventive maintenance of imaging

equipment• Evaluation of shielding around x‐ray facilities

From Deming: 3 basic concepts in quality evolved: Quality control, quality assurance, & total quality management.Let’s look at each separately and note the examples from radiology.

[QC Reference: Hynes, DM. Quality Management, The Canadian Association of Radiology Journal 1994: 45 (5): 353-4.]

20

21

Definitions


2. Quality Assurance: A quality management program to ensure excellence in health care through the systematic collection and evaluation of data.• Repeat rate, pathology correlation,

appropriateness of utilization, timeliness of scheduling

From Deming: the second basic concept in quality, quality assurance.[QA Reference: Erturk, SM et al. Quality management in radiology: Historical aspects and basic defintions, JACR 2005;2:985-991.

21

22

Definitions


3. Total Quality Management: Overarching institutional culture of safety and quality.

‐This term is not utilized often in the health care field, It is more a philosophical term that Deming used.CQI (next page) is more often used in health care to describe the same concept.

From Deming: the third basic concept in quality, total quality management . [TQM Reference: Steele and Schomer, Continuous Quality Improvement Programs Provide New Opportunities to Drive Value Innovation Initiatives in Hospital-Based Radiology Practices. JACR 2009;6:491-499.]

22

23

4. Continuous Quality Improvement:

Combine medical knowledge with attention to evaluating and improving the whole system. In radiology, the system refers to every step, from the ordering of a radiology exam/ procedure through the delivery of final report.

• Contrast reaction resulting in anaphylaxis: First you would submit a QA report, which tracks and reports event; this alone may not always improve a process. Therefore, approach utilizing CQI, which would evaluate the whole system for ways to prevent further contrast reactions.

Two other quality concepts that have evolved since Deming: Continuous Quality Improvement and Practice Quality Improvement[CQI Reference: Hynes, DM. Quality Management, The Canadian Association of Radiology Journal 1994: 45 (5): 353-4.Six Sigma (a CQI program): Seshadri, S. Six sigma gains momentum in health care, Radiol Manage 2002;(24)2:16-8.

23

24

Definitions

5. Practice Quality Improvement (PQI)

A tool to implement CQI, making its application more tangible.

(Examples to follow later)

We’ll be describing PQI and providing examples of projects later in this presentation.

24

25

Quality

Background: Errors and Systems

Now that we have introduced the history of quality and listed basic quality terms, we want to take a step back and look at the big picture again, this time concentrating on the psychology of error and systems based approach to managing error.

25

26

Error

Isn’t good, good enough? • 99.9% accuracy is equal to:

‐ 2 airplane crashes/year at O’Hare

‐ 500 incorrect surgeries/week

‐ 20,000 incorrect prescriptions/year• Good is not good enough!

How do you know when good is good enough? Isn’t 99.9% really good? You would be really happy if you scored 99.9% on your boards, or if you only missed 0.1% of the findings you see annually, wouldn’t you? But what does it really mean to have 99.9% accuracy? We are not even at 99.9% in medicine though…good is not good enough. Note: Robert Wachter, MD in a book describing systems based approach to minimizing medical errors, was written essentially in response to the IOM report.

26

27

Today’s Medicine: NOT Good Enough

• The world health report 2000: US health system spends a higher portion of its gross domestic product than any other country but ranks 37 out of 191 countries according to its performance

• Since 2000, costs have continued to increase without clear improvement

Bottom line: health care has a lot of room for improvement.[World Health Report, 2000]

27

28

– Estimated that in 1997 between 44,000 to 98,000hospitalized patients died annually due to preventable medical errors

(Medical errors still cause more deaths than motor vehicle accidents, breast cancer or AIDS.)

1999, IOM publishes To Err is Human

What really brought home the point to both medicine and the public was the IOM…Which stated that

28

29

This estimate would be equivalent to a jumbo jet crashing every day with all the passengers dying!

‐ Even if the IOM report numbers are overestimated by 100‐200%, this estimate would still be equivalent to one jumbo jet crashing every four days…which isn’t great either

This probably would have made page three on the daily paper except for one ingenious spin - the comparison to jumbo jet crashes.You can imagine how the FAA and the public would respond!The IOM report ignited the public interest in safety and quality in healthcare.[1999, IOM publishes To Err is Human]

29

30

FAA: Study in Safety Improvement

• 1960s‐70s: Risk of dying in an aviation accident 1 in 2 million

• 1990: Risk of dying in an aviation accident ‐ 1 in 8 million

So basically, medicine was in the spotlight for errors after the IOM report and once the comparison was made to aviation safety. Let’s take a closer look at the FAA.Research was performed by the FAA Office of System Safety and showed the risk of dying in an aviation accident.[FAA Office of System Safety]

30

31

By Comparison,

• Adverse events in American hospitals occur at a rate of 3‐4% (3‐4/100)

• Yes, medicine is different than flying…it is complex, dynamic, and intermittently very intense

….sort of like landing on an aircraft carrier

By comparison, adverse events in American hospitals occur at a rate of 3-4/100.But you might say that these two industries are not truly comparable.[Source: Brennan et. al. NEJM 1991;324 370-6]

31

32

What can we learn from this aircraft carrier?

This is an aircraft carrier - pilots must land on this short runway, immediately under the runway are nuclear weapons, and enemies are trying to shoot them down. Like medicine, this situation is complex, internally dynamic and intermittently, intensely interactive. [BMJ Reason 2000]

32

33

Error and Systems

• Human error is unavoidable.• Pilots don’t just try really hard not to make mistakes.

• However this concept has historically been the culture in medicine.– work harder with the expectation that practice makes perfect

• Systems or processes that depend on perfect human performance are inherently flawed.

On a positive note… let’s use error as opportunity…

33

34

How to Reduce ErrorThe FAA’s and military’s methods to reduce errors:

• Ongoing systematic study of errors yielding better, more error resistant systems– For example a “root cause analysis” where after an adverse event, the event is broken down into multiple steps and evaluated for all the possible contributing factors to the error.

– If we instead blame the individual, we cannot find the ways the environment or system contributes to the error.

• In medicine, CQI and PQI are a framework for studying our errors and improving systems.

The FAA and the military have developed extensive systems that successfully reduced error: How?Kelvin (inventor of the heat scale) said “You cannot improve things that you cannot measure.”

34

35

Adverse eventAdverse event

Individual punished and system unchanged


Human error identified


Identify the Cause of Errors

What happens when we just blame the person who is most proximal to the error? This is the historical way of approaching error; where we look for someone to blame.

35

36





Teaching opportunity and system improved

Teaching opportunity and system improved



Whole system evaluated:

system errors identified

Whole system evaluated:

system errors identified

Identify the Cause of Errors

But, when we evaluate the system after an error occurs by conducting a root cause analysis, as part of Continuous Quality Improvement, there is a more comprehensive solution. The latter example is more complete and should help prevent future similar errors from occurring. We’ll apply some of these techniques later when we describe some medical errors and show how to identify the root causes.

36

37

Quality

III. Requirements

• ACGME (RRC) Requirements

• ABMS (MOC) Requirements

So what does all this mean to you?Let’s see how what we have covered so far applies to you and the requirements for residents and/or practicing radiologists.

37

38

ACGME

• ACGME recognizes the importance of integrating quality improvement into resident training

Let’s look at an overview of the ACGME requirements first.

38

39

This graphic shows all the program requirements.For more detail refer to:ACGME (RRC Diagnostic radiology requirements PDF, 7/2008)

40

Our focus here is on Part IV, the Educational program.

41

Included in the curriculum are these six competencies.

ACGME (RRC Diagnostic radiology requirements PDF, 7/2008)IV. Educational Program A. Curriculum components 5. ACGME Competencies

42

In this presentation is our focus will be on practice based learning and improvement (PBLI) and systems-based practice (SBP).ACGME (RRC Diagnostic radiology requirements PDF, 7/2008)IV. Educational Program A. Curriculum components 5. ACGME Competencies c. Practice-based Learning and Improvement Residents must demonstrate the ability to investigate and evaluate their care of patients to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and life-long learning. Develop skills and habits to meet the following goals: 1. Identify strengths, deficiencies and limitations in knowledge and expertise2. Set learning and improvement goals3. Identify and perform appropriate learning activities4. Systematically analyze practice using quality improvement methods and implement changes with the goal of practice improvement5. Incorporate formative evaluation feedback into daily practice6. Locate, appraise, and assimilate evidence from scientific studies related to their patients’ health problems7. Use information technology to optimize learning8. Participate in education of patients, families, students, residents and other health professionals

43

ACGME (RRC Diagnostic radiology requirements PDF, 7/2008)IV.A.5.f) Systems-based Practice Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care, as well as the ability to call effectively on other resources in the system to provide optimal health care. Residents are expected to: Work effectively in various health care delivery settings and systems relevant to their clinical specialty; Coordinate patient care within the health care system relevant to their clinical specialty; Incorporate considerations of cost awareness and risk-benefit analysis in patient and/or population-based care as appropriate; Advocate for quality patient care and optimal patient care systems;Work in interprofessional teams to enhance patient safety and improve patient care qualityParticipate in identifying system errors and implementing potential systems solutionsDemonstrate an understanding of how the components of the local and national healthcare system function interdependently, and how changes to improve the system involve group and individual efforts. The residents must function as consultants for other health care professionals, and act as a resource for information regarding the most appropriate use of imaging resources, and Be trained to identify existing systems problems that compromise their ability to provide the most efficient and effective patient care. They must be trained to analyze systematically the problems, develop solutions, implement solutions, and evaluate the effectiveness of the intervention. Systems-based problems can be identified at the departmental, institutional, local, or national level. Evidence of participation in identifying system errors and implementing potential systems solutions must be documented in the resident’s learning portfolio.

44

ABR Maintenance of Certification


Six Essential CompetenciesSix Essential Competencies

The six core competencies of the ACGME are identical to the American Board of Radiology, maintenance of certification (MOC).Next we’ll review the requirements for practicing radiologists.

45

ABR

• ABR recognizes the importance of integrating quality improvement into continuing training

ABMS (MOC) Requirements

The ABR requirements as of December, 2009.

45

46



The ABR has added Maintenance of Certification to their requirements for practicing radiologists. Within MOC there are six essential competencies and 4 components to evaluate the competencies. Graphically represented here are the six competencies.

46

47



Six Essential CompetenciesSix Essential Competencies

The Competencies that we are going to focus on in this presentation are practice based learning and improvement (PBLI) and systems-based practice (SBP).

47

48

Of the four components used to evaluate the competencies, we are going to focus on PQI.

48

49

Most radiologists are required to do a PQI project. This chart represents the PQI timeline and milestone tracking for diagnostic radiology diplomates to be completed in a 10 year MOC cycle.[References: Strife white paper on MOC and PQI & Pina Sanelli, ABR video on PQI]

49

50

PQI: Example

• You are working at a new hospital and you notice that the neck CTAs are almost always difficult to interpret.

• You have identified a problem (and a possible PQI project).

• What is the next step?

• Let’s run this PQI example through the 10 year MOC timeline.

Here is an example of a PQI project. We will use this example to walk you through the steps of completing the 10 year MOC cycle.

50

51

Here we use neck CTA an example to demonstrate the PQI timeline and milestone tracking for diagnostic radiology diplomates (to be completed in 10 year MOC cycle).

51

52

Example continued to demonstrate the PQI timeline and milestone tracking for diagnostic radiology diplomates (to be completed in 10 year MOC cycle).

52

53

Categories of PQI projects

• 5 categories of PQI

1. Patient safety

2. Accuracy of interpretation

3. Report timeliness

4. Practice guidelines and technical standards

5. Referring physician surveys

http://theabr.org/moc/moc_dr/moc_dr_comp4.html

According to the ABR, the goals of PQI are to improve the quality of care delivered, reduce errors and improve safety. The ABR has outlined five categories of PQI[Source- http://theabr.org/moc/moc_dr/moc_dr_comp4.html]The ABR site also has excellent descriptions of how to document the PQI project.

53

54

• Clinical practice guidelines and technical

• Report turn around time

• Interpretation accuracy

• Service satisfaction

Some Examples of PQI projects

To get you thinking about sources for PQI projects, listed are some examples of categories of PQI projects.

[Source: http://theabr.org/moc/moc_dr/moc_dr_pqiguides.html]

55

Clinical Practice Guidelines & Technical Standards

• Assessment of how well a practice adheres to clinical practice guidelines and technical standards, for example those developed by the American College of Radiology

(ACR)


Let’s look at these one at a time, and add some specifics.

56

Report Turn Around Time• Assessment of report turn around time or how long it

takes from the time an image is captured to when you give a report to the referring physician.


Some of these metrics are included in ACR National Data Registry Databases (NRDR). For more detailed information on ACR’s approved PQI programs seehttp://acr.org/nrdr.aspx

57

Interpretation Accuracy• Assessment of how accurately radiologists interpret

images through a process of double reading, in which two radiologists read the same image and compare the results. For example using the RADPEERTM program developed by the ACR.


Some ACR programs capture performance data as the electronic reports are filled out. The National Mammography Database utilizes direct transfer of the necessary information that can be used by participating radiologists as national baseline data and for developing performance improvement projects.

58

Service satisfaction• Survey referring physicians or patients regarding

their satisfaction with your service


.

59

MOC/PQI ResourcesPQI Reading Resources: http://theabr.org/moc/moc_dr/moc_dr_pqiread.html

• American Roentgen Ray Society (ARRS) ‐ PQI Connect• ABR articles on PQI for diagnostic radiologists• JCAHO's National Patient Safety Goals• National Quality Forum's 30 Safe Practices for Better Healthcare• National Council on Radiation Protection and Measurements• Institute of Medicine Reports (IOM)

To Err Is Human: Building a Safer Health SystemCrossing the Quality Chasm: A New Health System for the 21st Century

• Institute for Healthcare Improvement (IHI)Improvement Methods

• RSNA's Continuous Quality Improvement Initiative• Agency for Healthcare Research and Quality's "Effective Healthcare"• Mayo Clinic, Department of Radiology, Quality Improvement Reading List• Society for Interventional Radiology ‐ Quality Improvement Articles• Radiology Physician Performance Measurement Set

from the American College of Radiology/Physician Consortium for Performance Improvement/ National Committee for Quality Assurance

Guidelines for PQI projects, with some examples at the ABR• http://theabr.org/moc/moc_dr/moc_dr_pqiguides.html

Please note that this text is not actually hyperlinked! Go to ABR website for the links.

59

Documents

ACR Education Department Presentationstatic.crowdwisdomhq.com/acr/contents/3960/3960_Quality_part1.pdf · medicine, we need to review some of the terms and basic definitions. The