2013 APDS SPRING MEETING

Jump Starting a Quality and PerformanceImprovement Initiative to Meet the UpdatedACGME Guidelines

Nessa Miller, MD,* Heather MacNew, MD,* Jane Nester, DrPH,† Jean B. Wiggins,*Cynthia Shealy, BSN,* and Christopher Senkowski, MD*

*Department of Surgery, Mercer University School of Medicine, Savannah Campus, Memorial UniversityMedical Center, Savannah, Georgia; and †Department of Family Medicine, UNC Chapel Hill School ofMedicine, Cone Health, Greensboro, North Carolina

BACKGROUND: The Accreditation Council for GraduateMedical Education requires residents to be trained inpractice-based learning and improvement as well assystems-based practice. In an effort to establish a formalcurriculum for graduate medical education, a PerformanceImprovement (PI) Training Program was initiated atMemorial University Medical Center. Training for the chiefresidents across all residency programs focused on the basicSix Sigma framework. Chief residents chose faculty sponsorsand were also mentored by Six Sigma–trained staff. Facultyand physicians who participated in the initiative received PI/Continuing Medical Education credit.

METHODS: A total of 17 presurveys and postsurveys werecompleted on 7 outcome measures. Nonparametric Wil-coxon signed rank 2-tailed tests were performed to test forsignificant change from presurvey to postsurvey.

RESULTS: Analysis of the 2-year data (2009-2011) foundstatistically significant improvement for all 7 outcomemeasures. The surgical residents’ PI Project for 2011included the development of the Venous Thromboembo-lism Reassessment Tool. The project included a multi-disciplinary team to develop a computer prompt thatcontinued to trigger if the physician launched the prophy-laxis or treatment form without ordering anticoagulation.The new prompt resulted in a 391% increase in anti-coagulant orders.

CONCLUSIONS: This study demonstrated that theresident-based PI Training Program was innovative, prac-tical, and comprehensive. Education, tools, and skill devel-opment were provided on quality and PI theory and practicefor resident physicians in support of the Accreditation

Correspondence: Inquiries to Jean B. Wiggins, Memorial University Medical Center,Medical Education, 4700 Waters Avenue, Savannah, GA 31404; e-mail:wiggije1@memorialhealth.com

758 Journal of Surgical Education � & 2013 Association of ProElsevier Inc. All rights reserv

Council for Graduate Medical Education core competenciesof professionalism, practice-based learning and impro-vement, and systems-based practice. ( J Surg 70:758-768.JC 2013 Association of Program Directors in Surgery.Published by Elsevier Inc. All rights reserved.)

KEY WORDS: surgery resident, PI/CME credit, perform-ance improvement, Six Sigma, leadership, quality

COMPETENCIES: Professionalism, Practice-Based Learn-ing and Improvement, Systems-Based Practice

INTRODUCTION

Performance improvement is important for any organiza-tion to provide a structure for examining problems andsetting up methods to ensure quality outcomes aredelivered. The Accreditation Council for Graduate Med-ical Education (ACGME) requires residents to be trainedin practice-based learning and improvement as well assystems-based learning.1 Traditionally, surgical residencyprograms teach these core competencies through observa-tion, lectures, and mentorship. In today’s quality-drivenhealthcare environment coupled with practicing physi-cians’ need for Continuing Medical Education (CME)credits, there was an opportunity for development of acurriculum that teaches Performance Improvement (PI) toresidents. This curriculum also created a bottom-upmentoring strategy for introducing PI/CME concepts toour medical staff and faculty. We chose to use the SixSigma theory that originated in industry, beginning withchief residents in each residency program.2 This required ateam effort from the Quality and Patient Safety (QPS),Graduate Medical Education (GME), and CME Depart-ments for the development and strategic delivery of the PI

gram Directors in Surgery. Published byed.

1931-7204/$30.00http://dx.doi.org/10.1016/j.jsurg.2013.06.016

training and follow through. Goals were to have theresidents improve the practice of using teams and peersin the area of evidence-based medicine surroundingQPS. Chief resident physicians applied PI and leadershipskills by conducting a PI initiative within their department(6 ACGME accredited programs) to enhance patientoutcomes, healthcare delivery, and the work environment.PI initiatives also provided an opportunity for FacultySponsors and attending physicians to learn PI in a non-threatening way by participating in a resident-drivenPI initiative and earn PI/CME credits toward theirAmerican Medical Association Maintenance of Licensureand American Board of Medical Specialties Maintenanceof Certification.3

TABLE 1. Pretest

Journal of Surgical Education � Volume 70/Number 6 � November

METHODS

Design

The curriculum development allowed for a concise deliverythat took place over a 4-hour time span (Table 1). Chiefresidents’ training included an abbreviated explanation ofthe Six Sigma improvement process. The accepted institu-tional standard for PI provided the basis for teaching theSix Sigma quality model. Six Sigma is a methodology thatoccurs through measurement-based strategies. The SixSigma theory and process of DMAIC (Define, Measure,Analyze, Improve, and Control) and the tools advocatedby that program were taught to all chief residents from

/December 2013 759

TABLE 2. Project Deadlines and Timeline

6 residency programs.4 The 4-hour training supplied toolsfor managing the PI project as well as presentations ofseveral examples of fully completed PI projects to demon-strate the Six Sigma process. The didactic portion of thetraining session emphasized interpersonal communication,conflict resolution, leadership style, team development,and delegation within a team. In addition, a timeline gavethe group direction, which included a project charterdeadline, how to engage and work with the mentor andfaculty sponsor, and culmination date for the institutionalgrand rounds (Table 2). The project charters delineatedthe road map and expectations of the project (Table 3).Project charters included 5 components: problem state-ment, goal statement, scope of the project, deliverables,and members of the team. Development of initial projectcharters occurred by the mentor providing guidance to theresident chief to clarify and define their project. Impor-tantly, a rough draft of a project charter was developed anddiscussed at this initial training session. A presentation ofthe initial project charter by the resident chief from eachprogram allowed the group at large to provide feedbackand encouragement. Over the next 6 months, organizedmeetings occurred between the resident chiefs, the mentor,and team members as to the needs of the project and theresident’s schedule. Pretests were given to all chief (final

760 Journal of Surgical E

year) residents before the 4-hour PI core training sessionled by the QPS Department mentors (Table 4). A posttestwas conducted a week after the PI grand rounds programto determine the resident’s level of learning, understand-ing, and expertise in the use of PI processes (Table 5). Thisevaluation on the training process was used for buildingrecommendations for the next academic year PI initiative.One month after the 4-hour training with the chiefresidents, a mandatory 1-hour didactic introductory coursewas given to all the junior resident physicians. This coursenot only introduced the idea of Six Sigma theory to therest of the residents, but also unveiled the project chartersthat had recently been developed by the chief residentleadership teams.

Participants

The GME Department strongly supported the PI projectinitiative by recruiting each residency department. All6 residency programs incorporated the PI education intotheir curriculum and training programs as an institution-wide initiative. The 6 program chairs, program directors,and Memorial University Medical Center (MUMC)-des-ignated institutional official supported and mandated theinitiative across all residencies. As a result, the program

ducation � Volume 70/Number 6 � November/December 2013

TABLE 3. Project Road Map

directors appointed the chief residents to engage allresidents within their departments to participate in the PIprocess. However, owing to clinical responsibilities, not allresidents were able to participate actively. For the chiefresidents, only the 4-hour training and the capstonepresentation of the project results at grand rounds becameprotected times. This process allowed the residents toacquire one aspect of professionalism and therefore, theresidency programs learned to incorporate the demands ofthe PI projects into their workday. Six Sigma–trainedmentors were identified amongst the staff of QPS, GME,CME, and hospital leadership. These PI mentors helped toshepherd each residency program through their project.Each chief resident identified a faculty sponsor within his/her residency to assist in championing the project whilelearning the PI process. The faculty sponsor worked withthe team and learned the PI process as well as earned PI/CME credit. The teams were given 6 months to complete aPI project initiative. The overall PI project culminated withan institution-wide grand rounds approximately 6 monthsafter the annual initial training. Program chiefs presented

Journal of Surgical Education � Volume 70/Number 6 � November

their projects in a judged competition at the institution-wide grand rounds with CME credit provided for partic-ipation. The chief resident leaders completed posttests1 week after the PI grand rounds. The PI projects werejudged by leaders from the institution including thepresident/CEO, chief nursing officer, chief informationofficer, and other Six Sigma Black Belt team members.The capstone presentations at grand rounds were moder-ated by leadership from QPS and GME with participationby the chief medical officer. The judges’ scorecard isincluded as (Table 6).

Surgery Resident PerformanceImprovement Project

The surgical residents’ PI project included the developmentof the Venous Thromboembolism (VTE) ReassessmentTool. Previously, on admission, assessment occurred on allpatients using a paper VTE Risk Assessment tool. This toolmet compliance requirements for the National QualityForum Safe Practice #28 VTE Prevention and is endorsed

/December 2013 761

TABLE 4. Pretest

by Blue Cross/Blue Shield Quality Hospital Incentive Pro-gram, Leapfrog, and is considered a best practice.5 However,with the implementation of electronic medical records, acomputer-based procedure and process were necessary toprompt the physician to reassess VTE requirements as thepatient status changed. The project required a multidiscipli-nary team to develop a computer prompt that searched eachchart to identify the patients that were not on anticoagula-tion. When a target patient was identified, a prompt wouldfire each time the physician logged on to place electronicorders. It would continue to trigger if the physician launchedthe prophylaxis or treatment form without ordering anti-coagulation. The prompt had built-in opt-out reasons, justlike the paper form, when anticoagulation was

762 Journal of Surgical E

contraindicated. Additionally, it would stop triggering if thepatient was on appropriate anticoagulation.

RESULTS

The chief resident training pretests and posttests wereevaluated for all the chief residents. During the pilot year,9 presurveys and postsurveys were completed for the 7 out-come measures (Tables 4 and 5). Given the small samplesize, nonparametric Wilcoxon signed rank tests were per-formed to test for significant change from pretest to posttestfor the 7 outcomes. This analysis found statistically signifi-cant improvement for 6 of 7 outcomes in the pilot year.

ducation � Volume 70/Number 6 � November/December 2013

TABLE 5. Posttest

Journal of Surgical Education � Volume 70/Number 6 � November/December 2013 763

TABLE 6. Judges' PI Project Scorecard

During year two, 17 pretest and posttest (includingthe 9 from the pilot year) were analyzed in the samemanner from the chief resident training and found 7 of7 outcomes with statistically significant improvement(Table 7).

764 Journal of Surgical E

The judging score cards from the capstone PI grandrounds also supported the improvement in quality of theprojects from the pilot year to year two (Tables 8 and 9).Specific to the surgery project, the new prompt for VTEprophylaxis resulted in a 391% increase in appropriate

ducation � Volume 70/Number 6 � November/December 2013

TABLE 7. Pretest and Posttest Combined Analysis of Results Pilot Year and Year Two

anticoagulant orders. Success of the project led tothe MUMC governance committee utilizing the computer-ized VTE reassessment prompts throughout the institution.

CME Credit

The institutional CME committee for accreditationreviewed all applications for PI/CME credit from thefaculty. PI/CME credit was awarded once approved bythe committee. During the pilot year, 15 faculty membersreceived 105 PI/CME credits. During year two, 26 facultymembers received 460 PI/CME credits.

DISCUSSION

The overarching goal for the development of the PIinitiative was to create a curriculum for resident trainingin practice- and systems-based learning. Additionally, there

Journal of Surgical Education � Volume 70/Number 6 � November

were several unrecognized benefits to the PI curriculum.MUMC benefited from bottom-up training to introduce PIto faculty and medical staff using residents to teach-the-teacher economically. Additionally, real-time worthwhileprojects were completed in a timely fashion using a work-force already in place. The short time duration of projectsforced the scope to be workable and maximized results forinvestment. Future projects would strengthen now that thefoundation of PI has “buy in from the hospital commun-ity.” Several house-wide teamwork benefits were alsorealized. For example, the competition between residencyprograms was beneficial for motivation and cross-programcommunication. PI projects promoted a collaborative,interprofessional, team-based environment of continuingeducation with the resident physician as leader of multipledepartment members. Certainly, there remain challenges toinstitution-wide adoption of PI projects. Some departmentsstill question the benefit to their residents in the limitedwork hour reality. Although the short time limit can be

/December 2013 765

TABLE 8. Overall Resident PI Project Score Summary (Pilot Year and Year 2)

helpful to scoping a project, it can also limit the sizeand applicability of a larger project. It is especially impor-tant to conduct handoff and sustainability meetings atthe conclusion of projects for the improvements to finda management home within the institution. Efforts are

766 Journal of Surgical E

ongoing to examine the long-term sustainability ofthese important projects. The importance of integra-tion and strategic planning between GME, CME, andQPS for enhanced institutional performance cannot beoveremphasized.

ducation � Volume 70/Number 6 � November/December 2013

TABLE 9. Overall Resident PI Project Score Summary (Pilot Year and Year 2)

CONCLUSIONS

This study demonstrated that our resident-based PI trainingprogram was innovative, practical, and comprehensive. Wedeveloped a workable curriculum for education, providedtraining and tools for skill development on PI theory andpractice to resident physicians in support of the ACGMEcore competencies of professionalism, practice-based learn-ing, and systems-based practice. The pretest and posttestresults demonstrated that residents are engaging in real-timePI projects while building leadership and professionalism.Hospitals already have the personnel and the projectstructure in place for successful and important resident-ledPI projects.

Journal of Surgical Education � Volume 70/Number 6 � November

REFERENCES

1. Accreditation Council for Graduate Medical Education.Common Program Requirements. IV.A.5.c p. 8. Retrievedfrom ⟨http://acgme.org/acgmeweb/Portals/0/dh_dutyhoursCommonPR07012007.pdf⟩; 2011 Accessed 01.07.11.

2. Liker Jeffrey. The Toyota Way: 14 Management Prin-ciples from the World’s Greatest Manufacturer. NewYork: McGraw-Hill; 2004.

3. Bailey S. Report of the Council on Medical Education.CME report 3-A-10. Retrieved from ⟨http://www.ama-assn.org/resources/doc/council-on-med-ed/cme-rep3-a10.pdf⟩; 2010 Accessed 20.07.10.

/December 2013 767

4. De Koning H, Verver J, van den Heuvel J, Bisgaard S,Does R. Lean Six-Sigma in healthcare. J Healthc Qual.2006;28(2):4-11 [PubMed PMID: 16749293].

5. National Quality Forum (NQF). Safe Practices forBetter Healthcare-2009 Update: A Consensus Report.

768 Journal of Surgical E

Washington, DC: NQF; 2009 http://www.qualityforum.org/Publications/2009/03/Safe_Practices_for_Better_Healthcare%e2%80%932009_Update.aspx.

ducation � Volume 70/Number 6 � November/December 2013