Upload
mohamed-chancey
View
233
Download
3
Tags:
Embed Size (px)
Citation preview
Department of Graduate Medical Education (GME)
Conflict of Interest
Kim Walker – No conflicts of interest to disclose Ann Dohn – No conflicts of interest to disclose Nancy Piro – No conflicts of interest to disclose
Kim Walker, PhD
Program Manager/Education Specialist
Ann Dohn, MA
DIO & GME Director
Nancy Piro, PhD
Program Manager/Education Specialist
PC002c Coordinators and Clinical Competency
Committees: How to Streamline and Support the Work of your Program’s CCC
Department of Graduate Medical Education (GME)
Session Outcomes
Participants will be able to:
Identify new aspects of the coordinators’ evolving role in program administration.
Understand and use program requirements as a guide for planning, organizing and implementing educational and assessment tools.
Develop and utilize a newly developed comprehensive resident performance profile tool to streamline the work of the CCCs.
Department of Graduate Medical Education (GME)
New Role
Department of Graduate Medical Education (GME)
Evolving Role for Coordinators in Evaluations
• Deliver evaluations• Develop evaluation forms for PDs
to approve• Schedule semi annual evaluations• Ensure summative evaluations
completed and filed
• Constructing new milestone evaluations to pilot/deliver
• Reviewing evaluation completion data for accuracy
• Aggregating data for the CCC from multiple sources and forms
• Milestone data to ACGME
Administrator Scheduler Supreme
Education & Evaluation
Coordinator/ManagerExtraordinaire
Department of Graduate Medical Education (GME)
Now I’m really confused!
CCCsEPAs
Milestones
Outcomes
Goals and Objectives
Evaluations
Department of Graduate Medical Education (GME)
Where do I begin?
Department of Graduate Medical Education (GME)
Know the NAS Building Blocks: Concepts defined
NAS Core Competencies
– Milestones– EPAs– Curriculum and Evaluations
Clinical Competency Committee (CCC)
Department of Graduate Medical Education (GME)
NAS – Next Accreditation System
What is NAS – in a nutshell:“an outcomes-based accreditation process through which the doctors of tomorrow will be measured for their competency in performing the essential tasks necessary for clinical practice in the 21st century.”
http://www.acgme.org/acgmeweb/tabid/435/ProgramandInstitutionalAccreditation/NextAccreditationSystem.aspx
Department of Graduate Medical Education (GME)
Major Changes:Accreditation based on…
Pre-NAS– Competencies– Site Visits – Up to 5+
year cycles– Internal Reviews– ADS Updates– PIFs– Resident Surveys
Current (New) NAS– Competencies with
Milestones– Self-Studies at ~ 8-10 year
intervals– Detailed ADS Updates– CLER Visits ~ 18 – 24
months (Institution)– Resident & Faculty
Surveys
Department of Graduate Medical Education (GME)
The New Accreditation System (NAS)… Outcomes
Increased Annual reporting by Programs (online) Reduced volume of accreditation demands … but
increased attention to accuracy and completeness of information submitted online
PIF-less Surveyor visits (unless new application)– Two Field Surveyors per visit
No Faculty CVs (only PD)….but Faculty & Resident Scholarly Activity required.
Department of Graduate Medical Education (GME)
The Six AGME Core Competencies
MedicalKnowledge
Six CoreCompetencies
For QualityPatient Care
Practice-based
Learning & Improvement
Professionalism
Systems-based Practice
PatientCare Interpersonal &
Communication Skills
Department of Graduate Medical Education (GME)
What Are Milestones?
High Level - Milestones are simply defined as areas of competency/expectations for our trainees
Linked to six core competencies Defined as a continuum of progressive
growth/learning
Department of Graduate Medical Education (GME)
Dreyfus Model (1980): Stages of developing expertise
1 Novice
* Rigid adherence to taught rules or plans* Little situational perception* No discretionary judgment
2 Advanced Beginner
* Guidelines for action based on attributes or aspects* Situational perception still limited* All attributes and aspects are treated appropriately and given equal importance
3 Competent
* Coping with 'crowdedness'* Now sees actions at least partly in terms of longer-term goals* Conscious deliberate planning* Standardized and routinized procedures
4 Proficient
* Sees situations holistically rather than in terms of aspects* See what is most important in a situation* Perceives deviations from the normal patterns* Decision-making less labored* Uses maxims for guidance, whose meaning varies according to the situation
5 Expert
* No longer relies on rules, guidelines or maxims* Intuitive grasp of situations based on deep tacit understanding* Analytic approaches used only in novel situations or when problems occur* Vision of what is possible
Source: Eraut, M. Developing Professional Knowledge and Competencies. (1994)
Department of Graduate Medical Education (GME)
Milestone Level Definitions
Level 1: The resident is a graduating medical student/experiencing first day of residency.
Level 2: The resident is advancing and demonstrating additional milestones.
Level 3: The resident continues to advance and demonstrate additional milestones; the resident consistently demonstrates the majority of milestones targeted for residency.
Department of Graduate Medical Education (GME)
Milestone Level Definitions (continued)
Level 4: The resident has advanced so that he or she now substantially demonstrates the milestones targeted for residency. This level is designed as the graduation target – not requirement.
Level 5: The resident has advanced beyond performance targets set for residency and is demonstrating “aspirational” goals which might describe the performance of someone who has been in practice for several years. It is expected that only a few exceptional residents will reach this level.
Department of Graduate Medical Education (GME)
Reporting the Milestones
V.A.1.b).(1).(b) prepare and assure the reporting of Milestones evaluations of each resident semi-annually to ACGME (Core)
Milestones are reported directly through ADS Reporting windows are:
– November 1- December 31– May 1- June 15
Department of Graduate Medical Education (GME)
Entrustable Professional Activities (EPA)
Professional life activities that define a medical specialty: – Ground the competencies in a physician’s everyday
work – Activities lead to some outcome that can be observed – Complexity of the activities requires an integration of
knowledge, skills and attitudes across competency domains
Department of Graduate Medical Education (GME)
Examples of EPAs
Facilitate handovers to another healthcare provider either within or across settings
Contribute to the scholarly work of the subspecialty
Co-manage patients with generalists and other subspecialists
Source: https://www.abp.org/abpwebsite/taskforce/reslib/24.ppt
Department of Graduate Medical Education (GME)
Curriculum: Rotation-specific goals and objectives & links to milestones
Department of Graduate Medical Education (GME)
Milestones Impact on Evaluations:Linking questions to milestones
Step Two: Ensure specific evaluation questions are linked to milestones
Advises the referring health care provider(s) about the appropriateness of a procedure in routine clinical situations
Department of Graduate Medical Education (GME)
Milestones Impact on Evaluation System
Allows for more objective methods of assessment and provide better feedback
Provides a process for early identification of residents that are having difficulties
All old and new evaluations and questions should be aligned with and tracked to milestones
Department of Graduate Medical Education (GME)
Clinical Competency Committee (CCC)
V.A.1. The program director must appoint the Clinical Competency Committee.(Core)
V.A.1.a) At a minimum the Clinical Competency Committee must be composed of three members of the program faculty.(Core)
V.A.1.a).(1) Others eligible for appointment to the committee include faculty from other programs and
non- physician members of the health care team.(Detail)
ACGME Common Program RequirementsApproved: February 7, 2012; Effective: July 1, 2013 Approved focused revision: June 9, 2013; Effective: July 1, 2013
Department of Graduate Medical Education (GME)
V.A.1.b).(1) The Clinical Competency Committee should:
V.A.1.b).(1).(a) review all resident evaluations semi-annually; (Core)
V.A.1.b).(1).(b) prepare and assure the reporting of Milestones evaluations of each resident semi-annually to ACGME; and, (Core)
V.A.1.b).(1).(c) advise the program director regarding resident progress, including promotion, remediation, and dismissal.(Detail)
Clinical Competency Committee (CCC)
Department of Graduate Medical Education (GME)
V.A.1.b) There must be a written description of the responsibilities of the Clinical Competency Committee.(Core)
ACGME Common Program RequirementsApproved: February 7, 2012; Effective: July 1, 2013 Approved focused revision: June 9, 2013; Effective: July 1, 2013
Clinical Competency Committee (CCC)
Department of Graduate Medical Education (GME)
Clinical Competency Committee (CCC):Written description
Department of Graduate Medical Education (GME)
How the CCC does its work can be decided by the Program Director – Subcommittees– Assigning residents to faculty members for pre-review– Pre-review work will vary– Scheduling and frequency of meetings
Clinical Competency Committee (CCC)
Department of Graduate Medical Education (GME)
What Should a CCC Do First?
Understand their specialty Milestones (Posted on acgme.org)
Decide how to assess the Milestones – Program Evaluation Strategy
If necessary, identify new evaluation tools from program director associations, societies, colleges
Department of Graduate Medical Education (GME)
NAS and Milestones and YOU
The program coordinator will play a crucial role in developing, implementing, collecting data on and reporting of milestone evaluation tools.
Department of Graduate Medical Education (GME)
Managing it all…
Department of Graduate Medical Education (GME)
U - R - IT!
1. Understanding RRC program requirements
2. Requirements applied to evaluation methods/process
3. Implementing new evaluation system
4. Tracking completion and accuracy (outliers) for data aggregation
Department of Graduate Medical Education (GME)
1. Understand Your Program’s New Requirements
Core
Outcomes
Details
Department of Graduate Medical Education (GME)
Each standard/requirement is categorized: Outcome - All programs must adhere Core - All programs must adhere Detail – Considered mandatory for new
programs and those that fail to meet core requirements. Allows high-performing programs to innovate.
Source: Implementing The Next Accreditation System ACGME Webinar John R. Potts, III, M.D.: 4 November 2013
1. Understand Your Program’s New Requirements
Department of Graduate Medical Education (GME)
Example Program Requirement: VI.B. Transitions of Care
1. Understand Your Program’s New Requirements
(Core)
(Core)
(Outcome)
Department of Graduate Medical Education (GME)
Example Program Requirement: VI.B. Transitions of Care
When core and outcome not in compliance, then:
1. Understand Your Program’s New Requirements
Details
Department of Graduate Medical Education (GME)
U - R - IT!
1. Understanding RRC program requirements
2. Requirements applied to evaluation methods/process– Linking milestones/EPAs and objectives to evaluation
questions– Utilizing milestone scales
3. Implementing new evaluation system
4. Tracking completion and accuracy (outliers) for data aggregation
Department of Graduate Medical Education (GME)
2. Requirements Specific to CCC Review of Trainee
Aggregating/compiling multiple evaluations of individual trainees (V.A.1. Formative Evaluation)
Tracking trainee participation in conferences, journal clubs, didactics (IV.A.3. Didactic Sessions)
Monitoring duty hour compliance (VI.G. Duty Hours) Reviewing involvement in quality improvement and
patient safety activities (IV.A.5.c. PBLI) Reviewing scholarly work (IV.B. Scholarly Work) Monitoring and reporting procedure logs (IV.A.5.a)
Department of Graduate Medical Education (GME)
U - R - IT!
1. Understanding RRC program requirements
2. Requirements applied to evaluation methods/process– Linking milestones/EPAs and objectives to evaluation
questions– Utilizing milestone scales
3. Implementing new evaluation system
4. Tracking completion and accuracy (outliers) for data aggregation
Department of Graduate Medical Education (GME)
3. Implementing
Evaluation systems– Milestone-based/EPAs– Rotation-specific– Patient handovers– Define evaluator groups (faculty, staff, patients)– Set up and timing of delivery systems
Department of Graduate Medical Education (GME)
3. Implementing
Documentation and reporting systems for:– Conference attendance– Scholarly work (Learning Portfolios)– Quality Improvement and Patient Safety
(Learning Portfolios / Safety reporting systems)– Duty Hours (recording, monitoring, reporting)– Case Logging (if applicable)
Department of Graduate Medical Education (GME)
U - R - IT!
1. Understanding RRC program requirements
2. Requirements applied to evaluation methods/process– Linking milestones/EPAs and objectives to evaluation
questions– Utilizing milestone scales
3. Implementing new evaluation system
4. Tracking completion and accuracy (outliers) for data aggregation
Department of Graduate Medical Education (GME)
4. Tracking and Reporting…
Start with the end in mind:– CCC biannual reporting windows to ACGME
November 1- December 31 / May 1 - June 15
Back track and set calendar events for:– Periodic monitoring of evaluation completion– Running aggregate reports and reviewing milestone
evaluation data– Reviewing case logs, learning portfolios, duty hours
Department of Graduate Medical Education (GME)
Pulling the Data Together
Clinical Competency Committee
End-of-Rotation
Evaluations
Safety Incident Reports
Case Logs
Patient/ Family
Evaluations
Clinical Skills
Assessment
Nursing and Staff / Techs Evaluations
Progress on Milestones
SimLab
In-service training exams
Quality Improvement
Activities
Department of Graduate Medical Education (GME)
Managing it all:How will I pull this off?
Department of Graduate Medical Education (GME)
The Toolbox
Department of Graduate Medical Education (GME)
Creating a Resident Performance Profile Goals to support your CCC
Resident performance data that is: Comprehensive Consolidated / Aggregated Easy for CCC to identify strengths, areas for
improvement, opportunities for advancement
Department of Graduate Medical Education (GME)
Creating a Resident Performance Profile:Compiling and centralizing data
Department of Graduate Medical Education (GME)
Creating a Resident Performance Profile:Apply visual formatting for trends
Department of Graduate Medical Education (GME)
EvaluationsMilestone End of Rotation (Total)Medical KnowledgePatient CareInterpersonal & Communication SkillsProfessionalismSystems-Based PracticePractice-Based Learning & Improvement
Creating a Resident Performance ProfileStep 1 – Defining what to track
Department of Graduate Medical Education (GME)
In-service Assessments (MK; PC)Routine procedure technical skills assessment: Level 2-3Complex procedure technical skills assessment: Level 3-4Medical Knowledge Assessments
Creating a Resident Performance ProfileStep 1 – Defining what to track
Department of Graduate Medical Education (GME)
Case Logs / Clinical Experience (PC)VAGINAL DELIVERYCAESAREAN SECTIONPEDIATRICSPEDIATRICS UNDER 3CARDIACENDOVASCULAR
Creating a Resident Performance ProfileStep 1 – Defining what to track
Department of Graduate Medical Education (GME)
Quality Improvement/ Patient SafetyQI ProjectQI Committee ParticipationQI Course Work (e.g., IHI )SAFE Report / Adverse Event ReviewPatient Handover Evaluations
Creating a Resident Performance ProfileStep 1 – Defining what to track
Department of Graduate Medical Education (GME)
Practice-Based LearningPatient outcomes / Case Study PresentationScholarly Activity: Research studyScholarly Activity: PublicationsPresentation(s) at Internal/National Meetings
Creating a Resident Performance ProfileStep 1 – Defining what to track
Department of Graduate Medical Education (GME)
Interpersonal and Communication SkillsPatient Feedback
Staff EvaluationMedical Student FeedbackPatient Handover Evaluations
Creating a Resident Performance ProfileStep 1 – Defining what to track
Department of Graduate Medical Education (GME)
Creating a Resident Performance ProfileStep 2 - Link data sources to milestones
Evaluations MilestonesMilestone End of Rotation (Total) 24Medical Knowledge MK A1Patient Care PC A1-10Communication ICS A1-2Professionalism PROF 1-5Systems-based Practice SBP A1-2Practice-based Learning PBLI A1-4
Quality Improvement/ Patient SafetyQI Project PBLI1-2QI Committee Participation PBLI1-2QI Course Work (e.g., IHI ) PBLI1-2SAFE Report / Adverse Event Review PROF1/ 2; SBP1Patient Handover Evaluations SBP1; ICS1
Department of Graduate Medical Education (GME)
Creating a Resident Performance ProfileStep 3: CCC defines performance ranges
At or Above Expectation: 2.8 and higher
Below Expectation:1.7 – 2.7
Remediation:Below 1.7
Example:
For all aggregate milestone evaluation scores for a PGY 3, the CCC has defined these ranges by PGY level in advance of the meeting:
STRENGTH WATCH AT RISK
Department of Graduate Medical Education (GME)
Creating a Resident Performance ProfileStep 4 – Set conditional formatting
Department of Graduate Medical Education (GME)
Example: Aggregate milestone evaluation data cells
Highlight cells to apply the conditional formatting
Creating a Resident Performance ProfileStep 4 – Set conditional formatting
Department of Graduate Medical Education (GME)
Creating a Resident Performance ProfileStep 4 – Set conditional formatting
Set Ranges: > , < , between
Department of Graduate Medical Education (GME)
Select, “Greater Than” “Less Than” or “Between” to Set Value RangesChoose the corresponding fill color (e.g., red, yellow, green)
Creating a Resident Performance ProfileStep 4 – Set conditional formatting
Department of Graduate Medical Education (GME)
Resident Performance Profile:Step 5: Enter in data
Department of Graduate Medical Education (GME)
Creating a Resident Performance ProfileVisual trends and detailed data
Department of Graduate Medical Education (GME)
More tools…
Department of Graduate Medical Education (GME)
Leveraging Resident Management System (RMS)Tools, if Available
RMSs – becoming more feature rich Curriculum
– Goals and Objectives and learning outcomes by rotation
– Teaching and Assessment methodologies Evaluation tool development
– Sharing between programs and institutions
Department of Graduate Medical Education (GME)
Conference attendance statistics– Core competencies linked to specified
conferences– Attaching conference materials for later reference
Procedures and levels; linked procedure evaluations
Leveraging Resident Management System (RMS)Tools, if Available
Department of Graduate Medical Education (GME)
• Resident portfolio toolsQI participation and outcomesScholarly Activity logs
Leveraging Resident Management System (RMS)Tools, if Available
Department of Graduate Medical Education (GME)
Aggregate reporting and graphic summaries Peer or departmental average, individual average, minimum
and maximum scores, standard deviation or listing of all scores
Leveraging Resident Management System (RMS)Tools, if Available
Department of Graduate Medical Education (GME)
Set “data gathering and reporting” appointments with yourself
Remember to start with the end in mind (e.g., CCC meeting dates)
Break down large tasks into smaller tasks to keep it manageable
Leveraging Calendaring and Task Management Software
Department of Graduate Medical Education (GME)
Leveraging Calendaring and Task Management Software
Department of Graduate Medical Education (GME)
Learning through Experience
Department of Graduate Medical Education (GME)
When a CCC Meeting…Doesn’t go well
Data – not complete– not organized– not accurate
PD or faculty member dominates meeting
Prolonged inefficient decision making with inability to gain consensus
Unsubstantiated/unreliable conclusions
Does go well
Data– complete – organized– accurate
Cooperative, collaborative decision making
Efficient use of time Sound valid conclusions
aligned with data
Department of Graduate Medical Education (GME)
Successful Resident RankingPGY1- 4: Ready to graduate
Thetical, Hypo Date Completed 6/30/08 6/30/09 6/30/10 6/30/11Evaluations MilestonesMilestone End of Rotation (Total)
251.5 2.3 3.1 3.9
Medical Knowledge MK A1-6 1.3 1.9 3.1 3.6Patient Care PC A1-10 1.4 2.0 2.9 4.0Interpersonal Communication Skills ICS A1-2 1.1 2.3 2.9 3.9
Professionalism PROF 1-3 2.1 2.8 3.5 4.2
Systems-based Practice SBP A1-2 1.3 2.0 3.0 4.1Practice-based Learning & Improvement PBLI A1-2 1.5 2.8 3.2 3.8
Department of Graduate Medical Education (GME)
PGY 1 Ranked at Graduation Level
1
?
Milestone range for a PGY1 should not be a 4.0, 4.5 or 5.0 …
Department of Graduate Medical Education (GME)
Improvement evident
Thetical, Hypo Date Completed 6/30/08 6/30/09 6/30/10 6/30/11Evaluations MilestonesMilestone End of Rotation (Total)
250.9 1.5 2.7 4.0
Medical Knowledge MK A1-6 0.8 1.9 2.6 4.0Patient Care PC A1-10 0.9 0.9 2.9 4.2Interpersonal Communication Skills ICS A1-2 0.7 0.8 2.7 4.2
Professionalism PROF 1-3 0.9 0.5 2.0 3.9
Systems-based Practice SBP A1-2 1.0 2.0 3.0 4.1Practice-based Learning & Improvement PBLI A1-2 1.1 2.8 3.2 3.8
Department of Graduate Medical Education (GME)
We Should not be at this Point
Department of Graduate Medical Education (GME)
Beyond data…Creating a climate of CCC Success
Gentle Words of Wisdom
Tight efficient meetings Ground Rules Beware of Negative Group Think Schedule firm standing meeting dates in advance Reserve room of appropriate size with required
audio-visual tools if needed …..and have snacks
Department of Graduate Medical Education (GME)
Use Technology to Your Advantage…You can be a ‘Rock Star’
Department of Graduate Medical Education (GME)
Session Recap in a Nutshell…
Know your program requirements and follow them unconditionally
Use simple spreadsheet, calendaring and task organizational tools to manage, track and present resident performance data to your CCC
Resident education is a cyclical process – revisit and revise tools and processes each year
Department of Graduate Medical Education (GME)
Questions
Department of Graduate Medical Education (GME)
Contacts
Kim Walker - [email protected] Ann Dohn - [email protected] Nancy Piro - [email protected]