University of ManitobaPre-Survey Meeting with

Resident Representatives &Senior Residents

Date: July 3, 2013Time: 2:00 to 4:00 p.m.Room: Pharmacy Apotex, Theatre # 264

Objectives of the Meeting

To review the:• Accreditation Process• Categories of Accreditation• Standards of Accreditation• Role of residents in the accreditation

process

•Program Administration• Member of the Residency Program Committee• Must be elected• Communication to and from Residency Program

Committee• Evaluations• Evaluation of the program• Rotations, teachers, teaching

• Understand the Standards• Feedback on how program is meeting

standards

Role of Residents in the Accreditation Process

• Continuing quality improvement process

• Peer-reviewed• Medical education experts• Outside discipline

• Based on Standards• Includes competency framework

• Categories of Accreditation• Regular Schedule

Principles of Accreditation

Internal Reviews

1

2

34

5

6

Monitoring

Six Year Survey Cycle

• Chair - Dr. Sarkis Meterissian– Responsible for general conduct of survey

• Deputy chair – Dr. Maureen Topps– Visits teaching sites / hospitals

• Surveyors • Resident representatives – CAIR• Regulatory authorities representative – FMRAC•Teaching hospital representative – ACAHO

The Survey Team

•Questionnaires and appendices– Completed by program

•Program-specific Standards (OTR/STR/SSA)

•Report of last regular survey•Specialty Committee comments

– Also sent to PGD / PD prior to visit•Exam results for last six years•Reports of mandated Royal College reviews

since last regular survey, if applicable

Information Given to Surveyors

Includes:• Document review (30 min)

• Meetings with:– Program director (75 min)– Department head (30 min)– Residents – per group of 20 (60 min)– Teaching staff (60 min)– Residency Program Committee (60 min)

The Survey Schedule

Document review (30 min)• Residency Program Committee Minutes• Resident Assessments Files

The Survey Schedule

•Program director• Overall view of program• Evaluation of Standards

•Department head• Support for program• Resources available to program

•Teaching faculty• Involvement with residents• Communication with program director

Meeting Overview

•Group(s) of 20 residents (60 min)

• If off-site, tele- or video- conferencing•Looking for balance of strengths &

challenges•Focus on Standards•Evaluate the learning environment

Meeting with ALL Residents

– Objectives– Educational experiences– Service /education balance– Increasing professional responsibility– Academic program / protected time– Supervision– Assessments of resident performance– Evaluation of program / assessment of faculty– Career counseling– Educational environment– Safety

Topics to discuss with residents

• Complete the CAIR questionnaire • Confidential, not given to survey team

•Meet together as a group to discuss the strengths & challenges of your program • 1 to 2 months before survey

•Obtain a copy of the pre-survey questionnaires and the previous survey report

• If you feel you need more time with surveyor, request it

•Be open and honest with surveyor• Comments in meetings are anonymous

Preparing for the SurveyRole of the Resident

All members of RPC attend meeting, including resident members

• Review Committee responsibilities• Opportunity for surveyor to provide

feedback on information obtained during survey

Meeting with Residency Program Committee

•Survey team discussion– Evening following review

•Feedback to program director– Exit meeting with surveyor – Survey team recommendation

• Category of accreditation• Strengths & weaknesses

The Recommendation

New terminology – June 2012• Approved by the Royal College, CFPC and CMQ.

Categories of Accreditation

Accredited program• Follow-up:

– Next regular survey – Progress report (Accreditation Committee)– Internal review– External review

Accredited program on notice of intent to withdraw accreditation

• Follow-up:– External review

Categories of Accreditation

• Accredited program with follow-up at next regular survey

– Program demonstrates acceptable compliance with standards.

Categories of Accreditation Definitions

• Accredited program with follow-up by College-mandated internal review

– Major issues identified in more than one Standard

– Internal review of program required and conducted by University

– Internal review due within 24 months

Categories of Accreditation Definitions

• Accredited program with follow-up by external review

– Major issues identified in more than one Standard AND concerns -• are specialty-specific and best evaluated by a

reviewer from the discipline, OR• have been persistent, OR• are strongly influenced by non-educational issues

and can best be evaluated by a reviewer from outside the University

– External review conducted within 24 months– College appoints a 2-3 member review team – Same format as regular survey

Categories of Accreditation Definitions

• Accredited program on notice of intent to withdraw accreditation

– Major and/or continuing non-compliance with one or more Standards which calls into question the educational environment and/or integrity of the program

– External review conducted by 3 people (2 specialists + 1 resident) within 24 months

– At the time of the review, the program will be required to show why accreditation should not be withdrawn.

Categories of Accreditation Definitions

SURVEY TEAM

ROYAL COLLEGESPECIALTY COMMITTEE

ACCREDITATION COMMITTEE

Repo

rts

Repo

rts

&

Resp

onse

s

Recommendation

Reports

Responses

After the Survey

Report &Response UNIVERSI

TY

• Chair + 16 members• Ex-officio voting members (6)

– Collège des médecins du Québec (1)– Medical Schools (2)– Resident Associations (2)– Regulatory Authorities (1)

•Observers (9)– Collège des médecins du Québec (1)– Resident Associations [CAIR & FMRQ] (2)– College of Family Physicians of Canada (1)– Regulatory Authorities (1)– Teaching Hospitals (1)– Resident Matching Service (1)– Accreditation Council for Graduate Medical Education (2)

The Accreditation Committee

•All pre-survey documentation available to the surveyor

•Survey report•Program response•Specialty Committee recommendation•History of the program

Information Available to the Accreditation Committee

•Decisions– Accreditation Committee meeting

• May/June 2014• Dean & postgraduate dean attend

– Sent to• University• Specialty Committee

•Appeal process is available

The Accreditation Committee

“A” Standards• Apply to University, specifically the PGME office

“B” Standards• Apply to EACH residency program

“C” Standards• Apply to Areas of Focused Competence (AFC)

programs

General Standards of Accreditation

A1 University StructureA2 Sites for Postgraduate Medical

EducationA3 Liaison between University and

Participating Institutions

“A” Standards

B1 Administrative StructureB2 Goals & ObjectivesB3 Structure and Organization of the

ProgramB4 ResourcesB5 Clinical, Academic & Scholarly

Content of the ProgramB6 Assessment of Resident

Performance

“B” Standards

There must be an appropriate administrative structurefor each residency program.

• Program director• Time & support

• Residency Program Committee• Representative from each site and major

component• Resident member(s)

- Must include at least ONE elected resident• Meets regularly, four times a year

- Minutes

B1 – Administrative Structure

•Responsibilities of the Residency Program Committee• Selection, assessment & promotion of residents• Ongoing review of program• Assessment of program / teachers / rotations• Research environment• Appeal mechanism• Career & stress counseling• Resident safety

B1 – Administrative Structure

•Program director autocratic•Residency Program Committee

dysfunctional– Unclear Terms of Reference (membership, tasks

and responsibilities)• Agenda and minutes poorly structured• Poor attendance

– Department head unduly influential– RPC is conducted as part of a Dept/Div meeting

•No resident voice

B1 – Administrative Structure“Pitfalls”

B2 – Goals & Objectives

There must be a clearly worded statement outlining the goals of the residency program and the educational

objectives of the residents.

• Rotation-specific• Structured to reflect CanMEDS

Competencies• Circulated to residents & teaching

staff• Used in planning and assessment

of residents

CanMEDS Competencies

•Medical Expert•Communicator•Collaborator•Manager•Health Advocate•Scholar•Professional

B2 – Goals & Objectives“Pitfalls”

•Missing CanMEDS roles in overall structure

– Okay to have rotations in which all CanMEDS roles may not apply (research, certain electives)

•Goals and objectives not used by faculty/residents

•Goals and objectives dysfunctional – does not inform assessment

•Goals and objectives not reviewed regularly

There must be an organized program of rotations and other educational experiences, both mandatory

and elective, designed to fulfill the educational requirements and allow residents to achieve

competence in the specialty.

• Include all components of specialty• Equivalent opportunity• Senior residency

B3 – Structure & Organization of the Program

•Increasing professional responsibility•Appropriate supervision•Balance of service and education•Safe learning/educational environment

•Promotes resident safety•Free from intimidation, harassment or abuse

B3 – Structure & Organization of the Program

•Graded responsibility absent•Service/education imbalance

– Service provision by residents should have a defined educational component including evaluation

•Educational environment poor

B3 – Structure & Organization“Pitfalls”

There must be sufficient resources to provide the opportunity for all residents to achieve the educational objectives.

• Teaching faculty• Variety & number of patients, specimens

and procedures• Physical and technical facilities• Inpatient, ambulatory, emergency, ICU• Educational

B4 – Resources

• Insufficient faculty for teaching/ supervision

• Insufficient clinical/technical resources• Infrastructure inadequate

B4 – Resources “Pitfalls”

There must be a clinical, academic and scholarly program that prepares residents to fulfill all the roles of the

specialist.

• Academic program• Organized curriculum• Organized teaching in basic & clinical sciences

• Evidence of teaching the CanMEDS Competencies

• Attendance• Staff, residents

B5 – Clinical, Academic & Scholarly Content of Program

•Organized academic curriculum lacking or entirely resident driven

– Poor attendance by residents and faculty•Teaching of essential CanMEDS roles

missing•Role modelling is the only teaching

modality

B5 – Clinical, Academic & Scholarly Content of Program “Pitfalls”

There must be mechanisms to ensure systematic assessment of each resident.

• Based on goals & objectives• Uses appropriate and varied assessment

methods• Feedback

• Formal, timely, appropriate• Face-to-face• Adequately documented

B6 – Assessment of Resident Performance

• Mechanism to monitor, promote, remediate residents lacking

• Formative feedback not provided and/or documented

• Assessments not timely, not face to face

• Summative assessment (ITER) inconsistent with formative feedback, unclearly documents concerns/ challenges

B6 – Assessment of Resident Performance “Pitfalls”

44

University of ManitobaOn-site Survey

February 23 to 28, 2014

accred@royalcollege.ca613-730-6202

Office of Education

Sarah TaberAssistant Director

Education Strategy & Accreditation

Educational Standards Unit

Sylvie LavoieSurvey Coordinator

Contact Information at theRoyal College