Upload
noemi-toll
View
213
Download
0
Tags:
Embed Size (px)
Citation preview
Developing High Quality Developing High Quality Clinical Skills AssessmentsClinical Skills Assessments
University of North Carolina – Chapel University of North Carolina – Chapel Hill School of MedicineHill School of Medicine
November 10, 2011November 10, 2011
Ann Jobe, MD, MSNAnn Jobe, MD, MSNClinical Skills Evaluation Collaboration Clinical Skills Evaluation Collaboration
Philadelphia, Pennsylvania Philadelphia, Pennsylvania
The most important consideration in The most important consideration in developing a clinical skills developing a clinical skills assessment is to keep in focus the assessment is to keep in focus the purposepurpose of the assessment of the assessment
Formative or Summative?Formative or Summative?
Formative:Formative: to provide feedback for to provide feedback for improvement of performance, usually improvement of performance, usually midway through a course of study.midway through a course of study.
Summative:Summative: to distinguish between to distinguish between those who are competent, and those those who are competent, and those who aren’t, usually at the end of a who aren’t, usually at the end of a course of study.course of study.
Test Design Test Design Begin at the beginning…Begin at the beginning…
• What are you trying to assess?What are you trying to assess?
• What is the level of the examinee?What is the level of the examinee?
• How much time will the examinee have? How much time will the examinee have? Can they complete the task in the time Can they complete the task in the time allotted?allotted?
Test Design Test Design
• What are you trying to teach and What are you trying to teach and assess?assess?
Objectives for assessment – history taking Objectives for assessment – history taking skills, PE skills, communication skills, data skills, PE skills, communication skills, data interpretation skills?interpretation skills?
• What is the level of the examinee?What is the level of the examinee? Focus and length of checklists will be Focus and length of checklists will be
different for 1different for 1stst year medical students and year medical students and individuals seeking board certification individuals seeking board certification
USMLE Step 2 Clinical SkillsUSMLE Step 2 Clinical Skills
Purpose is to assess ability to:Purpose is to assess ability to: Gather information from patientsGather information from patients Correctly perform physical Correctly perform physical
examination maneuversexamination maneuvers Synthesize and communicate Synthesize and communicate
findings to patients and colleaguesfindings to patients and colleagues
Examinees are moving to supervised Examinees are moving to supervised patient care (PGY-1)patient care (PGY-1)
Development Of An Development Of An Examination BlueprintExamination Blueprint
A test blueprint defines the requirements A test blueprint defines the requirements for each examination, regardless of where for each examination, regardless of where or when it is administered. or when it is administered.
Commonly seen cases.Commonly seen cases.
Blueprint (continued)Blueprint (continued)
Identify the criteria used to define an Identify the criteria used to define an exam blueprintexam blueprint• Examination length Examination length • Case content Case content • Examinee tasks (e.g. history, physical, Examinee tasks (e.g. history, physical,
communication)communication)• Setting: inpatient or outpatient Setting: inpatient or outpatient • Patient gender, age Patient gender, age
Case SelectionCase Selection
Documentation of commonly seen Documentation of commonly seen casescases• if specialty focused assessment, cases if specialty focused assessment, cases
will be those common to that specialtywill be those common to that specialty Most common presenting complaints Most common presenting complaints
in clinical settingin clinical setting Health Department population Health Department population
statistics – especially for region, statistics – especially for region, state or local areastate or local area
Case Pool SizeCase Pool Size
Dependent on blueprint criteriaDependent on blueprint criteria Security concerns (more is better!)Security concerns (more is better!)
Case Content
Cardiovascular Respiratory
Gastrointestinal Musculoskeletal Constitutional Neurological Psychiatric Genitourinary Women’s health Other
Form
Patient age
Age less than 18 Age 18 – 44 Age 45 – 64 Age 65 +
Patient Gender
Male Female
Case Acuity
AcuteSubacute/Chronic
Blueprint Criteria Define Case NeedsBlueprint Criteria Define Case Needs
This information is provided to faculty This information is provided to faculty to guide their case developmentto guide their case development• Case content: GastrointestinalCase content: Gastrointestinal• Case acuity: AcuteCase acuity: Acute• Patient age: 18-44Patient age: 18-44• Patient gender: FemalePatient gender: Female
Complete a “Medically Relevant Case Complete a “Medically Relevant Case Details” sheetDetails” sheet
Case Development Committee Case Development Committee
Physicians (medical school faculty): Physicians (medical school faculty): content expertscontent experts
SP trainers: training expertsSP trainers: training experts SPs: portrayal expertsSPs: portrayal experts
Process evolved from experienceProcess evolved from experience
Usually break into several groups Usually break into several groups
Each group focuses on developing Each group focuses on developing several cases to fit blueprint needsseveral cases to fit blueprint needs
Case Development ProcessCase Development Process Chief Complaint Chief Complaint
““Medically Relevant” case details Medically Relevant” case details
Patient Personality ProfilePatient Personality Profile
Case checklistCase checklist
Examinee instructionsExaminee instructions
Patient note development Patient note development
Chief Complaint/ Chief Complaint/ Differential DiagnosesDifferential Diagnoses
Brainstorming processBrainstorming process List all of the possibilitiesList all of the possibilities Then narrow the choices as Then narrow the choices as
development proceedsdevelopment proceeds
Medically Relevant Case DetailsMedically Relevant Case Details Patient /Case NamePatient /Case Name RaceRace
• Any Any • African American African American • Caucasian Caucasian • Other _________________Other _________________
BMIBMI• AnyAny• Weight proportionate to heightWeight proportionate to height• Overweight: > 24Overweight: > 24• OtherOther
Medically Relevant Case DetailsMedically Relevant Case Details
GenderGender• Male Female EitherMale Female Either
Age RangeAge Range• 10 year age range – example 30 years old = 25-35 10 year age range – example 30 years old = 25-35
AcuityAcuity
• Acute Acute • Subacute/ChronicSubacute/Chronic
Category (Blueprint)Category (Blueprint) Exclusionary Medical conditions/scarsExclusionary Medical conditions/scars Differential DiagnosisDifferential Diagnosis
Medically Relevant Case DetailsMedically Relevant Case Details
Communication TasksCommunication Tasks• Fostering the RelationshipFostering the Relationship• Gathering InformationGathering Information• Providing InformationProviding Information• Making DecisionsMaking Decisions• Supporting EmotionsSupporting Emotions
Overview of Case Stimuli for Communication Overview of Case Stimuli for Communication TasksTasks
Thread of EncounterThread of Encounter Primary MDPrimary MD
Personality ProfilePersonality Profile
Giving life and individuality to the Giving life and individuality to the patients who are being seen – more patients who are being seen – more like “real patients”like “real patients”
Not verbatim statements from a Not verbatim statements from a script – more general ideas of each script – more general ideas of each patient’s perspective patient’s perspective
Personality ProfilePersonality Profile
Who am I and how would I describe Who am I and how would I describe myself?myself?
Why am I here?Why am I here? What made me come in today?What made me come in today? What do I think is going on and why?What do I think is going on and why? What are my expectations for the What are my expectations for the
visit and how will I react if my visit and how will I react if my expectations are not met?expectations are not met?
Personality ProfilePersonality Profile What questions do I have?What questions do I have? What concerns/anxiety/fears do I What concerns/anxiety/fears do I
have?have? How has this illness impacted my How has this illness impacted my
life?life? What do I bring with me?What do I bring with me? What do I look like?What do I look like? How do I behave/what’s my How do I behave/what’s my
communication style (mood/attitude communication style (mood/attitude related to my illness)?related to my illness)?
Personality ProfilePersonality Profile
What is my level of trust of the What is my level of trust of the medical profession?medical profession?
What type of communication style do What type of communication style do I appreciate in a health care I appreciate in a health care provider? How will I respond to provider? How will I respond to different styles of communication? different styles of communication? How much information do I want the How much information do I want the doctor to provide?doctor to provide?
Personality ProfilePersonality Profile
What words won’t I understand What words won’t I understand (jargon)?(jargon)?
What are my own words I use to What are my own words I use to describe what is going on?describe what is going on?
How motivated am I to accept How motivated am I to accept medical advice?medical advice?
How do I feel about the use of How do I feel about the use of complementary/alternative complementary/alternative medicine?medicine?
Case Checklist Case Checklist Recording instrument Recording instrument Checklist length Checklist length One concept per itemOne concept per item Evidence-based items Evidence-based items Use lay language Use lay language Example: Onset Example: Onset (“I’ve been coughing for about a (“I’ve been coughing for about a
month.”)month.”)
Refining the ChecklistRefining the Checklist
Brainstorm a large number of itemsBrainstorm a large number of items Role playRole play Checklist revisedChecklist revised
• A – essentialA – essential• B – importantB – important• C – relevantC – relevant• F - deleteF - delete
Examinee InstructionsExaminee Instructions
NameName AgeAge Setting Setting Presentation problemPresentation problem Vital signsVital signs Examinee Tasks Examinee Tasks
• Refers to the skills that the Refers to the skills that the examination assessesexamination assesses
Role Play with SP Role Play with SP
A physician from another group A physician from another group “sees” the patient“sees” the patient
Following the encounter, the Following the encounter, the physician provides his/her differential physician provides his/her differential diagnoses (see if it matches what the diagnoses (see if it matches what the group selected): face validitygroup selected): face validity
Revise the checklist based on Revise the checklist based on observing the physicianobserving the physician
Role Play with SP Role Play with SP
This allows an SP to provide feedback This allows an SP to provide feedback on any difficulties that arise in on any difficulties that arise in portrayal of the case – especially if portrayal of the case – especially if an SP needs to repeat the case an SP needs to repeat the case several times in an assessment several times in an assessment sessionsession
Development of Patient NoteDevelopment of Patient Note
Key essentials and case-based Key essentials and case-based scoring guidelinesscoring guidelines