MBBS, MPH, MCPS, MRCGP (UK), FRIPH (UK), FHAE (UK)

Osce preparation guidelines

Ass. Prof. Dr. Abdul Sattar KHAN

Family & Community Medicine Department

College of Medicine

King Faisal University

Think ! Short & Long case Exam

04/18/23 ASK 2

• Equality• Easy & difficult patients• Hawks & Doves examiners • Cost• Getting real patients• Unfair exam: complains

OSCE is an examination method

OR

Itself a Test?

04/18/23 ASK 3

Objectives

At the end of the session, participants are expected to:

•Understand the importance of OSCE•Aware about advantages & disadvantages•Know the purpose of OSCE•Understand reliability & validity of OSCE•Learn how to prepare & apply OSCE04/18/23 ASK 4

OSCE is a method not a test

OSCE (Objective Structural Clinical Examination)1

OSLER (Objective structural Long Examination Record)

OSPE (Objective Structural Practical Examination)

TOSCE (GOSCE)- Team (group) Objective Structural Clinical Examination

OSBE (Objective Structured Biostatistics Examination) Etc………

04/18/23 ASK 5

1 Harden RM, Gleeson FA. Assessment of clinical competence using anobjective structured clinical examination (OSCE). Med Educ. 1979;13(1): 41-54.

Advantages •

Test a student’s ability to integrate •knowledge, •clinical skills, and •communication with the patient

•An assessment tool that is custom-fit to the goals of a specific education program

• Provide chance for individualized feedback

04/18/23 ASK 6

Disadvantages

Costly

Time consuming

Case specific evaluation

Quarantining (‘corralling’)

Serious concern (‘yellow card’) reporting systems

04/18/23 ASK 7

What is the purpose of the OSCE?

Provide feedback on performance

Evaluate basic clinical skill

Measure minimal competency

04/18/23 ASK 8

Example of some skills•

Communication and professionalism skills (e.g. breaking bad news)

•bad news)

•History taking skills (e.g. taking a history from a patient)

•presenting with acute chest pain)

•Physical examination skills (e.g. performing a respiratory system examination)

•examination)

•Clinical-reasoning skills (e.g. interpreting clinical data

and then prescribing therapy on a drug chart)

04/18/23 ASK 9

What makes an OSCE a good form of assessment?

Reliability Validity Educational impact Cost efficiency Acceptability

04/18/23 ASK 10

Van der Vleuten CP. The assessment of professional competence:developments, research and practical implications. Adv Health SciEduc. 1996; 1(1): 41-67.

04/18/23 ASK 11

How to prepare OSCE ?

10 – steps approach

Step 1: Selection of OSCE committee

Select some peoples to be involved in following responsibilities: 1. determine the content of

the examination2. development and

implementation of OSCE3. to address decisions

related to reliability and validity

04/18/23 ASK 12

Step 2: Selection of coordinator

A coordinator is the catalyst that facilitates the smooth working of the committee and make liaison with all stakes holders for good arrangement of OSCE.

04/18/23 ASK 13

Step 3: Lists of what need to be assessed

The examination will measure objectively based on objectives of the course, the competencies in specific areas of; behavior techniques attitudesdecision-making strategies etc.

04/18/23 ASK 14

A Blueprint

04/18/23 ASK 15

How to develop case/scenario?

1. The purpose of the station 2. Candidate instructions3. Scoring checklist4. Standardized patient

instructions 5. Instruction for station set-up

04/18/23 ASK 16

1. The purpose of the stationState the skill and domain to be testedE.g; Skill –Physical examination .Domain –Internal medicine /cardiology .

2. Candidate instructions•Candidate instruction must be clear and concise.

04/18/23 ASK 17

3. Scoring checklist•The checklist should be complete and include the main components of the skill being assess.

4. Standardized patient instructions•These instruction must be detailed enough to guarantee standardization patient playing the same role.

5. Instruction for station set-up•List of all equipment required for the station

04/18/23 ASK 18

Step 4: The examinees / students

The examinee is the student, resident, or fellow in training or at the end of training of a prescribed course ORDuring course for practice – formative assessment

04/18/23 ASK 19

Step 5 : The Examiners

• Physicians/Surgeons etc.

• Simulated Patients

• Real patients

• Video recording only

• Self-assessor (Student as examiner)

04/18/23 ASK 20

Step 6: The Examination Site

•

Special rooms •

Big room with partition

• Outpatient facility

04/18/23 ASK 21

Step 7: Examinations Station

•The total number of stations vary based on a function of the number of skills and time managment, ranging from 10-25.

E.g., MRCGP exam has 14 stations

04/18/23 ASK 22

Duration of station•T

imes ranging from 4 to 15 minutes have been reported in different examinations (MRCGP – 10 minutes & USMLE – 15 minutes) and a five minute station probably most frequently chosen.

•This times depend to some extent on the competencies to be assessed in the examination.

04/18/23 ASK 23

Couplet/Linked Station

Some competencies may best be assessed by coupled or linked stations.

Finding Interpretation

Finding Interpretation

Treatment or ManagementTreatment or Management

04/18/23 ASK 24

Station No. 10

Station No. 11

Couplet/Linked Station

May measure competencies at different levels

Basic Level Intermediate Advance level

04/18/23 ASK 25

Station No. 10

Station No. 11

Station No. 12

Step 8: Observer Assessment Method

•Checklist

•Rating scale

04/18/23 ASK 26

04/18/23 ASK 27

The borderline regression (BLR) method

04/18/23 ASK 28

Step 9: Selection of Standard patient

•A standardized patient is an individual with a health problem that is in a chronic but stable condition;

•Standardized or simulated patient usually used when properly trained for history &physical assessment .

04/18/23 ASK 29

http://apt.rcpsych.org/content/8/5/342.full#ref-16

Step 10: Ensure Reliability & Validity

•The patient problem relevant and important to the curriculum

•The station assess skill that have been taught

•Content experts reviewed the station

04/18/23 ASK 30

Factor leading to lower reliability

•Too few station or too little testing time

•Checklists or items that don’t discriminate (too easy OR too hard)

•Unreliable patient or inconsistent portraits by standard patient

•Examiners who score idiosyncratically

•Administrative problem (disorganized staff OR noisy room)

04/18/23 ASK 31

Ready to run the show !

Before the exam:1.Standard setting2.Instructions to supervisors3.Draw a map

04/18/23 ASK 32

Station 1

Station 2

Station 3

Station 4

Station 5

Station 6

Station 7

Station 8

Station 9

Station 10

Station 11

Station 12

Example of 10 station OSCE accommodating 12 studentsExample of 10 station OSCE accommodating 12 students04/18/23 ASK 33

Enter in to station

2 – minute: Ending of station (A buzzer warning sound)

After 10 minutes: Exit station (A buzzer sound)

Enter new station

8 min

1 – minute: Standing outside of station

04/18/23 ASK 34

1 – minute: Standing outside of station

An algorithm for time management during movement in examination day

On the day of the exam:1.Arrive early2.Check completeness3.Note attendance4.Brief candidates

04/18/23 ASK 35

Ready to run the show !

Ready to run the show !

After the exam:1.Store resources2.Re-arrange venues3.Feedback4.Present results to board/committee5.Amend / bank stations6.Analyze the result

04/18/23 ASK 36

Some final tips !

Spare standardized patients and examiners

Back-up equipment

Staff available during the examination to maintain exam security

The bells or buzzers can be heard from all location with closed door

Prepare an extra station

04/18/23 ASK 37

Take Home Message

Attaining clinical competence is not a one-off event but a career long learning

routine

04/18/23 ASK 38

Epstein RM. Assessment in medical education. N Engl J Med. 2007;356(4): 387-96