Portfolios – The 5 WsPortfolios – The 5 Ws

David TaylorDavid TaylorSchool of Medical EducationSchool of Medical Education

University of LiverpoolUniversity of Liverpool

The five WsThe five Ws

WhoWho WhatWhat WhereWhere WhenWhen hoWhoW

Why ?Why ?

There are many ways of assessing There are many ways of assessing knowledge and skillsknowledge and skills

But we need some way of recording But we need some way of recording activitiesactivities

And developing and demonstrating And developing and demonstrating appropriate attitudesappropriate attitudes

WhoWho

Our course is five years long, and is Our course is five years long, and is followed by two years in foundation posts followed by two years in foundation posts before the Doctor is fully registered and before the Doctor is fully registered and allowed to practice independently.allowed to practice independently.

Who (2)Who (2)

We currently use We currently use portfolios in years portfolios in years 1,3 & 5 of the 1,3 & 5 of the undergraduate undergraduate course F1, F2course F1, F2

And for consultants, And for consultants, academics and academics and anyone seeking anyone seeking promotionpromotion

11 22 33 44 55 FF11

FF22

……

What is a portfolioWhat is a portfolio

It is a collection of documentsIt is a collection of documents Supported by a commentarySupported by a commentary

It can be electronic It can be electronic more convenient in the University contextmore convenient in the University context

It can be on paperIt can be on paper More convenient in the clinical contextMore convenient in the clinical context

What documents?What documents?

It depends on the reason for the It depends on the reason for the portfolio!portfolio!

DiariesDiaries Log-booksLog-books Case notesCase notes Examination resultsExamination results Longer discursive piecesLonger discursive pieces

Dewey (1933)Dewey (1933)

Reflection is…Reflection is… A purposeful form of thought A purposeful form of thought

provoked in learners when they provoked in learners when they recognize that their understanding is recognize that their understanding is incomplete.incomplete.

In order to reflectIn order to reflect

3 things are needed 3 things are needed something to reflect onsomething to reflect on reflection time & modellingreflection time & modelling motivation motivation

Albanese,2006,Medical Education Albanese,2006,Medical Education 40;288-290 40;288-290

What sort of commentary?What sort of commentary? TechnicalTechnical

What and how of practiceWhat and how of practice Facts and theoriesFacts and theories AuditAudit

PracticalPractical How decisions are madeHow decisions are made Articulate concernsArticulate concerns Resolve professional dilemmasResolve professional dilemmas

CriticalCritical Social and political context and Social and political context and

constraintsconstraints

What is includedWhat is included

The portfolios all include examination The portfolios all include examination results and the student commentary on results and the student commentary on them.them.

In the first year they also include In the first year they also include reflection on settling in and adjusting to reflection on settling in and adjusting to University life.University life.

In the third year they include reflections In the third year they include reflections on professional attitudes students have on professional attitudes students have experienced or observed.experienced or observed.

What is included in later What is included in later yearsyears

In fifth and subsequent years they In fifth and subsequent years they also include reflection on particular also include reflection on particular cases/treatment regimescases/treatment regimes

In F1/F2 and beyond they will include In F1/F2 and beyond they will include reflection upon critical incidentsreflection upon critical incidents

Final year PETAFinal year PETA

Final year PETA Bottom halfFinal year PETA Bottom half

WhenWhen

Right from the start and oftenRight from the start and often Habit helpsHabit helps

Novelty is importantNovelty is important So So wewe focus on different things each year focus on different things each year Settling in, clinical experiences, ethical Settling in, clinical experiences, ethical

behaviour, evidence base to medical practice, behaviour, evidence base to medical practice, audit, future plans, critical incidentsaudit, future plans, critical incidents

It has to be relevant It has to be relevant In the clinical arena it will focus on cases In the clinical arena it will focus on cases

determined by the agreed learning outcomesdetermined by the agreed learning outcomes

WhereWhere

Wherever the student worksWherever the student works In the first/second year this is often at the In the first/second year this is often at the

computer, so we use electronic portfolioscomputer, so we use electronic portfolios In the UK the hospital computers won’t In the UK the hospital computers won’t

“talk” to the internet, so where a VPN can’t “talk” to the internet, so where a VPN can’t be set up we use….be set up we use….

Paper, and interviews, formal and informal, Paper, and interviews, formal and informal, with the educational supervisorswith the educational supervisors

HowHow

How is it assessed?How is it assessed?

Through discussion with a supervisorThrough discussion with a supervisor The whole thing?The whole thing? The reflective component?The reflective component? Parts chosen by the student?Parts chosen by the student? Parts chosen by the supervisor?Parts chosen by the supervisor?

Marks?Marks?

Best practice currently is to use criteria Best practice currently is to use criteria rather than a “checklist”rather than a “checklist”

The criteria need aggregating in domainsThe criteria need aggregating in domains KSAKSA Burch, Seggie and Gary 2006 SAMJ 96;430-433Burch, Seggie and Gary 2006 SAMJ 96;430-433

The number of raters needed for validity The number of raters needed for validity depends on the criteria and the domainsdepends on the criteria and the domains Roberts et al.,2006 Med Educ 40;363-370Roberts et al.,2006 Med Educ 40;363-370

How (2)How (2)

It is the main form of assessment for our final It is the main form of assessment for our final year students.year students.

They sit “finals” at the end of their fourth year.They sit “finals” at the end of their fourth year. Their final year is a clinical apprenticeship, Their final year is a clinical apprenticeship,

assessed through the portfolios.assessed through the portfolios. 5 different assessors (3 meetings each - plus 5 different assessors (3 meetings each - plus

referees in the case of difficulty/expected referees in the case of difficulty/expected difficulty)difficulty)

Mid-Year portfolio review (all borderline/fails and Mid-Year portfolio review (all borderline/fails and sample of others)sample of others)

End of year review of borderline/failsEnd of year review of borderline/fails

BenefitsBenefits

Allows the student to demonstrate insight Allows the student to demonstrate insight and understandingand understanding

Allows student and staff to observe Allows student and staff to observe development and progressdevelopment and progress

Models the “real world”Models the “real world” Goes one step beyond the “competencies” Goes one step beyond the “competencies”

model – in developing professionalismmodel – in developing professionalism Provides a focus (and evidence) for the Provides a focus (and evidence) for the

discussion between student and doctordiscussion between student and doctor

F1F1

StaffStaff

David Taylor’s response..David Taylor’s response..