Principles of Electronic Portfolios

© 2004, School of Medical Education Development, University of Newcastle upon Tyne

ePortfolios Ottawa Conference, 2004

Principles of Electronic PortfoliosPrinciples of Electronic Portfolios

Simon Cotterill, Tony McDonaldSchool of Medical Education DevelopmentUniversity of Newcastle upon Tyne, UK



The workshop draws on the experiences of developing, implementing and evaluating Web-based ePortfolios, at the University of Newcastle, UK.

ObjectivesObjectives

• To discuss some of the principles of electronic portfolios• To gain familiarity with the potential educational and technical

‘value-added’ features of using electronic media. • To consider approaches to the implementation of ePortfolios

Proposed StructureProposed Structure

• Presentation: Principles of ePortfolios • Exercise: ePortfolios in your context• Presentation: Case study: FDTL4 ePortfolio project• Exercise: Strengths and weaknesses• Plenary / discussion



UnstructuredUnstructured

FormativeFormative

SampleSample

Best workBest work

Factual / Factual / QuantitativeQuantitative

Learner OwnedLearner Owned

StructuredStructured

SummativeSummative

All workAll work

RepresentativeRepresentative

Reflective / InterpretiveReflective / Interpretive

Employer OwnedEmployer Owned

Introduction – What are Portfolios ?Introduction – What are Portfolios ?



Introduction – What are Portfolios?Introduction – What are Portfolios?A collection of work or evidence on performanceA collection of work or evidence on performance

– Processes (the ‘journey’)Processes (the ‘journey’)– Learning Outcomes (the ‘destination’)Learning Outcomes (the ‘destination’)– Record volume and variety of experiencesRecord volume and variety of experiences

Essays, project work, logs of Essays, project work, logs of experiences/achievements, artwork, records of experiences/achievements, artwork, records of accomplished work, audits etc.accomplished work, audits etc.

Portfolios are defined by their PURPOSE(s)Portfolios are defined by their PURPOSE(s)– Employment: Builder gets commissions on the basis of Employment: Builder gets commissions on the basis of

his record of satisfied customershis record of satisfied customers– Portfolio for application: The researcher includes a Portfolio for application: The researcher includes a

personal profile in a grant application to show experience personal profile in a grant application to show experience – Professional requirements: Med/Law/Education -Professional requirements: Med/Law/Education -

evidence of essential experience and competency that evidence of essential experience and competency that can be auditedcan be audited



• A computer application which stores the user’s information on A computer application which stores the user’s information on a local drive.a local drive.

• A static Web page eg. CV on the InternetA static Web page eg. CV on the Internet

• A portfolio composed using ‘simple tools’ (e.g. Word, A portfolio composed using ‘simple tools’ (e.g. Word, Dreamwaever, Powerpoint etc.)Dreamwaever, Powerpoint etc.)

• A sophisticated database-driven system, typically accessed A sophisticated database-driven system, typically accessed over a network.over a network.

What might an What might an ElectronicElectronic Portfolio be? Portfolio be?



Model presented by Jan van Tartwijk (Utrecht University) Presented at the ALT-SURF Spring Conference & Research Seminar,Edinburgh 2004

Overview(Planning / PDP)

Content(Evidence)

Discussion

3D Model3D Model



1. Should bring significant advantages over paper-based alternatives2. Should be considered in the context of wider (human) processes3. Clarity of purpose(s)4. One size does not fit all5. Should be learner-centric6. Should be an integral part of the learning experience7. Should support life-long learning8. Research / evaluation is essential

….not an exhaustive list !




Principle 1. ePortfolios Principle 1. ePortfolios should bring significant should bring significant advantages over paper-based alternativesadvantages over paper-based alternatives



• Highly customisableHighly customisable• Multi-purpose Multi-purpose eg. formative & summative = reduced duplicationeg. formative & summative = reduced duplication

• Multiple structures / viewsMultiple structures / views• Easier cross-referencingEasier cross-referencing• Sharable / facilitate interactionSharable / facilitate interaction• TransportableTransportable• SearchableSearchable• Reduced adminReduced admin• Secure access from a range of locationsSecure access from a range of locations• Not left on the bus !Not left on the bus !

10 ‘value added’ features of an online approach to portfolios



Potential Weaknesses of Potential Weaknesses of ePortfoliosePortfoliosThere may be barriers to effective use of electronic portfolios e.g. There may be barriers to effective use of electronic portfolios e.g.

• Limited access to computers / InternetLimited access to computers / Internet• Varied IT experience & Technophobia ! Varied IT experience & Technophobia ! • Reliability of IT systemsReliability of IT systems• Possible duplication / overlap with paper systemsPossible duplication / overlap with paper systems

• Research / Evaluation is essential (Principle 8)Research / Evaluation is essential (Principle 8)• Optimisation will take a number of yearsOptimisation will take a number of years



Principle 2. ePortfolios Principle 2. ePortfolios should be considered should be considered in the context of wider (human) processesin the context of wider (human) processes



Humans are important too !Humans are important too !• Need buy-in of key stakeholders:

Learners Clarity of purpose ? Sense of Ownership ?

Tutors & Admin staff Extra work ?

Senior Curriculum staffEmployers / Professional bodies

• Integration with existing workflow and/or change in practice and culture ?• Training and support (all)• Monitoring and evaluation

http://project.cyberpunk.ru/idb/images/logos/cyborg.gif



Principle Principle 3. Clarity of purpose(s)3. Clarity of purpose(s)

• Choose / design your portfolio according to purpose

• Learners and assessors also need to be clear about purpose and requirements (motivation / fairness)

• ePortfolios can readily support multiple purposes

•Portfolio assessment



Summative and Formative Summative and Formative PurposesPurposes• Portfolios are often used to evidence the achievement Portfolios are often used to evidence the achievement

of learning outcomes for of learning outcomes for summativesummative assessment. assessment.

• There may be potential problems if the portfolio There may be potential problems if the portfolio serves both serves both formativeformative and and summativesummative processes. processes.

• Reflection is less likely to be open and honest if the Reflection is less likely to be open and honest if the learner knows that the work will be assessed (loss of learner knows that the work will be assessed (loss of authenticity).authenticity).

• The production of a portfolio can itself be a formative The production of a portfolio can itself be a formative learning process ie. it is as much a ‘journey’ as an learning process ie. it is as much a ‘journey’ as an end-point for assessment.end-point for assessment.



• The use of electronic portfolios may potentially reduce The use of electronic portfolios may potentially reduce the tension between formative and summative the tension between formative and summative processes by supporting both assessed and private / processes by supporting both assessed and private / non-assessed content.non-assessed content.

• Students have the Students have the choicechoice to select which content is to select which content is private and which is made available to assessors, private and which is made available to assessors, appraisers, tutors, peers, and others involved in their appraisers, tutors, peers, and others involved in their education. education.

Summative and Formative Purposes Summative and Formative Purposes (2)(2)



InstitutionalInstitutionalDataData

Portfolio for Presentation

Portfolio for Application

(job / promotion)

Portfolio for Assessment

Portfolio for Accreditation/Revalidation

PDP(shared)

PDP / Reflective(private)

Portfolio for Appraisal

Learner’sLearner’s‘‘repository’repository’

Central data:TranscriptMIS/ HR data

Programme data:Granular assessment dataOutcomes / skills sets

Supporting Multiple PurposesSupporting Multiple Purposes



Portfolio AssessmentPortfolio Assessment

Key issues:Key issues:

• ValidityValidity

• ReliabilityReliability

• Attitudes & behavioursAttitudes & behaviours

• TimeTime demands demands

Baume D. A Briefing on the Assessment of Portfolios. Baume D. A Briefing on the Assessment of Portfolios. LTSN Generic Centre. Assessment Series No.6 (2001)LTSN Generic Centre. Assessment Series No.6 (2001)

Friedman M, Davis MH, Harden RM, Howie PW, Ker J, Pippard MJFriedman M, Davis MH, Harden RM, Howie PW, Ker J, Pippard MJAMEE Medical Education Guide No. 24: Portfolios as a method of AMEE Medical Education Guide No. 24: Portfolios as a method of student assessment. Medical Teacher. 2001; 23:535-51student assessment. Medical Teacher. 2001; 23:535-51

Where to beginbegin ?

KnowsKnows

Knows howKnows how

DoesDoes

Shows howShows how

Miller GE. Miller GE. Acad MedAcad Med 1990:S63-7 1990:S63-7

Assessing theAssessing thedevelopment ofdevelopment ofcompetenciescompetenciesover timeover time



ePortfolios and Assessment

--PotentialPotential benefits of ePortfolios benefits of ePortfolios::• Formative inputFormative input during during portfolio building (not just at end) portfolio building (not just at end)• Random sampling & ‘drilling down’ Random sampling & ‘drilling down’ • Peer Assessment (formative and summative)Peer Assessment (formative and summative)• Automatic inclusion of assessment results + feedbackAutomatic inclusion of assessment results + feedback• Automated summary information (quantitative)Automated summary information (quantitative)• Reduced monitoring / admin demandsReduced monitoring / admin demands

-Piloting essential before use in ‘high-stakes’ assessment-Piloting essential before use in ‘high-stakes’ assessment-Need for further research in this field !-Need for further research in this field !

ePortfolioePortfolioofof assessment (results / essays etc) assessment (results / essays etc)

forfor assessment assessment



Principle 4. One size does not fit allPrinciple 4. One size does not fit all

Institutional Subject / context requirements vs. specific(standardisation) requirements

• ePortfolios should be highly flexible / customisable

• support diversity (accessibility, learning styles etc)



The need for flexibility & customisation

• install from a set of ‘generic’ tools • create context-specific tools via simple Web forms

Course Admin view

Student view

e.g. selecting tools by course / year groups



e.g. configuring learning outcomes / skills sets

Course Admin view

Student view

The need for flexibility & customisation (2)



• Personalisation / customisation

• Sense of ownership / privacy / sharing

• Constructivist theories. is eLearning contributing to ‘Paradigm shift’ ?

Student Independent Learner Teacher Facilitator Didactic Interactive + Explorative 1 location Distance learning 1 provider Multiple providers

Principle Principle 5. ePortfolios should be learner-centric5. ePortfolios should be learner-centric



Students Students controlcontrol which parts of their portfolio can be which parts of their portfolio can be viewed for specific purposes or by specific peopleviewed for specific purposes or by specific people

Mixed-Ownership ModelMixed-Ownership Model

Corporate(HR) data

PersonalData

Tutor’sData

Emphasis on Learner Ownership / ControlEmphasis on Learner Ownership / Control



Ability for Learners to Share their Portfolio Content

Users can control access to their portfolio



Flexibility for the Learner to Add, Link & Cross-Reference ePortfolio Artefacts

Adding ‘child’ objects to an artefact

Cross-referenceAn artefact withA skill



Principle 6. ePortfolios should be an Principle 6. ePortfolios should be an integral part of the learning experienceintegral part of the learning experience

• Educational (integration with the curriculum)

• Technical (integration within the MLE)



• ePortfolio integrated with the VLE

• Tool properties include ‘module code’

= dynamic links to Study Guides + resources

• Learning outcomes linked to curriculum databases: ‘modules’ and units

• Links to other systems eg. SSC selection

Educationalintegration

• Should not be perceived as a ‘bolt-on’

• Support a holistic view of curriculum / outcomes (as well as specific components)

Technicalintegration(example from

Medicine atNewcastle)



Principle 7. ePortfolios shouldPrinciple 7. ePortfolios should support life-long learningsupport life-long learning

• Help develop life-long learning skills

• Continuity in LLL (transferability)



Towards a life-long learning recordTowards a life-long learning recordSchool School HE HE Continuing Development Continuing Development

InteroperabilityInteroperability

(technical standards to support the interaction and transfer of data between IT systems)

• IMS specifications /standards:Learner Information Package (LIP)Enterprise / Content PackagingePortfolio (early stage of

development)CETIS UK Learner Profile (based

on IMS-LIP)

• ePortfolio interaction with VLEs / MLEs

Continuity in LLL (transferability)Continuity in LLL (transferability)



Preparation for professional requirements:

3. Independent learning skills4. Personal development planning5. Time-efficient evidencing of

performance / development for:- Appraisals- Assessment- Revalidation

Aiming to promote a reflective / deep-learning approach ….research

The ePortfolio process should promote The ePortfolio process should promote the develop life-long learning skillsthe develop life-long learning skills



Principle 8. Research / Evaluation is Essential Principle 8. Research / Evaluation is Essential

Are the purpose(s) being fulfilled ?



‘‘A good fit happens over time with lots of use’A good fit happens over time with lots of use’

Spandel, 1997Spandel, 1997cited by Jan van Tartwijk (ALT-SURF Conference, 2004)

• Action Research approaches

• Incremental development and refinement

• Qualitative research methodologies(+some quantitative - not mutually exclusive!)

• Ethical and practical considerations

Evaluation and RefinementEvaluation and Refinement



Research and Evaluation Research and Evaluation Does the ePortfolio improve learning ? How so ?Does the ePortfolio improve learning ? How so ?

Some evidence that PDP improves learning (more needed)Some evidence that PDP improves learning (more needed)A systematic map and synthesis review of the effectiveness of personal Development Planning A systematic map and synthesis review of the effectiveness of personal Development Planning for improving student learning. EPI Centre 2003 http://eppi.ioe.ac.ukfor improving student learning. EPI Centre 2003 http://eppi.ioe.ac.uk

Can ePortfolios foster a reflective approach to learning ?Can ePortfolios foster a reflective approach to learning ?• How defined ? How measured ?How defined ? How measured ?

How do particular groups of learners approach reflective portfolios ?How do particular groups of learners approach reflective portfolios ?• Learning StylesLearning Styles• Gender and AgeGender and Age• Cultural BackgroundCultural Background

John Mole. The geography of thinking. Clin Med 2002; 2:343-5 John Mole. The geography of thinking. Clin Med 2002; 2:343-5

Does the technology change the learning process ?Does the technology change the learning process ?

What are learner and staff perceptions / attitudes towards ePortfolios ?What are learner and staff perceptions / attitudes towards ePortfolios ?



1. Should bring significant advantages over paper-based alternatives2. Should be considered in the context of wider (human) processes3. Clarity of purpose(s)4. One size does not fit all5. Should be learner-centric6. Should be an integral part of the learning experience7. Should support life-long learning8. Research / evaluation is essential

…….the challenge is putting the principles into practice !.the challenge is putting the principles into practice !




Further information:Further information: [email protected]@ncl.ac.ukhttp://www.eportfolios.ac.uk

Documents

Principles of Electronic Portfolios