2013

2013; 35: 287–294

The role of ePortfolios in supporting continuingprofessional development in practice

JENNIFER A. GORDON & CRAIG M. CAMPBELL

Royal College of Physicians and Surgeons of Canada, Canada

Abstract

ePortfolios, based on models of reflective practice, are viewed as important tools in facilitating and supporting lifelong learning

across the medical education continuum. MAINPORT, the ePortfolio designed by the Royal College of Physicians and Surgeons of

Canada, supports the continuing professional development (CPD) and lifelong learning of specialist physicians practicing in

Canada by providing tools to develop CPD plans, set and track progress of established learning goals, document and reflect

on learning activities, and create the foundation for physicians to manage their learning. In this article, the authors summarize the

key design principles of the Royal College’s ePortfolio: learner-centered; interoperable; ease of access. The current core

functionality as well as future planned functionality for MAINPORT are described under three domains: recording and reflecting

on completed CPD activities; managing learning in practice; accessing learning resources and programs. The future MAINPORT

will evolve to become a foundational tool to support the shift towards competency-based medical education across the continuum

of medical education; from residency to retirement. MAINPORT will facilitate the ability of physicians to demonstrate their

expertise over time and how their learning has enabled improvements to their practice in contributing to improved health

outcomes for patients.

Introduction

Over the past two decades, systems of continuing professional

development (CPD), in response to concerns regarding

persistent gaps in physician performance (Grol et al. 2002),

adherence to health care indicators (McGlyn et al. 2003) and

quality of care measures (Institute of Medicine 1999; Forster

et al. 2004), have changed their focus from a primary

emphasis on knowledge dissemination strategies to facilitating

a culture of lifelong learning and self-improvement. This

cultural shift has been facilitated by CPD systems defining

values, educational principles, and measureable outcomes

and informed by a growing scientific evidence base on the role

of learning on behaviour change, performance improvement

and health outcomes (Davis et al. 1999; Grimshaw et al. 2002;

Overeem et al. 2007).

The cultural shift in CPD has been equally enabled by

technological innovations that have assisted physicians, for

example, to access CPD resources when addressing identified

practice needs, share stories and practice innovations within

inter-professional teams or communities of practice

(Parboosingh 2002), to use simulations (McGaghie et al.

2006; Gurusamy et al. 2008) and other self-assessment

strategies, and leverage data about how performance

and health outcomes can inform future learning (van der Veer

et al. 2010).

One of the technological innovations to support learning

in practice is the ePortfolio which may be defined as ‘a

purposeful aggregation of digital items – ideas, evidence,

reflections, feedback, etc., which ‘presents’ a selected

audience with evidence of a person’s learning and/or ability’

(Sutherland et al. 2007).

ePortfolios have been introduced across a range of health

professions within undergraduate (Finlay et al. 1998; Scholes

et al. 2004; Driessen et al. 2005; Ross et al. 2009; Hall et al.

2012), postgraduate (Teunissen et al. 2008; Wassef et al. 2012)

and CPD (duBoulay 2000; Austin et al. 2005; Davies et al. 2005;

Galbraith et al. 2008; Green et al. 2009; Kardos et al. 2009;

Tompkins et al. 2010) to support learning, reflection, self-

assessment, and the provision of feedback. The development

and implementation of ePortfolios within these contexts have

been viewed as important tools in facilitating the transition

towards competency-based medical education spanning the

continuum from residency to retirement. Several recent

systematic reviews have examined the educational effective-

ness of ePortfolios for learning and assessment within under-

graduate and postgraduate health education (Driessen et al.

2007; Buckley et al. 2009; Tochel et al. 2009). However, in

Practice points

. ePortfolios support lifelong learning across the con-

tinuum of education from residency to retirement

. ePortfolios can support reflection, assessment, and the

management of learning

. The design and functionality of ePortfolios may vary

based on needs and goals, but must be learner-centered

. Technology standards have facilitated interoperability,

data transfer, and reporting

Correspondence: Jennifer Gordon, Royal College of Physicians and Surgeons of Canada, Ottawa, Ontario K1S 5N8, Canada. Tel: (613) 730 6211;

fax: 613-730-2410; email: jgordon@royalcollege.ca

ISSN 0142–159X print/ISSN 1466–187X online/13/040287–8 � 2013 Informa UK Ltd. 287DOI: 10.3109/0142159X.2013.773395

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Reg

ina

on 0

9/07

/13

For

pers

onal

use

onl

y.

comparison to the initial phases of medical education, there

has been a relative paucity of literature on the role, utility or

functionality of ePortfolios to support or enable practice

reflection, lifelong learning and assessment of physicians in

practice.

Within professional practice contexts, portfolios were

introduced, initially using paper formats (Wilkinson et al.

2002), to provide a tool for health professionals to document

(for study credit) their participation in conferences, courses

or self-directed learning activities stimulated by practice. Over

time, electronic portfolios have supplanted paper versions

and transitioned from tools to document participation in

learning activities into tools that support multiple functions

including the ability to set and monitor goals, plan CPD

activities, manage learning projects, access learning resources

and document the outcomes of self-directed learning, self-

assessment and performance assessment for practice.

In Canada, the Royal College of Physicians and Surgeons of

Canada (www.royalcollege.ca), a not-for-profit professional

organization with a membership of 44,000 medical and

surgical specialists and resident affiliates, is responsible for

establishing the training requirements, accreditation standards

and national certification examinations for 67 specialties and

subspecialties. In 2001, the Royal College introduced a

mandatory Maintenance of Certification (MOC) Program to

support, enhance, and promote lifelong learning of Fellows

and other health care professionals in Canada. Participation in

the MOC Program, a mandatory requirement for maintaining

membership and the continued use of the Fellowship desig-

nation FRCPC or FRCSC, was supported by the provision of an

ePortfolio, MAINPORT.

In this article, the authors provide a brief review of the

theoretical models of practice reflection that have often

influenced the design and structure of ePortfolios; describe

the structure, design principles and key functional elements

included within MAINPORT, share data about how physicians

have used MAINPORT in practice; and discuss strengths,

barriers and plans for future development. This article is based

in part on the presentations included at the MedBiquitous

conference in May 2012.

Practice reflection: Theoreticalframeworks

Underlying theoretical frameworks that guided the original

development of ePortfolios by the Royal College were initially

based on models of practice reflection described by Schon

(1987) and Moon (2004). Donald Schon’s model of practice

reflection described a process where physicians are stimulated

by their ‘practice experiences (based on an awareness of a

need) to pause and reflect during (reflection-in-action) and/or

after (reflection-on-action) a process to frame and resolve the

issue identified. The learning process that seeks to gather,

analyze and apply evidence (both scientific and tacit) enables

physicians to learn from experience and guide the application

of new knowledge to reduce the risk of being ‘surprised’ in the

future. This process of pausing and learning before acting has

been recently described as self-monitoring by Eva and Regehr,

as an important component of physician self-assessment

(Eva & Regehr 2005). Moon defined reflection as ‘a form of

mental processing with a purpose and/or anticipated out-

comes that is applied to relatively complex or unstructured

ideas for which there is not an obvious solution’ (Moon 2004).

Moon’s model of reflection begins with noticing (an issue,

need or dilemma) followed by making sense, making meaning

that leads to transformative learning. A summary of the

evidence on the importance and impact of reflection and

reflective practice was the basis for a recent systematic review

(Mann et al. 2007). This literature review concluded that

‘Reflection is demonstrated among practicing professionals. It

appears that it fulfills several functions, including helping to

make meaning of complex situations and enabling learning

from experience’.

Reflection and reflective practice models emphasize the

importance of a purposeful critical analysis of experience or

knowledge to achieve greater understanding and meaning as

part of a continuous re-construction of practice (Parboosingh

2002). This process is central to how ePortfolios can facilitate

and support the capture the outcomes of reflection and

potentially facilitate the effectiveness of lifelong learning

defined as:

a continuously supportive process that stimulates

and empowers individuals to acquire all the know-

ledge, values, skills, and understanding they will

require throughout their lifetimes and to apply them

with confidence, creativity, and enjoyment in all

roles, circumstances, and environments. (Bankey

2007)

Definitions of reflection and lifelong learning emphasize

both the processes and skills required to continually seek,

acquire, renew, and upgrade knowledge, skills, competencies

and attitudes. Lifelong learning and reflection can be

stimulated by engaging in formal activities (where learners

have limited control over the learning objectives or methods),

informal or incidental activities (where learners have control

over the purpose of learning but the methods of learning

cannot be pre-planned) and self-directed learning (where the

learner controls the purpose and methods).

Structure and purpose ofePortfolios

Although there are numerous variations to the design and

structure of ePortfolios, the majority of ePortfolios fall within

one or more the following types (van Tartwijk & Driessen

2004):

Assessment portfolios: are usually organized to facilitate

learners with the opportunity to demonstrate achievement of

curricular objectives and for supervisors or evaluators to

review and assess specific learner achievements (products,

presentations, assessment scores, clinical outcomes).

Showcase portfolios: are intended to facilitate the ability of

learners to select, collect and display their best work or

evaluations from that work. Similar to portfolios in the visual

arts, showcase portfolios provide a record of personal devel-

opment and achievement that is dynamic and reflects a virtual

curriculum vitae.

J. A. Gordon & C. M. Campbell

288

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Reg

ina

on 0

9/07

/13

For

pers

onal

use

onl

y.

Development portfolios: enable learners to plan and keep

track of their own learning activities and personal develop-

ment plans. These development portfolios may be structured

around competencies and requirements to demonstrate pro-

gression and performance improvement towards mastery.

Reflective portfolios: provide learners and others with the

opportunity to reflect on experiences, learning activities,

critical incidents to make sense of one’s work with the

opportunity to receive feedback from others (mentors,

coaches, supervisors, etc.).

These basic types of portfolios are not mutually exclusive;

ePortfolios can be developed to integrate a combination of

these purposes depending on organizational needs.

MAINPORT incorporates several components from each of

the above-described types of ePortfolios. Regardless of the

type or purpose, typically, ePortfolio systems support most,

but not necessarily all, of the following processes (Becta 2008):

(1) Capturing and storing evidence

(2) Reflecting

(3) Giving and receiving of feedback

(4) Planning and setting goals

(5) Collaborating

(6) Presenting to a specific audience

ePortfolios for CPD at the RoyalCollege: Design characteristicsand functional elements

The development of the Royal College’s first ePortfolio, PC

Diary, was originally based on the Schon’s model of practice

reflection (Schon 1987) and designed to encourage specialists

to ‘consciously reflect on their practice experiences, identify

items of learning that they perceived had expanded or

consolidated their expertise and plan their professional devel-

opment’ (Campbell et al. 1996). With the implementation of

the Royal College’s mandatory Maintenance of Certification

(MOC) Program in 2001, all Fellows in full-time or part-time

practice or engaged in related professional development

activities, were required to document their learning activities

in a web-based ePortfolio, now called MAINPORT. The

structure and functionality of the Royal College’s MOC

Program and MAINPORT changed in 2011 to respond to the

findings from a survey completed by 2914 Fellows as part of a

formal MOC Program evaluation initiated in 2008 as well as

from an extensive review of the CPD research literature

applicable at that time.

The new MOC Program framework was simplified to reflect

how physicians learn and arranged an expanded number of

learning activities accepted for MOC credit under three

sections: group learning, self-learning, and assessment. Also,

a new revised credit system provided greater incentives for

MOC Program participants to engage in some self-learning and

all assessment activities. The new, re-designed MAINPORT

provided a more streamlined and user-friendly experience for

physicians with a number of new functional elements,

reflecting features from several of the types of ePortfolios

described above. Specifically, the new MAINPORT simplified

the documentation process for CPD activities and learning

outcomes for credit while providing tools to develop CPD

plans, set and track progress of established learning goals. It

also provided a learning space to access e-learning resources

(online journals and websites) that could be personalized and

created the foundation for physicians to manage their learning

in practice. These changes were intended to support the

transition toward competency-based CPD (Campbell et al.

2010).

Design characteristics

The key design principles of the Royal College’s MAINPORT

are summarized in Table 1.

Learner-centered

MAINPORT was designed so that its initial landing page

provided a high-level integrated ‘Dashboard’ (Figure 1). The

Dashboard presents a visual summary of the number of MOC

credits earned within the physician’s current MOC Program

cycle, the CPD activities completed, the learning activities in

progress or awaiting completion, and a listing of the CPD goals

being pursued. The Dashboard also offers an interface that is

intuitive to promote quick navigation and ease of use. The

provision of links to resources available from the Royal College

is complemented by the opportunity for each physician to

create a list of personal URL links to online resources such as

journals, websites or databases used by the physician. The

ePortfolio supports a learner-centered approach by offering

the user an ability to personalize the Dashboard to facilitate

access to learning resources commonly used in practice.

Interoperable

A key strategy for the current MAINPORT was to reduce the

burden of documentation by physicians. To accomplish this

design principle, multiple automation capabilities were

enabled including the sharing or transfer of data between

multiple technological systems. MAINPORT was designed to

receive data, including reflections documented by physicians,

from external third-party sources, directly into individual

MAINPORT accounts. One example of an automated

data transfer implemented to date includes the direct trans-

fer of data from InfoPOEMs (CMA website 2012) derived

from a physician’s review and recording of their reflections

Table 1. ePortfolio design principles.

� Learner-driven: the learner is responsible for the content of the

ePortfolio.

� User-friendly: an interface that is intuitive to promote ease of use.

� Flexibility: ability of learners to personalize access to tools, resources,

programs.

� Interoperability: to facilitate automation of data transfer from multiple

sources.

� Ease of Access: real-time access from multiple platforms including

mobile devices.

� Security: ensuring confidence in the security and privacy of data.

� Personal Growth and Professional Development: a ‘story of learning’

and expertise.

ePortfolios for CPD

289

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Reg

ina

on 0

9/07

/13

For

pers

onal

use

onl

y.

Figure 1. MAINPORT dashboard screen capture.

J. A. Gordon & C. M. Campbell

290

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Reg

ina

on 0

9/07

/13

For

pers

onal

use

onl

y.

on a summarized article provided by the Canadian

Medical Association. Other automated data transfer capabilities

include the reporting of completion of the Royal College’s

accredited online Bioethics self-assessment programs, as well

as participation in various accredited conferences held by

Royal College accredited CPD providers. The capability to

automatically transfer data from these external sources to each

physician’s MAINPORT account was made possible through

the MedBiquitous Activity Report standard (MedBiquitous

2012) that supported this design element of MAINPORT. The

data transfer process is based on a webservice API call, using a

standard XML format, to allow external CPD providers’ systems

to connect with MAINPORT via the Internet. These connec-

tions enable the Royal College to facilitate online, real-time

interactions with external organizations and reduce the docu-

mentation burden for physicians, saving them time and

focusing their documentation on what they learned, changed

or are planning to learn more about to enhance their practice.

In addition to helping reduce the documentation burden on

the physician, automation also provides a method of external

validation of the completion of an activity or attendance at an

event as compared to traditional physician self-reporting of

participation.

Ease of access

Beyond facilitating access to various online resources, the re-

designed MAINPORT ePortfolio enabled mobile access from

multiple platforms including the provision of BlackBerry and

Android apps in July 2012 and the development of a native

iPhone app in September 2012. The planned development

of mobile apps, based on demand from physicians, has not

only facilitated access to MAINPORT but encouraged phys-

icians to record and reflect as they learn. Key functionality of

the apps enables users to submit, modify or delete CPD

activities as well as review incomplete activities in their

MAINPORT accounts. Since the launch of the iPhone app in

September 2012, over 7000 Fellows have downloaded the app

and over 3000 Fellows have used the app to reflect and record

over 12,000 learning activities. The facilitation of recording

learning activities from mobile devices may contribute to the

inclusion of learning activities that would not have been

previously recorded in MAINPORT reflecting a more complete

summary of the lifelong learning of physicians over time.

Key functionality

The current core functionality as well as future-planned

functionality for MAINPORT are summarized in Table 2. The

key functional components of MAINPORT can be summarized

under three category headings:

Recording and reflection on completed activities

The ability to self-report participation in CPD activities and

reflect on the learning outcomes achieved provides evidence

of participation in learning activities and how learning

improves and contributes to competence, performance or

health outcomes. The documentation of lifelong learning

demonstrates the profession’s willingness to be transparent

and accountable for the privilege of professional self-regula-

tion, a benefit currently held by physicians practicing in

Canada. This functional component of recording and reflecting

on completed activities illustrates several common elements

among Showcase portfolios and Reflective portfolios.

MAINPORT assists reflection through the use of a common

set of questions within each documentation template, guiding

learners to record the details of their CPD activities for MOC

Program credit.

In the future, the structure of the documentation templates

will need to allow physicians to grant access to peers,

colleagues, mentors or coaches to provide feedback

(Driessen et al. 2007) and assist physicians to identify future

learning plans. Expansion of the current interoperability

standards will promote increased automation, and is expected

to even further reduce the burden of documentation for

individual physicians.

Table 2. ePortfolio functional components.

Current Future

The ability to: The ability to:

� Report on completed learning activities and reflect on learning outcomes

for practice

� Develop a practice-specific CPD learning plan

� Identify CPD goals, design and track learning activities to accomplish

stated goals

� Link CPD activities to CanMEDS roles

� Store ideas for future learning

� Manage learning activities in progress

� Access multiple search engines, online resources, and self-assessment

programs

� Personalize resources

� Access MAINPORT in either English or French

� Establish private and public layers

� Provide access to peers, mentors, coaches, and assessors for the

purposes of feedback (formative or summative)

� Expand automation strategies

� Describe professional practice domains using a ‘scope of practice’ tool

� Document cases and procedures completed in practice

� Link learning activities to scope of practice, competencies or milestones

� Develop tools to report on assessments of knowledge, competence, or

performance

� Access a calendar of events specific to one’s professional practice

� Link ePortfolio with social media and discussion forums to create online

communities of practice

� Expand the role of ePortfolio in supporting learning and assessment

across the continuum of medical education; from the beginning of

residency until retirement

ePortfolios for CPD

291

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Reg

ina

on 0

9/07

/13

For

pers

onal

use

onl

y.

Managing learning in practice

Learning in practice is stimulated by formal, informal and self-

directed activities through a process that is dynamic and

iterative. Physicians continually identify questions, issues or

concerns (Barrie et al. 1997; Ebell 1999) ‘at the point of care’

but only pursue a fraction of these identified needs. The

Holding Area of MAINPORT provides physicians with an

opportunity to store the questions they intend to pursue, the

articles that they are planning to review, the assessments they

are planning to complete etc., and revisit, revise or delete these

entries at any time in the future. In addition, the Holding Area

allows physicians to start a learning project and save the

activity for completion at a future date. Finally, MAINPORT

provides a CPD planning tool to enable physicians to reflect on

their key roles and responsibilities across multiple dimensions

of practice to identify key learning goals or initiatives. These

functions reflect some of the common elements of

Development portfolios and Showcase portfolios by enabling

learners to plan and keep track of learning activities and to

keep a record of specific goals, personal development, and

achievements relevant to their practice.

In the future, the planning and recording of learning

activities and outcomes (Moore et al. 2009) will be enhanced

by the development of a ‘scope of practice’ tool to allow

physicians to describe their professional practice and link

learning activities to specific competencies or milestones.

Identification of gaps from participation in various assessments

can then be framed into specific learning goals to create an

action plan either independently or with the help and

assistance of a mentor or coach (Wilkinson 2002).

Accessing learning resources and programs

Beyond the recording and management of learning is the

ability for MAINPORT to provide users access to search

engines and online resources. These resources support how

physicians scan their environment for new ideas or innov-

ations in development, search for evidence in response to

specific questions, and access online self-assessment programs

across multiple specialty and content domains. The ability to

centralize search engines and online resources with tools to

support the planning, management and recording of learning

activities creates the opportunity to build a learning space that

can be personalized.

In the future, accessing learning resources will become

increasingly specialty-specific and will be facilitated by the

adoption of a calendar of CME/CPD events, the population

of specialty-specific self-assessment programs and the ability

to integrate social media tools to access online communities

of practice.

Use of MAINPORT in supportinglifelong learning

By the end of 31 January 2012, which was the MOC Program

reporting deadline to record 2011 CPD activities, the

MAINPORT site (www.mainport.org) had 158,091 visits by

32,234 unique MOC Program participants. In comparison to

2010, the 2011 overall participation rates of Fellows practicing

in Canada rose from 89% to 92%. Participation of Fellows

practicing outside of North America stayed approximately the

same, moving from 59% to 54%.

There were significant changes in the percentage of

Fellows who used Section 2: Self-Learning during 2011 than

in 2010, with relatively stable participation in the other two

sections (Table 3). The total number of credits reported in

MAINPORT rose substantially from 4,506,187 in 2010 to

6,052,191 in 2011.

Further analysis will explore if the new MAINPORT has

changed usage patterns (frequency and type of reporting) and

how participants have used tools to plan, set goals, and access

online learning resources.

Discussion and final thoughts

Over the past decade, there has been a progressive cultural

shift in how the CPD of physicians has been conceptualized,

supported, and enabled. As residency education programs and

CPD systems transition towards a competency-based medical

education model, it is anticipated that ePortfolios will play an

important role in supporting physician learning and assess-

ment (Campbell et al. 2010; Harris et al. 2010; Iobst et al. 2010)

to address the changing learning needs and expectations of

physicians, regulatory authorities, and the public (Federation

of Medical Regulatory Authorities of Canada).

Table 3. Comparison of participation across the MOC program framework: 2010 and 2011.

Reporting YearSection 1:

Group learningSection 2:

Self-learningSection 3:

AssessmentTotal credits

reported

2010 97% 59% 37% 4,506,187

2011 94% 84% 30% 6,052,191

In 2011, the specialties with the highest numbers of unique Fellows reporting in MAINPORT included:

Psychiatry 14538 Obstetrics and Gynecology 4996

Anesthesia 9127 General Surgery 4848

Internal Medicine 8200 Ophthalmology 3611

Diagnostic Radiology 7794 Orthopedic Surgery 3187

Pediatrics 5523 Anatomical Pathology 2115

J. A. Gordon & C. M. Campbell

292

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Reg

ina

on 0

9/07

/13

For

pers

onal

use

onl

y.

However, a number of conceptual and practical barriers

remain to the adoption and integration of ePortfolios to

enhance their ability to support learning in practice. Further

research will need to address the following questions: What

structure or functionality of ePortfolios best supports learning

and reflection? What level of documentation provides suffi-

cient evidence of learning in practice? What is the role of a

mentor or coach in promoting ePortfolio use among residents

and physicians in practice? How can ePortfolios best support

both formative and summative assessment of achievement of

competencies, milestones or quality of care metrics? What is

the appropriate balance between facilitating reporting of

participation in learning activities with the need for physicians

to individually or collectively reflect and record meaningful

outcomes for practice? What skills are required by residents

and physicians in practice to manage their knowledge base

and plan their lifelong learning to address their professional

practice needs?

An ePortfolio which describes the learning activities

completed, the self-learning opportunities pursued, and the

areas of practice assessed and improved reflects the purpose

of CPD in demonstrating ‘progression of competence’ towards

the attainment of expertise. The future of ePortfolios in

supporting lifelong learning across the continuum of medical

education will require a greater degree of personalization, the

embedding of specialty-specific information that is pushed to

users based on their scope of practice, the provision of

continuous performance metrics, and the inclusion of data

from peers to inform the learning process and guide the

development of strategies that contribute to safe, evidence-

informed, and quality health care.

Declaration of interest: The authors report no conflicts of

interest. The authors alone are responsible for the content and

writing of the article.

Notes on contributors

Ms. JENNIFER A GORDON, MEd, is the Assistant Director of Continuing

Professional Development at the Royal College of Physicians and Surgeons

of Canada. Ms. Gordon oversees the Maintenance of Certification (MOC)

Program and the MAINPORT ePortfolio.

Dr. CRAIG M CAMPBELL FRCPC is an Associate Professor in the Faculty

of Medicine at University of Ottawa. He serves as Executive Director,

Office of Professional Affairs at the Royal College of Physicians and

Surgeons of Canada in Ottawa.

References

Austin Z, Marini A, Desroches B. 2005. Use of a learning portfolio for

continuing professional development: A study of pharmacist in Ontario

(Canada). Pharm Educ 5(3/4):175–181.

Bankey R. 2007. Technical report on lifelong learning. Ottawa: The Royal

College of Physicians and Surgeons of Canada.

Barrie AR, Ward AM. 1997. Questioning behaviour in general practice: A

pragmatic study. BMJ 215:1512–1515.

British Educational Communications and Technology Agency, JISC InfoKit.

2008. Understanding how e-portfolios work – Impact of e-portfolios on

learning (Hartnell-Young et al 2007). [Accessed 25 March 2013]

Available from http://www.jiscinfonet.ac.uk/infokits/e-portfolios/

Buckley S, Coleman J, Davison I, Khan KS, Zamora J, Malick S, Morley D,

Pollard D, Ashcroft T, Popovic C, et al. 2009. The educational effects of

portfolios on undergraduate student learning: A Best Evidence Medical

Education (BEME) systematic review. BEME Guide No. 11. Med Teach

Apr 31(4):282–298.

Campbell CM, Parboosingh JT, Gondocz ST, Babitskaya G, Lindsay E,

De Guzman RC, Klein LM. 1996. Study of Physicians’ Use of a Software

Program to Create a Portfolio of Their Self-directed Learning. Academic

Medicine 71(10):S49–S51.

Campbell CM, Silver I, Sherbino J, Cate OT, Holmboe E. 2010. Competency-

based continuing professional development. Med Teach; 32:657–662.

Davies H, Khera N, Stroobant J. 2005. Portfolios, appraisal, revalidation and

all that: A user’s guide for consultants. Arch Dis Child 90:165–170.

Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey

A. 1999. Impact of formal continuing medical education: Do

conferences, workshops, rounds and other traditional continuing

education activities change behavior or health care outcomes? JAMA

282(9):867–874.

Driessen EW, van Tartwijk J, Overeem K, Vermunt JD, van der Vleuten CP.

2005. Conditions for successful reflective use of portfolios in under-

graduate medical education. Med Educ 39(12):1230–1235.

Driessen E, van Tartwijk J, van der Vleuten C, Wass V. 2007. Portfolios in

medical education: Why do they meet with mixed success? A systematic

Review Med Educ 41:1224–1233.

duBoulay C. 2000. From CME to CPD: Getting better at getting better? BMJ

320(7332):393–394.

Ebell M. 1999. Information at the point of care: Answering clinical

questions. J Am Board Fam Pract 12:225–235.

Eva K, Regehr G. 2005. Self-assessment in the health professions: A

reformulation and research agenda. Acad Med 80:s46–s54.

Federation of Medical Regulatory Authorities of Canada. [Accessed 25

March 2013] Available from http://www.fmrac.ca/policy/revalidation_

eng.html

Finlay IG, Maughan TS, Webster DFT. 1998. A randomized controlled study

of portfolio learning in undergraduate cancer education. Med Educ

32:172–176.

Forster AJ, Clark HD, Menard A, Dupuis N, Chernish R, Chandok N. 2004.

Adverse events among medical patients after discharge from hospital.

CMAJ 170(3):345–349.

Galbraith RM, Hawkins RE, Holmboe ES. 2008. Making self-assessment

more effective. JCHEP 28(1):20–24.

Green M, Reddy SG, Holmboe ES. 2009. Teaching and evaluating point of

care learning with an internet-based clinical-question portfolio. JCEHP

29(4):209–219.

Grimshaw JM, Eccles MP, Walker AE, Thomas RE. 2002. Changing

physicians’ behaviour: What works and thoughts on getting more

things to work. JCEHP 22(4):237–243.

Grol R, Baker R, Moss F. 2002. Quality improvement research:

Understanding the science of change in health care. Qual Saf Health

Care 11(2):110–111.

Gurusamy K, Aggarwal R, Palanivelu L, Davidson BR. 2008. Systematic

review of randomized controlled trials on the effectiveness of virtual

reality training for laparoscopic surgery. Br J Surg 95:1088–1097.

Hall P, Byszewski A, Sutherland S, Stodel EJ. 2012. Developing a

sustainable electronic portfolio (ePortfolio) program that fosters

reflective practice and incorporates CanMEDS competencies into the

undergraduate medical curriculum. Acad Med 87(6):744–751.

Harris P, Snell L, Talbot M, Harden RM. 2010. Competency-based medical

education: Implications for undergraduate programs. Med Teach

32:646–650.

InfoPOEMs. 2012. accessed through the Canadian Medical Association at

http://www.cma.ca/clinicalresources/k4p

Iobst WF, Sherbino J, Cate OT, Richardson DL, Dath D, Swing SR, Harris P,

Mungroo R, Holmboe ES, Frank JR. 2010. Competency-based medical

education in postgraduate medical education. Med Teach 32:651–656.

Institute of Medicine Report. 1999. To Err is Human: Building a

Safer Health System. [Accessed 25 March 2013] Available from

http://iom.edu/Reports/1999/To-Err-is-Human-Building-A-Safer-Health-

System.aspx

Kardos RL, Cook JM, Butson RJ, Kardos TB. 2009. The development of an

ePortfolio for life-long reflective learning and auditable professional

certification. Eur J Den Educ 13(3):135–141.

ePortfolios for CPD

293

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Reg

ina

on 0

9/07

/13

For

pers

onal

use

onl

y.

Mann K, Gordon J, MacLeod A. 2007. Reflection and reflective practice in

health professions education: A systematic review. Adv Health Sci Educ

Theory Pract 14(4):595–621.

McGaghie MC, Issenberg AB, Petrusa ER, Scalese RJ. 2006. Effect of practice

on standardized learning outcomes in simulation-based medical

education. Med Educ 40:792–797.

McGlynn EA, Asch SM, Adams J, Keesey J, Hicks J, DeCristofaro A, Kerr EA.

2003. The quality of health care delivered to adults in the United States.

NEJM 348:2635–2645.

MedBiquitous. 2012. [Accessed 25 March 2013] Available from http://

www.medbiq.org/, and standards web page at http://www.medbi-

q.org/std_specs/standards/index.html#ActivityReport

Moon J. 2004. A handbook of reflective and experiential learning: Theory

and practice. London: Routledge-Falmer.

Moore DE, Green JS, Gallis HA. 2009. Achieved desired results and

improved outcomes: Integrating planning and assessment throughout

learning activities. JCEHP 29:1–15.

Overeem K, Faber M, Arah O, Elwyn G, Lombarts K, Wollersheim H, Grol R.

2007. Doctor performance assessment in daily practice: Does it help

doctors or not? A systematic review. Med Educ 41:1039–1049.

Parboosingh JT. 2002. Physician communities of practice: Where learning

and practice are inseparable. JCEHP 22(4):230–236.

Ross S, MacLachlan A, Cleland J. 2009. Students’ attitudes towards the

introduction of a Personal and Professional Development portfolio:

Potential barriers and facilitators. BMC Med Educ 9:69. doi:10.1186/

1472-6920-9-69.

Royal College of Physicians and Surgeons of Canada. [Accessed 25 March

2013] Available from www.royalcollege.ca

Scholes J, Webb C, Gray M, Endacott R, Miller C, Jasper M, McMullan M.

2004. Making portfolios work in practice. J Adv Nurs 46(6):595–603.

Schon D. 1987. Educating the reflective practitioner. San Francisco. Jossey-

Bass.

Sutherland S, Powell A. 2007. JISC InfoNet. http://www.jiscinfonet.ac.uk/

infokits/e-portfolios/; updated 14 November 2012.

Teunissen PW, Dornan T. 2008. Lifelong learning at work. BMJ

336:667–669.

Tochel C, Haig A, Hesketh A, Cadzow A, Beggs K, Colthart I, Peacock H.

2009. The effectiveness of portfolios for post-graduate assessment and

education: BEME Guide No. 12. Med Teach 31:299–318.

Tompkins M, Paquette-Frenette D. 2010. Learning portfolio models in

health regulatory colleges in Ontario, Canada. JCEHP 30(1):57–64.

van der Veer SN, de Keizer NF, Ravelli ACJ, Tenkink S, Jager KJ. 2010.

Improving quality of care. A systematic review of how medical

registries provide information feedback to health care providers. Int J

Med Inform 79(5):305–323.

van Tartwijk J, Driessen E. 2004. ePortfolio scenarios: Types of

ePortfolios. [Accessed 25 March 2013] Available from http://insight.

eun.org/ww/en/pub/insight/school_innovation/eportfolio_scenarios/

portfolios_types.htm

Wassef ME, Riza L, Maciag T, Worden C, Delaney A. 2012. Implementing a

competency-based electronic portfolio in a graduate nursing program.

Comput Inform Nurs 30(5):249–250.

Wilkinson TJ, Challis M, Hobma SO, Newble DI, Parboosingh JT, Sibbald

RG, Wakeford R. 2002. The use of portfolios for assessment of the

competence and performance of doctors in practice. Med Educ

36(10):918–924.

J. A. Gordon & C. M. Campbell

294

Med

Tea

ch D

ownl

oade

d fr

om in

form

ahea

lthca

re.c

om b

y U

nive

rsity

of

Reg

ina

on 0

9/07

/13

For

pers

onal

use

onl

y.