E Portfolios: Are they useful in occupational therapy ...€¦ · While e-portfolios have been used in health education (Buckley et al., 2009), there is no reported use of an e-portfolio

Page | 1

Isbel S, Wicks A and Nuessler S. MedEdPublish 2014, 3: 41

http://dx.doi.org/10.15694/mep.2014.003.0041

E-Portfolios: Are they useful in occupational therapy education?

Stephen Isbel, Alison Wicks and Shane Nuessler

Received: 05/11/2014

Accepted: 24/11/2014

Published: 31/12/2014

Abstract

Context: E-portfolios can be a useful teaching, learning and professional tool. This study aimed to determine

if an e-portfolio (Mahara) could be use to map occupational therapy student competency as they moved

through a graduate entry course. The study also aimed to determine whether an e-portfolio was a useful tool

in developing reflective practice and to report on student perceptions when using an e-portfolio.

Methods: A longitudinal, mixed methods design was used. Questionnaires were given to all first year

students (n=16) in the first semester of study and then again in the final semester of study (n=9).

Results: Students reported that using an e-portfolio is a potentially useful teaching, learning and professional

tool, particularly as a tool to develop reflective practice. However, students preferred existing methods of

recording and storing this information and did not engage with the e-portfolio if the associated tasks were not

assessable. Students were more engaged with an e-portfolio when using it as a tool during practice education

to develop critical reflection skills. The main barrier to using an e-portfolio was the perceived ease of use of

the technology.

Discussion: In this study, an e-portfolio was not successful in mapping student competency, but was useful as

a tool for reflective practice. An e-portfolio can be useful in occupational therapy education, however teachers

should carefully consider the purpose of the portfolio, the associated tasks and the ease of use of the

technology.

Keywords: Assessment portfolios, outcome based curriculum, reflection and elearning portfolios.

Article

Introduction

An e-portfolio is a digitized collection of authentic and diverse evidence drawn from a larger archive

representing what a person has learned over time (Lewis and Baker, 2007). The use, application and

effectiveness of e-portfolios are well reported in the literature (Bashook, Gelula, Joshi & Sandlow, 2008;

Knight, Hakel, Gromko, 2008; Anderson, Gardner, Ramsbotham & Tones, 2009; Mason, Pegler & Weller,

Article Open Access

Corresponding author: Stephen Isbel [email protected]

Department: Faculty of Health, University of Canberra ACT 2601, Australia

Page | 2



2004). E-portfolios are reported as being useful tools for students for developmental, presentation and

assessment purposes (Mason, Pegler and Weller, 2004; Beetham, 2005).

In 2011, the University of Canberra commenced a graduate entry-level occupational therapy course. The

primary goal of this program is to produce graduates who meet the Australian Minimum Competency

Standards for New Graduate Occupational Therapists (OTA, 2010). The literature reports e-portfolios can be

used as a means of mapping and assessing student competencies (Ring and Ramirez, 2012). And it has been

found that e-portfolios can also be useful in the development of reflective practice (Parkes, Dredger & Hicks,

2013), which is a critical component of occupational therapy practice.

The literature also reports that asking students to adopt an e-portfolio can be challenging if the purpose is not

clear, if the relevance is not explained or if the instructions are not clear (Gaitan, 2012; Ring & Ramirez,

2012). With such information in mind the University of Canberra conducted a small pilot study using an e-

portfolio to map student competencies and as a tool to develop reflective practice. The participants were one

cohort of occupational therapy students at the University of Canberra.

While e-portfolios have been used in health education (Buckley et al., 2009), there is no reported use of an e-

portfolio used specifically in occupational therapy for the purposes of competency mapping and the

development of reflective practice. This study reports how an e-portfolio (Mahara) was used to map

occupational therapy competencies and as a reflective practice tool in a small cohort of occupational therapy

students. It also reports the students’ experience when using an e-portfolio and uses the Technology

Acceptance Model (Venkatesh and Davis 1996) to explain some of the findings.

There were three aims of the study. These were to determine the feasibility of using Mahara:

i) for mapping occupational therapy student competencies;

ii) as a tool for reflective practice and;

iii) to describe students’ experiences of using an e-portfolio

Methods

This study, approved by the University of Canberra Human Research and Ethics Committee, adopted a

longitudinal, mixed methods design. All first year students studying the Masters of Occupational Therapy at

the University of Canberra (n=16) consented to be involved in the study. They attended an initial workshop

and then completed a ten minute questionnaire at the end of their first semester and then completed the

questionnaire again in their final year of study (n=9). Only full time students completed the final questionnaire

as they were presented with critical information and tasks during their final practice education unit in which

part time students were not enrolled.

Workshop 1 for instructions in the use of Mahara and for task 1: Recording competencies

The first part involved instruction in the use of Mahara. Students were required to attend a workshop at the

end of the first teaching semester in first year where Mahara was explained to them. The guidelines of Moores

and Park (2010) for introducing e-portfolios were followed. The elements of these instructions are as follows:

1. Identification of the added value of using an e-portfolio

At the end of the first semester and prior to the first two-week practice education placement, students were

introduced to Mahara in a workshop setting. During this workshop the purpose and potential benefits of

developing an e-portfolio were explained to students. These benefits included the ability to electronically store

information that could demonstrate the OTA (2010) competencies. Benefits of storing professional artifacts

for job interviews and sharing information with their peers were also highlighted.

2. Consideration of the long and short-term use of an e-portfolio

Mahara is the e-portfolio that has been purchased by the University of Canberra. Students were asked to do

two specific tasks (mapping competencies initially and then undertaking critical reflection tasks) so that the

Page | 3



students could see the short-term application of the portfolio. Students were also shown an exemplar portfolio

showing how historical information could be stored in Mahara, demonstrating the long-term use of an e-

portfolio.

3. Having assessment guidelines that are transparent but not too prescriptive

There was no formal assessment attached to first task (mapping competencies). It was hoped that the students

would begin populating their competency pages (see Figure 1) without attaching a mark to this process. The

critical reflection pieces were marked on a pass/fail basis, meaning the students needed to submit their

reflections to pass that item of assessment.

4. Providing students with clear guidance on confidentiality and the use of digital media

Students were given clear instructions on posting material on Mahara. Specific instructions on de-identifying

people and situations were given with examples of appropriate inappropriate postings. Mahara was also set up

so only the members of the class were able to see postings and students were able to keep postings private if

they wished.

5. Emphasising that e-portfolios do not teach reflective practice

An e-portfolio is a tool that can be used to help students reframe and analyse experiences (Kjaer, Maagaard &

Wied, 2006). An e-portfolio is a way of recording experiences in a specific way that can be explored in depth

during face-to-face teaching. It is for this reason that the methodology in this study included a workshop that

specifically examined reflective practice.

6. Ensuring that students can access their e-portfolio

All students were shown how to set up their Mahara account in a face-face classroom setting. Exercises in

using the pages and groups were conducted and all students had access to a computer that could access their

Mahara site when away from University.

7. Making use of internal support available within the institution

The Teaching and Learning Centre (TLC) at the University of Canberra was critical in the set up and

implementation of Mahara. Staff from TLC ran face-face sessions to show the students how to set up Mahara

and provided technical and teaching support to the academics who conducted the subjects that used Mahara.

Instructions to students also included information on the competency domains contained in OTA (2010).

Refer to Figure 1. Each domain has associated performance criteria. Students were instructed to use these

criteria when classifying experiences or assessment items into each domain. At the end of the workshop

students were given examples of how to relate an assessment task or experience to the domains. Students were

also asked to populate these pages with de-identified experiences they would have on their upcoming two-

week practice education placement. They were also asked to populate these pages with items of assessment

they had completed during their first semester of study. These tasks were not assessable.

Questionnaire 1

Four months after the initial Mahara workshop and after the first practice education placement, students

completed a questionnaire to determine their use and perceptions of Mahara. The questionnaire contained

open and closed ended questions with four Likert scale rated statements, which were based upon those used

by Gerbic, Lewis and Northover (2009). The statements were:

1. Mahara is providing a place to store examples of coursework.

2. Mahara is allowing me to store examples of my extra-curricular activities relevant to my future career.

Page | 4



3. Mahara can help me organise material and experiences in relation to the Australian Minimum

Competency Standards for New Graduate Occupational Therapists (OTA, 2010).

4. Mahara is allowing me to keep track of learning experiences and to be able to reflect on any weak

areas.

Workshop 2 for additional instructions in use of Mahara

Prior to the next practice education placement of eight weeks, the students were again asked to attend a

Mahara workshop as part of a pre-placement workshop. During this workshop students were introduced to the

concept and importance of reflective practice. Students were shown an example of how practice education

experience could be reported in Mahara by using a format developed by the Victorian Government (2007).

See Figure 2.

Students were asked to post on their Mahara portfolio, one critical reflection each week of their eight-week

placement. Their peers, in a closed group on Mahara could see these postings. Students were also asked to

comment on at least one posting of a peer per week for eight weeks. Both the personal critical reflections and

the peer postings were assessable (pass/fail). This process was repeated for the following seven-week practice

education block in the final year of study.

It was clear from results from questionnaire one, that a more prescriptive approach to using Mahara was

required (see Tables 1 and 2). Most students reported problems with duplication of information, relevance and

complexity. Following this feedback it was decided to limit the scope of what was required of the students but

to make the content assessable.

Questionnaire 2

In the final semester of study, students were asked to repeat questionnaire one.

Figure 1: Mahara pages view of OTA (2010) competency domains

Page | 5



Figure 2: Critical reflection template on Mahara

Results

This study was designed to evaluate the feasibility of giving students an electronic platform to store any

relevant educational or professional material relating to the Australian Minimum Competency Standards for

New Graduate Occupational Therapists (OTA 2010). Students were given the framework in which this could

work in terms of setting up electronic pages titled the same as the practice domains in OTA (2010) and asked

to populate these pages. Only three out of sixteen students used Mahara as instructed for this non-assessable

task. See Table 1.

Although poor engagement with Mahara was identified, the students reported that they could appreciate

potential uses for Mahara, yet they did not use it. For example 69% of students agreed or strongly agreed that

Mahara provided a place to store examples of coursework. Sixty nine percent also agreed or strongly agreed

that Mahara organized materials and experiences to OTA (2010). See Figure 7.

There are two main reasons why students did not use Mahara for the first task (competency mapping) despite

thinking it was potentially beneficial. First, students reported difficultly navigating around the pages,

uploading material in the correct area and generally finding the process cumbersome. Second, students

reported that this process was duplicating what they already had in a paper-based format or stored

electronically elsewhere. Therefore, the methodology used in this study did not demonstrate that Mahara

could be used to map occupational therapy competencies.

Following the results from the first stage of the study, various technical and pedagogical issues were

addressed in preparation for task 2 (critical reflection). From a pedagogical perspective it was decided that

students’ submission of their critical reflections to Mahara needed to be assessable (pass/fail). Teaching

sessions on critical reflection and clinical reasoning were added to pre and post practice education workshops.

From a technical perspective, changes were made to reduce the complexity of navigating Mahara. For the

second task, all students were required to do was upload a critical reflection form onto their Mahara portfolio.

Page | 6



Previously, students had to decide where a particular event or experience fitted with the seven OTA (2010)

domains and upload the material.

Following these changes it was not surprising then to see that 66% of students agreed or strongly that Mahara

was able to keep track of learning and to reflect on weak areas compared to 43% previously. See Figure 5.

Writing a critical reflection each week on practice education, commenting on peer reflections and receiving

peer comments were seen by the students as a valuable learning experience. Therefore this study showed that

it is feasible to use Mahara as a tool for reflective practice.

A similar result was seen when students were asked if Mahara provided a place to store examples of

coursework. Initially 69% agreed or strongly agreed with this statement with 89% agreeing or strongly

agreeing following the changes. See Figure 3. However, when students were asked about Mahara being useful

for organising data for future employment or storing extra curricular materials and resources, a decrease in the

number of agreed or strongly agreed ratings was noted. See Figure 6. Students also reported ongoing technical

issues around cumbersome navigation, uploading files and site layout.

A selection of answers is presented in Tables 1-3 and Figures 3-7.

Table 1

Approximately many times have you used Mahara since the initial workshop?

T1*

N=16

T2**

N=9

20 plus 1

20-25 1

15-20 5

10-15 1

5-10

0-5 3

0 13

*T1 refers to the first questionnaire after the first semester of study

**T2 refers to the final questionnaire in the final semester of study

Table 2

Have you used Mahara for anything else except the assessment tasks required for practice

education? N=9

No Yes

8 1

Page | 7



Table 3

What are 3 good things about using Mahara?

T1 n=16 T2 n=9

Central storage

Good place to store information from

practice education

Being able to see others’ e-portfolios

Being able to use it as a portfolio for future

employers

Professional looking

Easy to use and add to. Accessible

Many functions

Good for networking

Good for keeping competencies together in

one place

Able to easily reflect, document experiences

and show to others

Would be useful for lecturers to have

students responses in one place for the

purpose of competencies

Good for reflection

Mapping learning

Easy to reflect back

Good to see others learning experience

Communication with other students

Seeing what other students are doing on prac

Feeling supported by fellow students when

reflecting own experience

I can easily store client examples

Seeing other people’s experiences

Being able to communicate through it

Being able to re-read your reflections and

review where you have come from

It enabled us to reflect critically and share

our reflections and comment on others

It was fairly easy to use once you got used to

it.

Online access from anywhere

Information kept all in one place

Can access information from any location

Page | 8



Table 4

What are 3 bad things about using Mahara

T1 n=16 T2 n=9

It is a lot of work compared to doing it my

own way. (U)*

I found it somewhat confusing when learning

how to use it. (E)

Very complex (E)

What is the relevance to practice? (U)

Seems a lot of effort (E)

At first introduction it seemed complex (E)

Won’t use it after graduation (U)

Can’t see the relevance of it to the course in

general or getting a job (U)

I feel like Mahara was a lot of unnecessary

work (E)

It is hard to learn (E)

Rather use written information (U)

It is confusing to learn (E)

A bit complicated (E)

Confusing to set up (E)

Takes a while to learn how to use (E)

Could be more user friendly/intuitive (E)

Difficult to use without familiarity (E)

Difficult to set up at first (E)

The layout navigation need to be more simple

(E)

Adding content needs to be easier. (E)

Not logically set out (E)

Not something I would use by choice (U)

Not something I will use after graduation (U)

* Each negative comment has been categorised into the two factors of the Technology

Acceptance Model that contribute to the attitude towards using Mahara. That is: perceived

usefulness (U) and perceived ease of use (E)

Figure 3

0

50

100

T1 T2 T1 T2 T1 T2

Strongly Agree or Agree Neither Agree orDisagree

Disagree or StronglyDisagree

%

o

f

s

t

u

d

e

n

t

s

Mahara is providing a place to store examples of coursework

Page | 9



Figure 4

Figure 5

0

50

100

T1 T2 T1 T2 T1 T2

Strongly Agree orAgree

Neither Agree orDisagree


%

o

f

s

t

u

d

e

n

t

s

Mahara allowed me to store relevant extra-curricular material

0

50

100

T1 T2 T1 T2 T1 T2




%

o

f

s

t

u

d

e

n

t

s

Mahara allowed me to keep track of learning experiences and to reflect on weak areas

Page | 10



Figure 6

Figure 7

Discussion

Students are more likely to use an e-portfolio if the purpose is clear, issues around disclosure of information

are clarified, technical issues do not impede engagement and tutors provide adequate guidance (Gaitan, 2012).

In this study, technical issues impeding the use of Mahara (Table 4) was a common theme. This can be further

explored by exploring these results in relation to the Technology Acceptance Model (Venkatesh and Davis

1996)

0

50

100

T1 T2 T1 T2 T1 T2




%

o

f

s

t

u

d

e

n

t

s

Mahara helped me organize my work for future employment

0

50

100

T1 T2 T1 T2 T1 T2




%

o

f

s

t

u

d

e

n

t

s

Mahara has organised material/experiences in relation to OTA (2010)

Page | 11



The Technology Acceptance Model

The technology acceptance model (TAM) has been used extensively to explain why new technologies are or

are not used. The TAM was first developed by Davis, Bagozzi and Warshaw (1989) and then refined by

Venkatesh and Davis (1996). The TAM is based upon the theory of Reasoned Action (Fishbein and Ajzen,

1975) and proposes that the use of new systems (such an e-portfolio) is dependent upon a mixture of the

motivation of the user combined with the features of the technology. Refer to Figure 8. The TAM has been

applied extensively in a variety of settings including information technology, banking, health and education

(Chuttur, 2009). It has been tested empirically and found to be a valid theoretical model in relation to e-

portfolio use (Shroff, Deneen and Ng, 2011). In this model the end use of the technology is dependent on the

attitude towards the use (A) of the technology, which is influenced by the user’s perceived usefulness (U) of

the technology and the perceived ease of use (E) of the technology.

Figure 8: Technology Acceptance Model (Venkatesh and Davis 1996)

Results and the technology acceptance model

The results from this study can be applied to the technology acceptance model. This is useful to identify the

factors that contributed to a reluctance to use Mahara for anything other than an assessment item. It was hoped

that students would use Mahara in other ways such as developing a professional portfolio and storing relevant

course material but this did not happen. To highlight the factors within the TAM that prevented students from

fully engaging in Mahara, the negative qualitative responses in relation to using Mahara have been categorised

into the components of the TAM (Table 4). The negative comments relating to the use of Mahara were mainly

about the Perceived Ease of Use (E). Of the twenty-three negative comments, seventeen were classified as

issues around Perceived Ease of Use (E)

The students could see the uses (U) of Mahara in critical reflection and in some aspects of information storage

and organisation. However, the students found Mahara difficult to use (E) and this had a negative effect on

their attitude toward using it and therefore their actual use of Mahara outside the required assessable tasks.

Conclusion

E-portfolios can be useful in teaching, learning and professional development. The methodology used in this

research showed it was not feasible to use Mahara to map student competencies. However, the study did show

it was feasible to use an e-portfolio as a tool for reflective practice. Although students could see the potential

benefits of using an e-portfolio, issues related to the task that was set for the competency mapping, and more

Page | 12



importantly, the difficulty students encountered when using e the e-portfolio, were the main factors limiting

student engagement with their e-portfolios.

This study showed that being specific in the instructions given to students and linking e-portfolio activities to

assessment improved engagement. The use of e-portfolios in occupational therapy education is worth

pursuing.. Clearly, there is potential for an e-portfolio to be a teaching, learning and professional tool. The

challenge for educators is to provide a teaching and learning experience that engages students to recognise the

potential of e-portfolios. When designing teaching and learning experiences that involve e-portfolios, it is

important to ensure students can easily use the e-portfolio.

The Authors

Stephen Isbel 1 HScD MOT; MHA B.App Sc (OT)

Alison Wicks 2 PhD, MHSc(OT), B.AppSc (OT)

Shane Nuessler 3 MHE, B.IT

1Corresponding author

Faculty of Health

University of Canberra ACT 2601

Australia

2 Faculty of Health


Australia

3 Teaching and Learning Centre


Australia

Page | 13



References

Anderson D, Gardner G, Ramsbotham J, & Tones M. 2009. E-portfolios: developing nurse

practitioner competence and capability. Australian Journal of Advanced Nursing 26(4):70-76).

Bashook P, Gelula M, Joshi M, & Sandlow L. 2008. Impact of student reflective e-portfolio on

medical student advisors. Teaching and Learning in Medicine 20(1):26-30.

http://dx.doi.org/10.1080/10401330701798113

http://dx.doi.org/10.1080/10401330701798113

Beetham H. 2005. Eportfolios in post-16 learning in the UK: Developments, issues and

opportunities. Accessed 28 March 2013 from www.jisc.ac.uk

Buckley S, Coleman J, Davison I, Khan K, Zamora J, Malick S, Morley D, Pollard D, Ashcroft T,

Popovic C, & Sayers J. 2009. The educational effects portfolios on undergraduate student learning: a

best evidence medical education (BEME) systematic review. Medical Teacher 31(4):282-298.

http://dx.doi.org/10.1080/01421590902889897

http://dx.doi.org/10.1080/01421590902889897

Chuttur M. 2009. Overview of the technology acceptance model: origins, developments and future

directions. Sprouts: working papers on information systems 9(37). Retrieved 13 June, 2014.

Available from http://sprouts.aisnet.org/9-37

Davis F, Bagozzi RP, & Warshaw P. 1989. User acceptance of computer technology: a comparison

of two theoretical models. Management Science 35(8):982-1003.

http://dx.doi.org/10.1287/mnsc.35.8.982


Fishbein M, & Ajzen I. 1975. Belief, attitude, intention and behavior: an introduction to theory and

research. Reading, MA: Addison-Wesley.

Gaitan A. 2012. Understanding students' experience of e-PDP and the factors that shape their

attitudes. International Journal of ePortfolio 2(1):29-38.

Gerbic P, Lewis L, & Northover M. 2009. Student perspectives of eportfolios: a longitudinal study

of growth and development. Auckland: Ascilite Proceedings.

Kjaer N, Maagaard R, & Wied S. 2006. Using an online portfolio in postgraduate training. Medical

Teacher 28(8):708-712. http://dx.doi.org/10.1080/01421590601047672

http://dx.doi.org/10.1080/01421590601047672

Knight W, Hakel M, & Gromko M. 2008. The relationship between electronic portfolio participation

and student success. Professional File, 107.

http://dx.doi.org/10.1080/10401330701798113

http://dx.doi.org/10.1080/01421590902889897


http://dx.doi.org/10.1080/01421590601047672

Page | 14



Lewis K, & Baker R. 2007. The development of an electronic educational portfolio: an outline for

medical education professionals. Teaching and Learning in Medicine 19(2):139-147.

http://dx.doi.org/10.1080/10401330701332219

Mason R, Pegler C, & Weller M. 2004. E-portfolios: an assessment tool for online courses. British

Journal of Educational Technology 35(6):717-727. http://dx.doi.org/10.1111/j.1467-

8535.2004.00429.x

Moores A, & Parks M. 2010. Twelve tips for introducing e-portfolios with undergraduate students.

Medical Teacher 32:46-49. http://dx.doi.org/10.3109/01421590903434151

Parkes K. Dredger K, Hicks D. (2013) ePortfolio as a measure of reflective practice. International

Journal of ePortfolio 3(2):99-115.

Ring G. & Ramirez B. 2012. Implementing eportfolios for the assessment of general education

competencies. International Journal of eportfolio 2(1):87-97.

Shroff R, Deneen C, & Ng E. 2011. Analysis of the technology acceptance model in examining

students’ behavioural intention to use an e-portfolio system. Australasian Journal of Educational

Technology 27(4):600-618.

Occupational Therapy Australia (OTA) 2010. Australian minimum competency standards for new

graduate occupational therapists. Retrieved 27 March, 2014. Available from

www.otaus.com.au/about/entry-level-program-accreditation

Venkatesh V. & Davis FD. 1996. A model of the antecedents of perceived ease of use: development

and test. Decision Sciences 27(3):451-481. http://dx.doi.org/10.1111/j.1540-5915.1996.tb01822.x

Victorian Government (2007). Critical reflection framework information sheet. Retrieved 27

March, 2013. Available from

http://www.education.vic.gov.au/childhood/professionals/support/Pages/reflectframework.aspx

http://dx.doi.org/10.1080/10401330701332219

http://dx.doi.org/10.1111/j.1467-8535.2004.00429.x

http://dx.doi.org/10.1111/j.1467-8535.2004.00429.x

http://dx.doi.org/10.3109/01421590903434151

http://www.otaus.com.au/about/entry-level-program-accreditation

http://dx.doi.org/10.1111/j.1540-5915.1996.tb01822.x

http://www.education.vic.gov.au/childhood/professionals/support/Pages/reflectframework.aspx

Documents

E Portfolios: Are they useful in occupational therapy ...€¦ · While e-portfolios have been used in health education (Buckley et al., 2009), there is no reported use of an e-portfolio