E-assessment and e-portfolios

Bernie Zakary

What comes to mind?

Coursework Driving test

FoldersKey skills

NVQsMultiple choice

Shading in boxes (OCR)

amazon.co.ukFirst Direct

So what is happening, and why?

Key skills (test component)Drivers:• External assessment to ‘give validity’ to the

judgements based on the portfolio• Manageability: thousands of candidatesSolution:• MCQs and OCR (optical character

recognition)• “computer marked”

Moving on with the key skills tests

Drivers:• When ready testing, in work based contexts• Near-immediate feedback• ICT as a Skill for Life• Growing pressure for e-assessment

Solution:• On-screen pilots (3000+ candidates)• Use of ‘test paper banks’

E-Portfolios - NVQs

Drivers:• Competence based assessment• Varied forms of evidence• Cross-referencing: the same piece of evidence

meeting more than one need

Solution:“OCR is committed to the use of e-portfolios and is keen

to support their use across a variety of NVQs” (http://www.ocr.org.uk/OCR/WebSite/Data/Publication/E-Assessment%20Materials/Getting_st13989.pdf)

Guidance to NVQ centres on e-portfolios

“We are happy for these users to continue in whateverformat is appropriate and convenient for the centre and its candidates,providing they can be verified by our external verifier team.”• ACCESSIBILITY • CROSS REFERENCING• CANDIDATE PROGRESS • AUDIT TRAIL• SECURITY • USER FRIENDLY It is essential that the e-portfolio product is able to accommodate a range of file types. Audio and video evidence provide a 'real life' assessment opportunity. Digital pictures and scanned images as well as the standard Word or Excel files should also be accommodated giving candidates many more opportunities to provide and present evidence. The product must be 'fit for purpose'.

The amazon effect

• Personalised web sites/services– Products for you– Recommendations– Prompts you to re-sell what you have bought– Text alerts– Email alerts– 24/7

E-portfolios: the policy framework

• Towards a unified e-learning strategy (DfES 2004)– Individualised learning joined up across different sectors, organisations and learning

environments, achieved through the universal provision of an e-portfolio which will allow both summative assessment and information about personal aspirations and interests to be owned by the learner

• QCA’s Blueprint for e-assessment (QCA 2004)– Within 5 years all awarding bodies should be set up to accept and assess e-portfolios

• 14-19 White Paper Education and Skills (DfES 2005)– “support and promote the Europass which provides a standardised portfolio of

documents for learners to use to record their qualifications and competences…”– In vocational qualifications, to minimise pressure (assessment burden)

• Higher Education in the Learning Society (NCIHE 1997)– “A transcript recording student achievement which follow a common format…”

• Careers Wales– 5 MB e-portfolio available to every Welsh citizen to support learning and career

development

E-assessment and policy

• 14-19 White Paper education and Skills (DfES 2005)In vocational and occupational qualifications, QCA are already seeking ways to reduce the burden of

assessment. In particular, they are exploring a range of different forms of moderation of practical assessments, including the use of e-portfolios, which should minimise pressure on students, teachers and trainers.

And in the medium term:– Reduce assessment burden– When ready assessment– Faster, more accurate feedback– Aspirational: on line

• E-strategy (DfES 2005)It is crucial that we fully examine the potential for technology to modernise the curriculum and its assessment. ICT supports assessment playing a more formative role – assessment for learning, not just for judging. And we want to see pilots of a wide range of applications of e-assessment: immediate feedback to learners and teachers, online tests, personalised diagnostics, online assessment and accreditation of e-skills, and inclusion of e-skills in the assessment of all curriculum topics.

Some motives for using e-assessment are:

• avoiding meltdown of paper-based assessment

• assessing valuable life skills

• aligning the curriculum and assessment

• providing on-demand testing

• accommodating student progress at different rates

• provision of adaptive testing

• better immediate feedback

• motivational gains

• better exemplification for students and teachers

• better ‘system’ feedback (performance of test items)

• faster information for higher education

• lower costs in some areas

E-portfolios: examples of content (from FD Learning)Coursework Ranging from brief notes to extensive assignments. May

be in any medium, for example, text, images, sound, video.

May be school homework examples, college assignments, for example.

Assessment work May include diagnostic, formative and summative

assessments.

Other pieces of work or “artefacts” For example, presentations, job or course applications, CVs.

Achievement of individual learning outcomes

May be formally or informally recorded.

Aggregated records of achievement, accreditation and credit towards awards.

Qualifications, awards (and credits towards awards),

certificates, completion of courses.

Evidence for assessment Including evidence for assessment of prior learning.Planning and reflection Journal entries, learning agreements, personaldevelopment plans, individual learning plans.Notes and annotations on other entries Formal or informal: made by the learner, or by teachers, mentors. Including verification of entries.Skills and competencies Taken from to a particular framework, for example, for a job, or informally recorded.Outcomes of appraisals, interviews, etc. With tutor, employer, for example.Self-assessments and appraisals.Peer-assessments and appraisalsLinks between entries Pieces of work contributing to an award: planning toachieve particular skills, etc.Entries shared with peers e-Portfolios can support peer group learning, withshared assignments, and commentary on each other’s

E-portfolios in other contexts-lifelong learning

• Dr Helen Barrett, University of Alaska– Creative writing

– Lifelong learning

• Uses Adobe Acrobat– “My hard drive is really my working portfolio. Once a year I “mine”

my hard drive for those “gems” that I want to include in my artifacts files”

• Adding “reflections”– “Reflections turn artifacts into evidence of achievement”

Artifacts Competencies Demonstrated -->

Electronic Portfolio

Knowledge

, Skills and

Experience

Digital

Storytelling (Digital

Video

Editing

Skills)

Multimedia

& Web Page Authoring

(Technolog

y Skills)

Presentation,

Instructional

Design &

Teaching Skills

Assessme

nt Knowledge

Writing Skill

s

http://electronicportfolios.orgMy Website on Electronic Portfolios X   x      

http://electronicportfolios.org/blog/My "Blogger" blog X         x

http://homepage.mac.com/eportfolios/iMovieTheater11.html

Choices - a digital story of learning (.mov)

  X        

http://homepage.mac.com/eportfolios/iMovieTheater13.html

Electronic Portfolios as Digital Stories of Learning (.mov)

x X        

http://homepage.mac.com/eportfolios/iMovieTheater4.html

Preparing for a Digital Storytelling Workshop (.mov)

  X        http://electronicportfolios.org/teachers/profdev.html

Professional Development for Implementing Electronic Portfolios

x     X x x

http://electronicportfolios.org/digistory/epstory.html

Electronic Portfolios as Digital Stories of Learning

x x     X X

http://electronicportfolios.org/systems/paradigms.html

Conflicting Paradigms in Electronic Portfolio Approaches

x       X X

http://electronicportfolios.org/systems/concerns.html

Differentiating ePortfolios and Assessment Management Systems

x       X X

http://electronicportfolios.org/portfolios/sitepaper2001.html

Using Adobe Acrobat for Electronic Portfolio Development

    X      

Dr. Helen Barrett

http://electronicportfolios.org

Creating your own e-portfolio system…(for an institution)

• Can buy systems– www.envq.co.uk– www.paperfree.co.uk – www.profile.uk.com/professional.html/

• Go open source– www.theospi.org (University of Minnesota)– Requires a webserver (Apache, Microsoft IIS)– Requires a servlet engine (Tomcat)– Requires Java and a relational database (mysql)

(Requires someone who knows what they are doing )

Creating your own e-assessment resources• Questionmark Perception is a standards based assessment

development and administration tool and can be used to deliver assessment over the web

– over 15 question authoring tools, including drag-and-drop, multiple choice and response, text response, numeric questions, support for Macromedia Flash based activities, matching and ranking exercises

• Hot Potatoes™– can be used to create six types of interactive assessments: multiple choices, short text

answers, jumbled sentences, crosswords, matching and fill in the gaps

– available free to non-profit educational organisations that are willing to publish their material on the internet for public access

• E-assessment by Shakeel Suleman, www.ferl.becta.org.uk

Priority for Becta:

• Working with the JISC to deliver the e-strategy

• Advisory role– To strategic partners– To communities of practice

• ‘Insight’ role– The use of ICT in formative assessment

Thank you.

Bernie Zakary

Head of pedagogical frameworks

British Educational Communications and Technology Agency

bernie.zakary@becta.org.uk