Macquarie 23 Sep 2016 - Transforming...

Dr Mathew Hillier Monash University

OLT e-Exams: Ramping up the authenticity of high-stakes supervised assessment for the 21st century

Presentation for Macquarie 23 Sep 2016

TransformingExams.com Acknowledgement Office of Learning and Teaching

TransformingExamsAcrossAustraliaAustralianGovernmentOfficeforLearningandTeachingNa:onalGrantID15-4747:AU$500Kover3Years+100KMonash

Acknowledgements-DiamondandGoldSponsors!500KinsupportforthisprojecthasbeenprovidedbytheAustralianGovernmentOfficeforLearningandTeaching.TheviewsexpresseddonotnecessarilyreflecttheviewsoftheAustralianGovernmentOfficeforLearningandTeachingorpar:cipa:ngins:tu:ons.100Kaddi:onalcashfromMonashUniversity.

Demo&Guide

s

Dr Mathew Hillier; Leader OLT national project. Seed leader

Monash University (was University of Queensland)

Dr Andrew Fluck; Originator of USB e-exam concept. Seed partner.

University of Tasmania

Dr Michael Cowling, Mr Kenneth Howah Seed trial site.

Central Queensland University

Dr Kim Blackmore Australian National University Assoc. Prof. Paul Newhouse Edith Cowan University Dr Matthew Bower, Prof. Dominic Verity Macquarie University Prof. Marilyn Baird (H), Mr Scott Grant (A) Monash University Assoc. Prof. Shona Leitch RMIT University Dr Ruth Geer, Mr Bruce White University of South Australia TransformingExams.com

Civil Service Exam Under Emperor Jen Tsung (fl.1022) from a history of Chinese emperors(colour on silk): Bibliothèque Nationale, Paris, France.

Is this your exam space?

Palace Examination at Kaifeng, Song Dynasty (960–1279), China.

Civil Service Exam Under Emperor Jen Tsung (fl.1022) from a history of Chinese emperors(colour on silk): Bibliothèque Nationale, Paris, France.

Actually it looked more like this and they locked you in for 3 days!

Imperial Examination Hall Gongyuan (1909), Beijing, China.

Ques:onstoponder1.  Whydowedoexams?2.  Areexamss:llneeded?3.  Canwedoexamsbe[er?4.  Howcanwedobe[erand

whatdoesittake?

5

6

Take 3 minutes § In pairs, discuss should we still keep doing exams?

Then, share with the room – Two reasons to keep (why we do them)

– Two reasons to reduce

PossiblereasonstokeepexamsHighstakes,supervisedtes:ngiss:llneededbecause:•  Qualifica:oniden:fyverifica:on(besureitisthestudent’swork–riseofghostwri:ng/outsourcing)

•  Atestofthestudent’s‘metal’underpressure.•  Accredita:onrequirements.•  Social/poli:calstakeholderexpecta:ons.•  Timeandworkloadefficiencies?•  Otherreasons…?

7

PossiblereasonstoreduceexamsPaperfeatures…•  Limitedpedagogicalscope…MCQ/TF,short/essayresponse,flat

andsta:c,images/diagrams.•  Endproduct(finalresponse)ratherthanprocess.•  Whattheyrememberratherthenwhattheycando(given

accesstorealworldcondi:ons).•  Hand-wri[enpen-on-paperismuchlesscommonasaproblem

solvingandinforma:onpresenta:ontool.•  Otherareasofeduca:ondelivery–progressive,forma:ve

assessmenthasmovedtowardsricherformsofassessment,moderntools,simula:ons,mul:media,wordprocessed,informa:onrepositories…

Isthislimi:ngourabilitytoaccreditgraduatesasbeingabletooperateintheworldoftoday,immersedintechnology,informa:onandcomplexity? 8

QualityAssessment?

9

DET (2016) Quality of assessment in vocational education and training - Discussion Paper, 8 Feb, Commonwealth of Australia https://docs.education.gov.au/system/files/doc/other/2016-02-08_quality_of_assessment_in_vet_-_discussion_paper_jan_2016.pdf

Ins:tu:onsmustbeabletocer:fytheircapabili:estooperateinthe21stCenturyworld.Weneedfitforpurpose,valid,authenAc,assessments.

21stcenturyemployabilityskills

Waysofthinking •  crea:vityandinnova:on•  cri:calthinking,problemsolving•  learningtolearn,metacogni:on

Waysofworking •  communica:on•  collabora:on(teamwork)

Toolsforworking •  informa:onliteracy•  ICTliteracy•  disciplineresourcesandtools

Livingintheworld •  ci:zenship–localandglobal•  lifeandcareer•  personalandsocialresponsibility(including

culturalawarenessandcompetence)

10

http://atc21s.org/ Binkley, M., Erstad, O., Hermna, J., Raizen, S., Ripley, M., Miller-Ricci, M., & Rumble, M. (2012). Defining Twenty-First Century Skills. In Griffin, P., Care, E., & McGaw, B. Assessment and Teaching of 21st Century Skills, Dordrecht, Springer.

Quality Assessment = fit for purpose!

21CLearners

•  skilleduseoftools…asappliances?forlearningpurposes?•  ac:velearningratherthanpassivereceivingofknowledge•  authen:clearningexperiencesratherthancontrivedtasks•  construc:onratherthaninstruc:on•  task(notprocess)oriented•  justin:melearning•  searchnotmemorise•  u:lisesocialnetworks•  doesn’tknowanswerbutknowswheretofindit•  Googlenotlibraries•  collaboratenotcompete.

11

Elliott, B (2007)

LearningandassessmentCommunity expects: Employers, professional bodies and the community are expecting graduates to be able to perform appropriately in proactive, task-directed ways where they implement their discipline specific knowledge in a context that involves many stakeholders. But… Universities are more often concerned with students mastering the conceptual understanding of core discipline principles and applying these to artificially constructed tasks designed for assessments that can be easily delivered and marked. Good for us?

12

Maybe not so good for student learning…?

Rethinkingcurriculumdesign–rethinkingassessment

A rethink is under way… in the broad curriculum.

We are doing better curriculum design… that facilitates both the development and assessment of discipline skills and capabilities, as well as the application of these skills and capabilities … when done well this reflects are more authentic environment with collaboration with other people, information resources, modern tools of the trade.

The real world is messy, complex, contradictory, uses a range of information sources, tools, perspectives, stakeholders and techniques.

We need to move along the continuum from the artificial towards the authentic.

13

Thingsalreadyathand:AssessmentinLMS/VLE

Many more Moodle examples transformingassessment.com Blackboard course archive available upon request. \14

Assessment options in LMSs <- Moodle Blackboard -> Some computer marked while others require human intervention. Drive towards higher order thinking. How about … have students use tools to develop solutions and then respond via LMS.

Moodlee-AssessmentExamples•  TransformingAssessment.com

15http://moodle.transformingassessment.com/

InteracAvetoolsinassessment

16

1.  Studentclicksalinkembeddedinthequiztolaunchaseparateapp.

2.  Undertakesataskasinstructed.

3.  Thenrespondsusingselectedresponseornumericalinputtosuit.

InteracAvetoolsinassessmentSpreadsheetlinkedtoquiz

17

18

Jmol: Open-source Java viewer for chemical structures in 3D Moodle Quiz

Studentsinteractwithonlinetoolstoobtaindatatoconstructaresponse

Interactive tools in assessment

More examples in Moodle at transformingassessment.com

SimulatedConversa:on(Monash)StudentsrespondtoaseriesstatementsviaMCQs(maybe,yes,no)withfeedbackperchoicetosimulateaconversa:one.g.MoodlelessonacAvity.

h[p://conversa:onsim.org/Nelson,R&Dawson,P(2013)Assessment-as-learning:introducingtheConversa:onSim,TAwebinar/e-AssessmentScotland,21Augh[p://transformingassessment.com/eAS_2013/events_21_aug_2013.php

Examples–ShorttextresponseStudentstypeinashortsentenceresponsewhichcanbemarkedbycomputerbasedonpa[ernmatching.AvailableinstandardMoodle.Exampleques:onExampleserngs

SallyJordan,OpenUniUK,TAwebinar,5June2013transformingassessment.com/events_5_june_2013.php

Kine:cenergywillbeatmaximumwhenatthebo[omoftheslide.

evalua:on

Examples–Confidenceques:onsConfidencebasedapproachespenaliseguessing.Studentsneedtochoosearesponseanddeclaretheirlevelofcertainty.AvailableinstandardMoodle.

21

UniversityCollegeLondon

CertaintyvMarkExpected

Certaintylevelsandconsequences

TonyGardner-Medwin,UCL,TAWebinar6April2011h[p://transformingassessment.com/events_6_april_2011.php

Usetechnologytoenhanceques:ons

22

VR,360deg,sphericalimages

http://www.pbs.org/wgbh/nova/subsecrets/nautconhi.html

Virtual reality, 360-degree images, spherical images. Take students to places they would not otherwise be able to go…

http://www.pbs.org/wgbh/nova/subsecrets/nauttorlo.html

LabSimula:onsPhysicalhardwarecanbeconnectedtotheinternetorwecanusesoIwaresimulaAonsoflabsandexperiments.

24

MoreSimula:onsA/Prof.PierreBenckendorff(UniversityofQueensland,Australia)

OLTwork:bizsims.edu.au(31Simslisted/reviewed)Complexsimula:onsdesignedtoteachstrategy,compe::veanalysis,finance,marke:ng,HRM,cross-func:onalalignment,andtheselec:onoftac:cstobuildasuccessfulbusiness.

25TA webinar: http://transformingassessment.com/events_4_march_2015.php

3DVirtualSpaces

26

•  Sloodle(SecondLifeinMoodle)

•  QuizHUD

•  Scriptedchatbots

MoodleandOpenSimWorkingTogether

Dataflowsasifthestudentwas

doingtheac7vityintheLMS

Asetofscripts

forMoodleand

VWthatactsas

abridge.

Student

undertakes

assessmentin

thevirtual

world

SetupQuizin

theLMS.

Resultsare

storedinthein

gradebook.

Videos:TransformingAssessmentYoutubeChannel

Undertakinganassessmentac:vityintheVWini:atesdatatransferstotheLMS.

UQ Virtual Business Environment

TAwebinarh[p://transformingassessment.com/events_5_august_2015.php

Sim Features - trading

1.Money2.A[ributes3.Avatar4.TextChat

Sim Features: Obtain a loan, resolve disputes, buy supplies

The VBE App

VirtualImmersiveLanguageLessons(1)

TAwebinarh[p://transformingassessment.com/events_1_april_2015.php32

Sco[Grant(MonashUniversity,Australia)h[p://www.virtuallyenhancedlanguages.com

VirtualImmersiveLanguageLessons(2)TaskBasedLearningorTaskBasedLanguageLearning(TBLL).Includescommunica:onac:vi:es,usinglanguagetocarryouttasks,languageusethatismeaningfultothelearnerandhasapurpose,andcommunica:onac:vi:esthatreflectreal-lifeac:vi:eswithauthen:cmaterials.Learnbydoing:Purchasesuppliesthencooknoodles–allinChinese.

33

so…

e-exams 34

Where does that leave exams?

Thedissonanceofitall!RealworldofworkExams

35

World Economic Forum – How will digital change your working world. https://agenda.weforum.org/wp-content/uploads/rtr2m8vm1-628x330.jpg

Exams at Monash Caufield in 2015 (mathew.hilier[at]monash.edu)

Wearefacedwithagrowingdisconnectbetweenthewayhighstakestes7ngisconductedusingpenonpaperexamsandstudents’everydayexperiencesofstudy,workandlife.

36

Take 3 minutes § In pairs, discuss what would be an ideal exam?

Then, share with the room – Two outcomes – Two enabling features of an ideal exam

Be[erexams•  Letsmakeexamsasgoodastheycanbe…

– Broadenthepedagogicallandscapeintheexamroom.

– Givestudentsaccesstothe‘toolsofthetradeofthetwentyfirstcentury’…

– Tosolveproblemstypicalofthosefacedintheworkandlifeworld–wicked,messy,complexproblemsthatrequireconstructedresponses.

•  Challenge:mustworkwiththepragma:csandrequirementsofexis:ngeduca:ondelivery.– Valid,robust,easytouse,efficient,economical,supportable…Itmustworkintheworldwehave,nottheworldwewishwehad! 37

Whyandhowofe-ExamsRa:onale:Concerns,drivers,possiblesolu:onsfore-Exams.Atruly'wicked','messy'problemandalongroadtogetitright!...

Presentedra:onalein2013andSeedgrantfindingsat2014,2015ASCILITEconfs.Moretocome…andexploredindepthinHillier&Fluck(2013)h[p://www.ascilite.org.au/conferences/sydney13/program/papers/Hillier.pdf

ta.vu/eexam-map

PossibleFuture-whenandhow?Anevolu:onratherthanarevolu:on.Someaspectsmayoccurquickerthenothersdependingonpar:cularimplementa:on,technicalmodelschosen,socio-cultural-policyenvironmentcondi:ons.OutlookforAustralasia.DenmarkandNorwayalreadyhave‘internetinexams’.

39Hillier, M & Fluck, A (2015) "A pedagogical end game for exams: a look 10 years into the future of high stakes assessment", ASCILITE Conference, Perth, 29 Nov - 2 Dec

Oure-ExamProject...

•  Supervised•  Highstakes•  Oncampus•  Largescale

(imagecredit:DrFluckUTAS)

Whatwearenotspecificallyaddressinghereisoffcampus,

onlineonly,distanceeduca7on,crossins7tu7onalstudents

–thereareextraissues(later!)and

somepossiblee-solu7onstoaddresstheseneeds.

40

e-Exams:Online,Offline,OnCampusorDistance

41

Online(net)

•  Spaceissuesforins:tu:ons.•  Improvedexammanagement

efficiency.•  Equipment:needcomputerlabsto

caterfor2000atonceorBYO.•  Mostsecure:liveITmonitoring/

controlandspacesaresupervised.•  Needsreliablenetwork.

•  Nospaceissueforins:tu:ons.•  Moreefficientexam

management.•  Studentssupplyequipment.

•  Lesssecure:ITmonitoringbutwiderspacesareunsupervised.

•  Needsreliablenetwork.

Offline

•  Spaceissuesforins:tu:ons.•  Lessefficientexammanagement.•  Equipment:needcomputerlabsto

caterfor2000atonceorBYO.•  Moresecure:ITcontrolpossible,

spacesaresupervised.•  Networkreliabilitynotanissue.

•  Nospaceissueforins:tu:ons.•  Lessefficientexammanagement.•  Studentssupplyequipment.

•  Notsecure:nousefulmonitoring/essen:allyunsupervised.

•  Networkreliabilitynotanissue.OnCampus(controlledspaces) Distance(athome)

Therearetrade-offsforanye-examsolu:on.

QuickGuidetoOLTe-ExamsAim:tobe[erassessstudentcapabili:esusingthe'toolsofthetrade'ofthe21stCentury.Todevelopfuturelooking,'workready’(andlifeready)graduatesweneedtobeabletoassesscomplex,constructed,even'wicked'tasks.e-Exam:asupervisedexamina:onundertakenbyacandidateusingtheirowncomputer.CandidatesstarttheircomputerfromacustomdesignedUSBwhichcontains:•  Theopera:ngsystemandafullsuiteofofficeapplica:ons.

•  Theques:onsandallancillarymaterialindigitalformat.

•  Areservedloca:onforcandidateresponsesWorksonmostAppleand‘Windows’laptops.

e-ExamSystemAffordances

43

Per:nentFeatures AffordancesA'Wholecomputer'environment(OS,LMS,applica:ons…)onas:ck.

Expandedpedagogicalscopeoverthatofapenorbrowserwindow.

TypedresponsesviaWordprocessor,constructedviaapps(humanmarked)oron-boardlearningmanagementsystemquiz(computermarked).

Catersforintroduc:ontoadvanceduses.Componentscanbeadded/removedtosuit.Electroniccollec:onfacilitatesanaly:cs,itemresponseanalysis…

Nolivenetworkrequiredduringexam,evenforLMSques:ons.

Robust.Greatercontrol.(networkcouldbeusedforadmin)

Studentownedequipmentusedashostandleyuntouched.

Anethicalapproachtoscalability(noinvasivesoywaretoinstall)

Modular,opensourcecodebaseandcommodity'offtheshelf'components.

Leveragingpopularandsustainableprojectsforbe[erefficiency.Fully'known'(no'blackbox').Available!

OneversionworksonmostIntelbasedlaptops-Apple,'windows',Linux,thathaveaUSBport.

Onesoywareversioncanserveall.Streamlinesdevelopmentandmaintenance.

Bootable ‘live’ USB drive

e-exams 44

Start>>> >currentphase1trials> >>> >>> >>> >>>Future>>GetReady Phase 1 Phase 2 Phase 3 Phase 4 Phase 5

Ins:tu:onalapprovals, Paper equivalent small scale. Post-paper small to medium. Medium to large scale. Whitelisted and logged Internet Open but fully logged Internet ResearchEthics, Basic doc exams only to begin! Expanding the media landscape. Adding the power of an LMS. Network during BYOD exam. Network during BYOD exam.

Hardware&infrastructure. Crawling Walking Running Jumping Flying! Type1:Paperequivalentdocs Type1:Paperequivalentdocs Type1:Paperequivalentdocs

Studentsgivenchoicetotype. Studentsgivenchoicetotype. Studentsgivenchoicetotype. Smallnumberstyping~20to50.Mediumnumbers~100-200. Largenumbers~200+ Simultaneous,oncampus,

supervised.Simultaneous,oncampus,supervised.

Simultaneous,oncampus,supervised.

Nolivenetwork.Paperfullback. Nolivenetwork.Paperfullback. Nolivenetwork.Paperfullback. BYOstudentlaptops,Venue

powersockets,sparelaptops.BYOstudentlaptops,Venuepowersockets,sparelaptops.

BYOstudentlaptops,Venuepowersockets,sparelaptops.

Type2.A:Postpaperviaworddocslinkedtomediaandothertools

Type2.A:Postpaperviaworddocslinkedtomediaandothertools

Type2.B:Postpaperviaworddocslinkedtomediaandselectonlinerefs

Type2.C:Postpaperviaworddocslinkedtomedia,openonlineaccess

Allstudentsmusttype. Allstudentsmusttype. Allstudentsmusttype. Allstudentsmusttype. Smalltomedium~50-200. Mediumnumber~100-200. Largenumbers~200+ Largenumbers~200+ Simultaneous,oncampus,

supervised.Simultaneous,oncampus,supervised.

Simultaneous,oncampus,supervised.

Simultaneous,oncampus,supervise

Nolivenetwork. Nolivenetwork. WhitelistInternet. Fullyloggedopeninternet. Docslinkedtomul:media,3rd

partyapps,simula:on,PDF,etc.Docslinkedtomul:media,3rdpartyapps,simula:on,PDF,etc.

Docslinkedtomul:media,3rdpartyapps,simula:on,PDF,etc.

Docslinkedtomul:media,3rdpartyapps,sims,PDF,etc

BYOstudentlaptops,Venuepowersockets,sparelaptops.

BYOstudentlaptops,Venuepowersockets,sparelaptops.

MixBYOstudentlaptops,VenuepowersocketsandLabPCs.

MixBYOstudentlaptops,Venuepower&LabPCs.

Type3.A:On-board(Moodle) Type3.B:On-boardLMS,onlineType3.C:On-b’dLMS,open Allstudentsmusttype. Allstudentsmusttype. Allstudentsmusttype. Mediumnumbers~100-200 Largenumbers~200+ Largenumbers~200+ Simultaneous,oncampus,

supervised.Simultaneous,oncampus,supervised.

Simultaneous,oncampus,supervise

Nolivenetwork. WhitelistInternet. Fullyloggedopeninternet. On-boardMoodlequiztool. On-boardMoodleQuiztool. On-boardMoodleQuiztool. BYOstudentlaptops,Venue

powersockets,sparelaptops.MixBYOstudentlaptops,VenuepowersocketsandLabPCs.

MixBYOstudentlaptops,VenuepowerandLabPCs.

Type4.B:Online/Networked.MixedMode,onlinerefs

Type4.C:Online/Networked.MixedMode,openonline

Allstudentsmusttype. Allstudentsmusttype. Mediumnumbers~100-200 Largenumbers200+ Simultaneous,oncampus,

supervised.Simultaneous,oncampus,supervise

WhitelistInternet. Fullyloggedopeninternet. OnlineLMS(Blackboard,

Canvas,D2Letc.),on-boardLMS,remotedesktops,docs,media,sims,appsMixnmatch!

OnlineLMS(Blackboard,Canvas,D2Letc.),on-boardLMS,remotedesktops,docs,media,sims,apps…

MixBYOstudentlaptops,VenuepowersocketsandLabPCs.

MixBYOstudentlaptops,VenuepowerandLabPCs.

Type

1:P

aper

equivalent-w

ordDo

cs

Type

2:P

ostp

aper

(Worddo

cslink

edto

med

ia)

Type

3:O

n-bo

ardLM

S(M

oodle)

Refer to handout or this http://ta.vu/e-exam-roadmap

Interac:vee-assessmentdesign

45

Interactive tool

Response

Question

Separatetheinterac:vetool/object/artefactfromtheques:onandthefeedback.

Nowapplythattoexamstoo!

Wherewearenow:PaperEquivalentWorddoc!Ques:ontypesused:shortanswer/essay,matching,constructatable,labeladiagram/image(byfillingatable).Manualmarking.

46

Wherewearenow:PaperPlusWorddoc,plusapps.Ques:ondocumentwithlinkstolaunchlocalappsandresources:graphics,Scratchprogrammingtools,presenta:onslides,PDFs.Allon-boardtheUSBs:ck.‘ITinEduca:on’exams:AndrewFluck,UTAS.

47

PDF

Scratch SDK

Exam doc

TheFuture:Post-paperexamsSimula:ons,tools,virtualexperiments,seriousgames…Business,history,language/communica:on,sciencelabs…

'Windows'soywareviaWINE.E.g.CAD/3Dmodeling,Celes:a.Moodlequizwithmedia(automarked).

48

Currente-Examv5Workflow

49

Postsession:retrieveresponsesandassessment

Examroomuse

ResponsesretrievedfromUSBs. Collatede-responsessenttoacademic.

CreatemasterUSB(tested)

USBsduplicatedperstudentAcademiccreatesexamlearningmaterial

Set-up:prepareexamlearningmaterials Pre-session:Studentlaptopsetup&prac:ce.

CollectUSBs(responses)

1. Studentsenterroom.

2. GivenUSB.3. Bootlaptop.4. Doexam.5. ReturnUSB.6. Leaveroom.

Ubuntu

LiveUSB.

LibreOffice,

Moodleetc

E-examprocessingsta:on

50

Prototype using commodity components.

Using an old laptop it can handle 98 USBs at once.

Helper scripts to

•  Initial ‘burn’ OS to each USB

•  Place exam materials.

•  Retrieve exam materials.

•  Set-up for the next exam.

User friendly interface planned.

e-Exam:WalkThrough

51

WalkThrough

2.HolddownALTthenpoweron.KeepholdingALTun:lyousee…

52

2.Poweronwhiletapping‘bootkey’(e.g.F12or…)Keeptappingun:lyousee…(orsimilar)

*Win8:firstneedtodisablesecureboot.

1. Start with the computer turned OFF. Then insert USB stick

Apple Other/Windows*

Taptaptap...Holddown

Pressandreleasepower

Pressandreleasepower

WalkThrough

3.Selectayellowicon.EFIbootor‘Windows’

Couldbeeitherone!Sojusttry.Ifyouget

Trytheotherone!53

3.SelectUSBdevice.Itmightbelabeledsomethingelseandprobablywontbefirst.

Apple Other/Windows

e-Exam

e-Exam

WalkThrough

4A.Somesystemmessagesmayappear,ifsojustwait.4B.Examsystemstarts.

5.Arriveate-Examsystemdesktop.

54

WalkThrough6.StudentnowtypesintheirstudentIDnumberandname*(canbedisabled).ClickStartExam.(whathappensnextdependentuponchosenexammode)

55

WalkThrough(Documentmode)7.Examfileopensreadytoenterexamdetailsandresponses.

56

Note:originalfilecopiedandstudentnumberprefixedtofilename.

WalkThrough(Documentmode)8.Readques:onsandtyperesponsesintoareasindicated.

57

WalkThrough(post-paper)8.1in‘post-paper’exams:Makeresourcesavailable(PDFs,Videos,images)andusespecialistsoywaretoolstoconstructresponses.Launchedvialinksinthedocument.

58

WalkThrough9.Remembertosave(CTRLS).WhendoneuseFile>Exit.

[thereisa2min‘autorecovery’save]

59

WalkThrough10.Shutdownthesystem.Whenthesystemhaspoweredoff,handintheUSBS:ck.

60

Shutdownbu[on

e-Exam:LMSGatewayMode

61

StartlaptopwithUSB.Insteadoflaunchingadocument,itopensawebbrowser.Networkconnec:onsonlypossibletopre-defineddes:na:on–e.g.UQBlackboard

e-Exam:LMSGatewayMode

62

UseBlackboardtestserngsforcontrol.BesttohaveseparateExamserver.

e-Exam:on-boardLMSMode

63

StartlaptopwithUSB.Launchesawebbrowsertologintoanon-boardMoodle.Moodleaccountcreated-on-the-fly.Networkconnec:onsNOTpossible.

e-Exam:on-boardLMSMode

64

Removenon-examrelatedmodulesandmenus.Largemul:mediaispossibledueon-boardstorage.

MoreQuesAonsandDiscussionPoints1)Howwellpreparedare:

– StudentstoBYODforexams?– Academicsfortakingonnewapproachestoexams?

– Ins:tu:onstosupportBYODinexamvenues?2)Transi:oningtoe-Examsvia'paper-replacement'examsappearsthemostfavouredstrategy(e.g.firstreplica:ngpaperexamsinanelectronicformbeforemovingontoamixofrichmul:media,linkedsoywareapps,andcomputermarkedques:ons).Doyouagree?3)Whatkindsof'post-paper'examscanweenvision?4)Howcane-Examsenablecurriculumtransforma:on?

Ready?

66

Moreinforma:on….Demoset-upGuide,StudentPrac:ceandUserGuideh[p://transformingexams.com

Demovideosstart-up,useandrecoveryexamples.Appleh[p://ta.vu/eexam-demo-a'Wintel'(Dell)h[p://ta.vu/eexam-demo-dContact:mathew.hillier[at]monash.edu

e-ExamProjectResources

67

End

CitethisresourceHillier,M(2016)“OLTe-Exams:Rampinguptheauthen:cityofhigh-stakessupervisedassessmentforthe21stcentury”,Presenta:onforMacquarie,Sydney,23Sep.mathew.hillier@monash.eduSessionfeedbacksurvey

h[p://ta.vu/e-exam-serv

ReferencesDermo,J.(2009).e-Assessmentandthestudentlearningexperience:Asurveyofstudentpercep:onsofe-assessment.

Bri:shJournalofEduca:onalTechnology,40(2),203–214.h[p://doi.org/10.1111/j.1467-8535.2008.00915.xFluck,A.(2013).Implementa:onofon-campusdigitalexamina:onprac:ces.PresentedattheTransforming

AssessmentinaDigitalEra,Melbourne,Australia.Retrievedfromh[p://www.eexams.org/stories/Documents/TA%20in%20a%20Digital%20Era%20-%20Fluck.pdf

Fluck,A.E.(2011).eExamsforTransforma:onofHigherEduca:on.PresentedattheAustralianAssocia:onforResearchinEduca:onConference,HobartTasmania.Retrievedfromh[p://www.aare.edu.au/data/publica:ons/2011/aarefinal00107.pdf

Fluck,A.,&Hillier,M.(2014).eExamsTransformingCurriculum.InNowIT’sPersonal(pp.151–158).Adelaide,Australia:ACEC.Retrievedfromh[p://acec2014.acce.edu.au/sites/2014/files/a[achments/eExams%20paperd%20-%20REV2b.docx

Fluck,A.,Pullen,D.,&Harper,C.(2009).CaseStudyofaComputerBasedExamina:onSystem.AustralasianJournalofEduca:onalTechnology,25(4),509–523.

Hillier,M.(2014).TheVeryIdeaofe-Exams:Student(Pre)concep:ons.PresentedattheAustralasianSocietyforComputersinLearninginTer:aryEduca:onconference,Dunedin,NewZealand.Retrievedfromh[p://ascilite.org/conferences/dunedin2014/files/fullpapers/91-Hillier.pdf

Hillier,M.,&Fluck,A.(2013).Arguingagainfore-examsinhighstakesexamina:ons.InH.Carter,M.Gosper,&J.Hedberg(Eds.),ElectricDreams(pp.385–396).MacquarieUniversity.Retrievedfromh[p://www.ascilite.org.au/conferences/sydney13/program/papers/Hillier.pdf

Lim,E.C.H.,Ong,B.K.C.,Wilder-Smith,E.P.V.,&Seet,R.C.S.(2006).Computer-basedversuspen-and-papertes:ng:students’percep:on.AnnalsoftheAcademyofMedicine,Singapore,35(9),599–603.

Mogey,N.,&Hartley,J.(2012).Towriteortotype?Theeffectsofhandwri:ngandword-processingonthewri[enstyleofexamina:onessays.Innova:onsinEduca:onandTeachingInterna:onal,50(1),85–93.h[p://doi.org/10.1080/14703297.2012.748334

Sorensen,E.(2013).Implementa:onandstudentpercep:onsofe-assessmentinaChemicalEngineeringmodule.EuropeanJournalofEngineeringEduca:on,38(2),172–185.h[p://doi.org/10.1080/03043797.2012.760533

220moreat:h[ps://www.zotero.org/groups/e-assessment/items/tag/e-exame-ExamsOnlineConferenceRecordings(2014)h[p://ta.vu/eAS2014

69