Embed Size (px)
Citation preview
PATRICK CRAVEN – DIRECTOR ASSESSMENT POLICY & RESEARCH
VocTEL 2015
Using Controlled Digital Frameworks to Assess Practical Synoptic Tasks
Presentation – 17th June
MATT WINGFIELD – CHIEF BUSINESS DEVELOPMENT OFFICER
ASSESSMENT: AN INEXACT SCIENCE
RELIABILITYTrust & Regulation Agenda Rigour and Quality Assurance
VALIDITY
Fit for Purpose and Authentic
Industry &AssessmentAgenda
VIABILITY
Robust and Sustainable
Service &Business
Agenda
THE SCOPE FOR INNOVATION
ConvergentAssessm ent
DivergentAssessm ent
M ultiple Choice,Hot-Spot,Drag & Drop,Interactive ...
OBJECTIVE SUBJECTIVE
ASSESSMENT TYPES
E-A
SS
ES
SM
EN
T C
OM
PL
EX
ITY
Single W ord/ValueResponse
Explanatory Phrase/SentenceForm ulae
Explanatory Paragraph
TechnicalFactualEssays
CreativeEssays
Skills/FunctionsTests
TechnicalPerformance
CreativeProfessionalPerformance
AUTOMATED ASSESSMENT
HUM AN MARKING
THE EVOLUTION OF E-ASSESSMENT
• E-assessment is no longer just on-screen MCQ:• Adaptive Testing• Simulations• Evidence-based assessments
• All are robust, scalable, proven and auditable
• There are great examples out there that...
• Support self-reflective learning• Evidence the learning process• Assess the application of knowledge & understanding
THE WISDOM OF ROUTINE
THE THEORY OF EVOLUTION
In the struggle for survival, the fittest win out at the expense of their rivals because they succeed in adapting themselves best to their environment.
(Charles Darwin)
8
WHAT NEXT …..?
• A leading provider of evidence-based assessment tools to support personalised assessment for learning;
• Used by institutions around the world to track and assess learner progression and to provide feedback;
• Used by professional/accreditation bodies since 2004 to support formal assessment of knowledge & skill application.
DIGITAL ASSESS
CAPTURE THE LEARNER JOURNEY
• People are good at:• Relative Judgements;• Ranking work.
• People are less good at:• Absolute Judgements, e.g. Scoring.
• In addition, when assessing student work we:
• are influenced by past experience;
• work independently – magnifying the effect of subjectivity.
THE ASSESSMENT CHALLENGE
Comparative Judgement:
Perhaps we should take a different approach?
• Measuring without subjectivity – adaptive scaling without needing scores;
• Presents a pair of items and asks the observer to make a value judgement – which is ‘better’?
• Repeat this through all possible combinations, and record the results to create a rank order;
• The Law of Comparative Judgement (Thurstone - 1927).
THE ASSESSMENT CHALLENGE
We have created a web-based ADAPTIVE comparative judgement solution for assessing student work...presents successive pairs of student work to assessors and asks them to say which is best at meeting assessment criteria:
ACJ – Adaptive Comparative Judgement:HOW IT WORKS
• The system dynamically organises the work into a scaled rank order as the students complete their judgements.
• Which can produce a grade/score if required.
HOW IT WORKS
• The benefits of structured synoptic e-assessment:• Enables quicker work-readiness through use of authentic tasks;• Provides a legitimate place for technology in assessment;• Facilitates richer, stronger formative support of learning
through improved visibility of process;• Provides ‘forward indicators of readiness’:
The right skills; The right attitude; In the right time;
• Provides greater transparency: • extending learning to address weaknesses.• of collaboration and teamwork• of progress, decision making, critique and explanation
WHY DOES THIS MATTER?
The eAssessment Assocation
Join the eAA for FREE @ www.e-assessment.com
