Mechanised Assessments that Promote Deep Learning

…and prevent plagiarism!!

Gareth Denyer & Dale Hancock,University of Sydney

Multiple Choice• Rapid, convenient

– Excellent for large classes• BUT…. do they encourage surface learning?

– Questions often designed to test unambiguous facts• Student PERCEPTION

– adopt a ‘rote’ approach– Names, facts, numbers

• Attempts at deeper, conceptual based possible…– BUT… often lead to ambiguous questions– At worst, questions favour less able students because better

students confused!• VERY SKILFUL JOB!!!

Marking MCQs• No credit for near misses or process

– No point in leaving answers ‘blank’• So you never know ‘why’

• Guessing can give 20%– Negative marking?

• Intimidates students• Forces meticulous non-ambiguity• Further driving surface learning!

• Post-Exam revision of mark scheme difficult– Unlike SAQs

• where mark scheme can be dynamically changed– Questions often ‘discarded’

• Disadvantages good students

Graded AlternativeNB. This is a SURFACE example!!

What is the capital of Australia?A MelbourneB SydneyC GlasgowD CanberraE Auckland

– The more complex the question, the more important partial marks become.

1

0.5

0.5

0

-1

XL Based Graded SolutionTypical student answer grid

Key Sheet

Marks

Calculations

• Blah, blah… What is the approximate ε450nm for the coloured compound formed in the assay?

A.0.199 mM-1 cm-1

B.250 mM-1 cm-1

C.4 mM-1 cm-1

D.2 mM-1 cm-1

E.19.6 mM-1 cm-1

Each option reveals a particular mistake

Especially good with a multi-step calculation

Can reward process

Opens up Deeper Styles

• The “What if?” Cause → Effect– What would be the consequences of inhibition

of lipolysis during the first few days of starvation?

• Shades of Grey– You decide that you need 50 mU of citrate

synthase (CS) in the cuvette. What is the MOST PRACTICAL way of doing this?

Advantages• Makes marking flexible

– Like an SAQ!– Post-hoc changes possible– Provides an alternative way of scaling

• Staff less stressed– More extrapolative questions

• Less attention to rigorous ‘fact’• Less concern about ‘getting it perfect’

– No conflict with colleagues over ambiguous questions• Feedback better

– Each option “useful”

Plagiarism!

• Much attention given to detecting cheating in assignments– Puts you into a conflict situation– But exams make up most of the marks!

• Cheating is easy in MCQ exams– Hand signals– Pattern recognition

• Several cases – A healthy literature!

1 2 3 4 5 6 7 8 9 101

4

7

10

0

5

10

15

20

Similarity of Incorrect Answers

Column

Row

adapted from data in: Harpp & Hogan (1993) Crime in the classroom: Detection and prevention of cheating on multiple-choice exams. J. Chem. Educ. 70, 306

Islands of corruption

Wise Words

Universities obviously want to stamp out cheating

But they also do not want any students to be falsely accused

Statistical evidence is generally not enough.

Pursuing a case is traumatic for all those involvedEspecially if an unwilling cheatee!

1 in 653,000

The Versions SolutionCreate FOUR versions of the same paper

Questions in same order, options just rotated

Layout so that each version is isolated from its clones

Very easy in Word

A database solution can generate even more versions.

Implementation

• Sweet Justice!!– Appropriate recompense without confrontation– Lots of ‘evidence’

• Sometimes even ‘check’ questions copied

• Easy– Just Word and Excel– Rotations take about 10 min per 100 MCQs

Take Home Messages

• Graded MCQs give flexibility• Different versions disincentivise cheating• All easily implemented

– Word and Excel• We are happy to provide templates and

instruction

Have a go! Easy and Liberating!