2012 Computer-Assisted Assessment

CMA in Moodle:Past, Present and Future

Tim HuntLeading Technical DeveloperThe Open University

Overview of this talkThis talk will focus on the ‘Present’

1 Overview

2 Questions

3 Activities

4 Conclusions

What are we trying to achieve?A CMA system must address• pedagogic• administrative• technical

considerations

Overview

Demonstration

http://v.gd/quizexampleou

http://v.gd/quizexample

Map of Moodle’s CMA tools

Questions

The quiz activity

The question engine

The question bank

The quiz activity

Administrative parts of Moodle

Other graded activities

Plugins• The various conceptual parts of Moodle are made up of -plugins-core Moodle functionality

• Plugins make it easy to add custom functionality• They break down the functionality into units that can be

combined in different ways

Questions

Question behaviours• How the student interacts with a question• For example

1 The Student gives a response and clicks ‘Check’

2 They immediately get told if they were right, with feedback

3 If they were wrong, the can click ‘Try again’

4 If they get it right on a later try, they get less marks

5 After 3 incorrect tries, they are just told the answer• Some parts of this are configurable

Question behaviours (to date)• Interactive with multiple tries – as on last slide• Deferred feedback• Immediate feedback• Deferred feedback with CBM• Immediate feedback with CBM• Adaptive mode• Adaptive mode (no penalties)• Manually graded

Question types – run-time• I described the behaviour without giving any details of

the question

• Question types say what the question is-what is shown-how the student responds-how it is graded-how the feedback is generated

Question types – authoring• Moodle questions are authored using web forms• Teachers can define the question, within the template

provided by the question type

• Questions are stored in Moodle’s question bank-an area for each course / quiz-hierarchical categories in each area

Question types (to date)• All question types combine a few basic interactions-entering words-entering numbers / symbolic notations-selection-positioning things – points, molecules, parts of a diagram

• Moodle now has question types for all of these separately-… but what about combinations?

Question import/export formats• A way to get questions-in to Moodle from other systems-out of Moodle

• Also useful for bulk authoring

Question display options• Information should, or should not, be visible at given times-marks-right/wrong-different types of feedback

• There are settings for this• and natural limits-e.g. no feedback until an answer is submitted

Putting questions together• In principle, any question can use with any behaviour• In practice, it is not that simple-e.g. essays can only be manually graded

• Still, many interactive questions can be assembled

Activities

The quiz – set up• Sequence of questions to be answered-specific question or random selection-fixed order or shuffled-answered in sequence, or freely

• Which students should have access• When and where attempts can be made

Quiz access rulesAll the different ways students may, or may not, be restricted from accessing the quiz• Open / close dates• Time limit• Number of attempts• IP address• Secure browser• Enforced delays between attempts• Acknowledge academic honesty statement first

The quiz – results• Question outcomes combined to give overall results• Results reported• Final grades sent to the grade-book

Quiz reportsAllow the outcomes to be viewed, analysed and downloaded• Grades• Responses• Statistics – standard test psychometrics – CTT• Efficient manual grading

Lesson activity• Like a “choose your own adventure” book• Since Moodle 1.2 – not yet using the question

bank/question engine

Other possibilities• Could be built in Moodle• Or IMS LTI could be used to connect to external tools

Summary

How does Moodle measure up?A CMA system must address• pedagogic• administrative• technical

considerations

Pedagogy• Behaviours allow a range of learning tasks-Gibbs and Simpson conditions for feedback to promote

learning → Interactive behaviour-Certainty Based Marking

• Question types-designed to allow detailed feedback

• Options for constructing quizzes• Quiz reports

Administration• Moodle provides a lot of this-courses, users, groups, authentication, enrolment-course as an sequence of activities-gradebook, event calendar, notifications

• Question bank• Quiz set-up UI• Quiz access rules• Quiz reports

Technical design• Small core + plugins• Moodle as a framework-runs on Windows/Linux/Mac, Mysql/Postgres/...-concepts of courses, users, groups, uploaded files...

• Simple but flexible database design for questions• Open source project-software as as a way to capture the learning of a

community of practice

Where should Moodle do more?• More different activities using questions-lesson should use question bank / engine-question practice-question authoring / peer review as a learning task

• Better facilities for collaborative authoring of questions-sharing between quizzes / courses / sites-versioning

• More flexibility for building quizzes out of questions

Three things to remember• Question = behaviour + type• Plugins to organise software with lots of functionality• Moodle quiz comes from 10 years of community of practice-Future contributions to the open source project welcome

Documentationhttp://docs.moodle.org/en/Quiz_module

eAssessment at the OU with open source software

http://v.gd/oueassessment

Map of Moodle’s CMA tools