INQUIRE: Interactive Questions Reinforcing

Education

Sophie Clarke and Katharine LindsayAcademic Computing Development Team (Learning

Technologies Group)

Chris McKenna and Steve New Saïd Business School

The INQUIRE Project

• To what extent can on-line multiple choice questions provide a useful support mechanism to undergraduate teaching at Oxford?

• Small scale investigative pilot

A complex learning environment

• Introduction to Management

• Lectures, Tutorials, Reading, Essays

• MCQ Tests primarily based on the lectures

How lectures fit in…A complex learning environment

Our MCQ model • Formative assessment• Carefully crafted questions• Pointers to other material

embedded• Low emphasis on technology• 6 lectures 6 tests

Our MCQ model

Our MCQ model

Our MCQ model

Our MCQ model

Research Questions (1)

• Reinforcement of Learning from other pedagogic elements– Other factors to consider

• Cementing Students understanding– Simple test of knowledge

Research Questions

Research Questions (2)

• Deepening Students knowledge– New topics– Credible resources

• Framing educational expectations– Confirm expectations

Research Questions

Evaluation strategy• Consulted experts & case studies on

evaluation• 6 possibilities - system logs, automated

tracking of link following, confidence logs, 3 minute papers, video observation, focus groups.

• 3 chosen methods – mix of qualitative & quantitative.

• Not cast-iron study

Evaluation - Focus Groups• 30 students recruited

on second day of term

• £5 HMV voucher• Group A: online tests• Group B: paper tests• Group C: control • Focus groups held in

week 6

Evaluation Strategy

Evaluation - Technology based• Following links

- simple PHP script to see which students followed which links (anonymous)

• Login times and frequency recorded by participant.

• Quia

Evaluation Strategy

Revisiting the Research Questions

(1)Reinforcement of learning from other

pedagogic elements.– Paper tests not popular– Light usage of online tests– Trend of late night quizzing

Student5: It’s like something you can do that you feel like you’re doingsomething useful . . .Student1: And that, it’s kind of like in between time, when you’redoing, after dinner, doing work and maybe you go out later to meetyour friends or something.

(2) Cementing Students Understanding– In general patchy retention of the details of

the lectures .• Especially in the paper & control groups.

– Online group had greater retention• The tests jogged their memories to reinforce

ideas and facts that had been discussed in the lectures.

..it kind of triggered your memory, I waited about a week, two until after the lectures to do it, I remembered it better in my mind, doing that.

• Images played an important part in recalling the lecture.

Revisiting the Research Questions

(3) Deepening students knowledge– No groups followed the lecture reading list.– Paper Group did not follow links or

references.• Not at a computer when doing the tests, too

cumbersome to type out URLs, too much reading.

– Online Group - trend of click-and-glance.• Contradiction between logs and what the students

said. Interviewer:. . . for the ones [questions] which had the links, which could take you off and give you something more, long to read, did anyone look at those?Student1: No, I didn’t.Student5: I read some, I just didn’t include them.Student3: Depends how long they were. Student3: Sometimes I read through them but sometimes I thought, ooh that’s too long, for the moment.

Revisiting the Research Questions

(4) Framing Educational expectations– Students realised that should be

remembering the lectures and return to the handouts.

– Acknowledge that they should be reading more.

– Realisation that spoon-feeding is not a part of the course.

Revisiting the Research Questions

I’d like more encouragement as to what we’re actually meant to be learning.

• Rigid study structure v optional online learning – hard for the students to see how it fitted in with the course requirements and blend their learning.

• Talismanic status of the lecture handout.• Low threshold of convenience (passwords/ internet

speed/ computer availability)• Given reading for the lectures as well as tutorials

but often this is not referred to.

Some General Findings

Oxford is like living in a bubble - I’m living within it and everything elseis completely outside

Conclusions• Small-scale evaluation project.• Worked to reinforce the lectures, cement

student understanding and frame educational expectations.

• Value appears to be in ‘jogging effect’ of re-living lecture, and in returning students to lecture notes/handouts

• Additional resources must be engaging & convenient.

• We need to look more closely at how we teach.

• Does use of MCQs assist exam performance? • Would the use of MCQs be different nearer to

the exams?• How can we fit the MCQs more into the

traditional Oxford system.• Look more into motivating students to deepen

their knowledge through wider reading. • Suitable software - reusability (QTI)• The return on gold-plating questions is high but

time consuming – would the outcome be as good if widened out across the department?

Further Investigation