How best to evaluate the effectiveness of a VLE

How best to evaluate the effectiveness of a VLEChris RowellTechnology Enhanced Learning Advisor – Regent’s College, London.

IntroductionSession outcomes:Identify ways of evaluating technology enhanced learning

Understand how to evaluate a VLE using qualitative data

Identify ways in which effective academic practice in VLE’s can be supported and developed

The contextRegent’s College:Central London7 SchoolsFees: £13, 000 to 21,0004000 students350 academics

My role: learning technologistBb VLE

How do you evaluate your VLE?

Spend 10 mins discussing the following:

Does your institution evaluate its VLE?How is your VLE evaluated?Who does it?Why do they do it?

Methodology

Baume (2010) paraphrasing Kirkpatrick (1992):

Did people like it?What have they learnt from it?Have they applied what they learnt to their

practice?Have the results improved?

My research project at Regent’s College:

Quantitative data:

My research:1. Online questionnaire2. Interviews3. Analysis/report

Questionnaire

Interview questionsYour task:Arrange the interview question in order of importance.

InterviewsInterview questions for lecturers Starting Bb1.a) How easy was it for you to start adding content to your module area? b) What were the main difficulties you faced when creating a new module area in Bb (if any)2. Have you experienced any technical difficulties using Bb?Staff/Student interaction 3. Do you think the students have an increased expectation of your availability since Bb was introduced? 4. Do you believe that Bb can replace a part of your face-to-face contact with your students?5. Do you think that Bb affects the attendance in your lessons?6. In your opinion, has Bb changed the quality of interaction with your students?7. Has the introduction of Bb affected your relationship with students?Teaching8. Do you think that Bb enables you to become more or less student-centred in your teaching?9. Has Bb affected (or could affect) the way you teach in the classroom? 10. How effective is Bb at supporting collaborative activities on your course?Assessment11. How has Bb affected the way you assess students work? 12. Have you changed the amount of feedback to students since the introduction of Bb?13. Do you use Bb to evaluate how much work the students are doing on your module? 14. Do you use Bb to identify any students who might be “at risk” on your module?Workload15. Has the introduction of Bb changed your overall workload?  Future needs16. Would you like more training or on-to-one support to develop your Bb module? If so, what areas of Bb would you like to develop?17. Would you like to make any other comments about Bb?

Report

Revisiting our evaluation criteria

Did people like it? What have they learnt from it? Have they applied what they learnt to their practice? Have the results improved?

Who gets the report?

Why bother?

ConclusionWhat I have got out of it: Justification of what I am already doing Influenced future direction of the VLE Ticks SM’s boxes Unintended consequences

Any questions?

Contact me:

rowellc@regents.ac.uk