View
3.995
Download
0
Embed Size (px)
DESCRIPTION
A presentation given to members of the School of Modern Languages at Durham University, exploring different ways of providing student feedback using an online learning environment such as Blackboard.
Citation preview
New Options for Online Student Feedback
Learning Technologies TeamInformation Technology Service
Learning Technologies Team
Judith JurowskaFaculty of
Arts & Humanities
Martin EdneyFaculty of Science
Julie MulveyFaculty of
Social Sciences & Health
Dr Malcolm MurrayLearning Technologies
Team Leader
Session outline
Voting systems – Julie MulveyExcel Spreadsheet – Judith Jurowska
Blogs – Christine Bohlander & David TualGradeMark – Judith JurowskaAudio/video – Martin Edney
Preview: Feedback Tool – Malcolm Murray
New Options for Feedback
Using Voting Systems
Julie Mulvey
Voting systems
Audience Response System (ARS)www.keepad.co.uk
TurningPoint plug-in to Microsoft PowerPoint
Voting Systems
Research by Marina Sawdon from Phase 1 Medicine in using an Audience Response System
• Assess prior knowledge Bransford 1999
Question asked before lecture starts
• ‘Learning occurred’ Same question asked at the end of lecture
• Knowledge retention/ decay at different time points
Question asked again at 1 week and 4 weeks after lecture
Voting Systems
Voting Systems
Evaluation
• The audience response system gives me feedback on my progress
99% student satisfaction• The audience response system aids my knowledge
recall98% student satisfaction
• The audience response system consolidates my knowledge
98% student satisfaction
Voting Systems
“Use of KEEpads was excellent, getting instant feedback is a good learning tool”
“I liked the KEEpad system because it allows you to answer questions without anyone knowing if you got it wrong or not.”
“I liked the use of the KEEpads, it helped break up the lectures and gave feedback”
“KEEpad use…gave instant feedback and kept me focused and immediately applying the lecture content to new situations”
Voting Systems
Marina’s Conclusions:
• Improves and facilitates learning• Increases student participation “…it wakes me
up!!!”
• Improves knowledge retention• Instant feedback• Low cost• Students love it!
Sawdon, M. 2009. Improving Knowledge Retention using KEEpad.
Medical Education 43(5): 487-487
Voting Systems
• KEEpad only one type of voting system
• PollEverywhere.com Online system
• Informal SIG at Durham• ESTICT (Engaging Students through
In-Class Technology) (http://estict.ning.com)
New Options for FeedbackAutomated Excel Spreadsheet
Judith Jurowska
Frederico Federici Spreadsheet
Dr F M Federici Spreadsheet
Creating criteria
Entering the scale
Pulling the comments together
Enter Candidate details
Printing the report
Finished student sheet
Results Summary
• Marking time reduced by 60%• Setup time was considerable. (once only)
• Double marking was possible• Second marker was impressed by the
volume of feedback provided• Needs further work before it can be
used widely.
Liverpool John Moores University
Dr Phil Denton
QED article by Kevin Hinde
QED Issue 8 Michaelmas Term 2008
New Options for Feedback
Using Blogs for feedback on written workChristine Bohlander
& David TualCentre for Foreign Language Study
Problems related to feedback and possible solutions
• Keeping track /a trace of past work and feedback
• Students’ follow-up on feedback
• Repetitive feedback to different students
• Collecting work, keeping records, photocopying/filing
• Ask student to re-write and resubmit
• Provide feedback in plenary session
A different solution: the blogs
• What is it?!
The blogs in practice
• Students are set homework to be submitted via the blogs on DUO
• We feedback on their work using the following colour code: - ‘slips/lapses’, ‘first-order mistakes’ and ‘second-order mistakes’ - correction of ‘errors’- word(s) or clusters to be deleted (because wrong or corrected in green)
The blogs in practice
• Students are invited to correct their mistakes in blue (and crossing out the original mistakes, but not to delete them) and to look at the rest of the corrections
• Thanks to the colour code, we can then quickly check whether the students have accurately corrected their mistakes.
The blogs in practice
A different solution: the blogs
Keeping track /a trace of past work and feedback
Students’ follow-up on feedback
Repetitive feedback to different students
Collecting work, keeping records, photocopying/filing
Ask student to re-write and resubmit
Provide feedback in plenary session
A different solution: the blogs
Online feedback leading to a better use of teaching time
Students can concentrate on main issues and easily resubmit
‘Unlimited’, easy-to-access virtual storage space
Students’ feedback
Students’ feedback
Essential aspects
• Definition of clear and concrete aims and learning outcomes for essay-writing
• Emphasis on learning process and monitoring of progress
• Collaborative aspect: peer feedback
The blogs in practice
The idea of a collaborative blog
• Students cannot only read each others‘ entries, BUT
• They can also post comments and• They can correct each others‘
mistakes
The blogs in practice
Anticipated problems
• No anonymity possible embarrassment
• Students’ feedback might not be constructive
• German umlauts (ä, ö, ü)
Results of questionnaire
Over 80% of students (strongly) agreed:
• Overall, I prefer posting my written work on the blog to handing in handwritten work because of its positive effect on my learning.
• The opportunity to be able to go back for amendments and revision is important for me.
• I can learn from reading my peers’ work and the tutor’s feedback on their work.
• I am happy for my peers to comment on my work.
Results of questionnaire
BUT only 50% (strongly) agreed:• I am happy to comment on my
peers’ work.
Conclusion
• Students are encouraged to reflect
on their own performance.
• A continuous process of learning with easy monitoring
• While receiving peer feedback is appreciated, students appear to be reluctant to give feedback
References
• James, Carl (1998), Errors in language learning and use, exploring error analysis, Longman: London and New-York
• Klapper, J. (2006), Understanding and developing good practice, Language teaching in higher education, CILT: London
New Options for Feedback
GradeMark by
TurnitinUKJudith Jurowska
Turnitin System used for:• student education about plagiarism• detecting text matches against a student database, journals and the internet
What is GradeMark?
GradeMark provides online marking and feedback
Create a GradeMark Assignment
1. Create a Turnitin assignment(you can turn off the plagiarism check if necessary)
2. Mark using GradeMark’s functions:• Rubrics • Comments• ‘Quick marks’
–Composition–Format–Punctuation–Usage
How can GradeMark help?
• save time marking • provide in context, legible feedback to
your students • provide consistent feedback for large
numbers of assignments • manage the marking process where there
is more than one marker• build a reusable comment resource
Case Study – Steve Lyon
Steve LyonSenior Lecturer : Department of Anthropology
Context
• 1st Year Social Anthropology course• Purpose (aims and objectives)
• Assess understanding of the topic• Improve essay writing skills
• Teaching method • lecture, seminars• formative essay with feedback
• Resources: • Postgraduate teachers• Marking rubric • GradeMark software (available through Turnitin in duo)
• Final assessment - summative essay
Rationale
…when we started doing this [on paper] we had students coming and saying, “I got lots of comments and so and so only got two little lines.”
I wanted somehow to, not impose, but encourage a more consistent amount of feedback for everyone and ensure similar things were being flagged up.
Quote from Steve Lyon
Rubric Criteria (6)
• Answer: Does the essay answer the assigned essay question?
• Sources: Does the essay properly cite all sources? Are all cited references included in the bibliography?
• Logic: Does the answer, build a demonstrable argument based on credible and appropriate evidence?
• Style: Is the essay articulate and coherent? Does it use appropriate language and vocabulary for the purpose? Is it too chatty and informal? Is it unnecessarily complicated?
• Spelling: Are there spelling and grammatical errors? Do these impede communication of the ideas?
• Presentation: Is the bibliography correctly formatted? Is the text in some unusual font? Are the margins correct
Rubric weightings
• 5 weightings for the 6 criteria• Poor (10)• Fair (49)• Good (59)• Very good (69)• Excellent (80)
• Grade by clicking the boxes in the rubric.• the maximum for each criteria is capped at 80%• can manually give more for something beyond the
criteria.
Results: Staff perspective
• Fatigue is less of an issue… The twentieth bad essay no longer invokes lots of exclamation marks, ‘What is this!’
Paraphrased from Steve Lyon QED article
Saved time for markers
Results: Staff perspective
Paraphrased from Steve Lyon QED article
Saved time for markers
• Fatigue is less of an issue…• The ‘marking rubric’… helps automate the process of allocating marks against set criteria. This was particularly helpful to the teaching assistants.
Results: Staff perspective
Paraphrased from Steve Lyon QED article
Saved time for markers
• Fatigue is less of an issue… • A ‘rubric’… helps automate the marking process
• GradeMark allowed Steve to manage this process ensuring a higher degree of quality control and consistency despite the distributed approach to teaching and marking.
Results: Student Perspective
• The student gets at least six comments even if the postgraduate marker [is] less confident…
• Students really like it especially as I give them the rubric before the assignment so they know what they are working to.
• Students like being able to get the feedback online.
• They get more feedback … the essays will be riddled with these little comments that are specific to a paragraph
Paraphrased from Steve Lyon QED article
Improved feedback
QED article
Lyon, Steve. “Making the grade: Helping postgraduate teaching assistants with their marking and feedback..” QED (Durham University), Michaelmas term 2008.
Comments options
Quickmarks
Accessing GradeMark
Accessing GradeMark
How it works
How it works - Markup
How it works – Student View
Click on the Rubric to Mark
New Options for Feedback
Audio/Video
Martin Edney
Audio / Video Feedback
What?• Lecturer records feedback as
• sound file• video file (talking head)• video file (screen capture of submitted
work, with audio commentary)
Audio / Video Feedback
Why?• Students like it – they report
• They get more feedback• They get quicker feedback• “it’s not face to face but it’s certainly one
to one”• Speed (as quick as or quicker than
writing)
Audio / Video Feedback
Why (continued)?• More effective
• Students pay attention• Tone of voice / inflection• Allows you to talk direct to student work
Audio / Video Feedback
How?• Record audio / video file
• e.g. use headset, digital dictaphone, Camtasia, Jing
• one file per student• Deliver file to student
• duo Grade Centre – feedback file option• shared area (e.g. duo Files tab)• email
Audio / Video Feedback
Tips• Individual feedback to each student +
generic feedback to group• Look through submitted work before
you start recording• Say student name or number at the
beginning• Summary / mark at end of recording
Audio / Video Feedback
Some feedback from students• I think this is a much better system than paper
feedback. I know I got a lot more from it than if you had just had to tick boxes and given me a comment in that little box on the piece of paper. It was also much easier to use than anticipated, and I definitely think you should recommend this to other lecturers
• I found the feedback for the formative much better than 'normal' written feedback.
(reported by Ian Greener, Applied Social Sciences, Durham University)
Students from Sheffield Hallam & Chester giving their opinion on audio feedback at a MEL-SIG event in Glasgow [sound file link]
Examples
• Audio example−Comment on draft dissertation work−Sound file available from “A Word In Your Ear
2009” website. −Part of Davies, D. Rogerson-Revell, P. &
Witthaus, G. An exploratory study of speech styles in audio feedback to M- level students
−Presented at “A Word In Your Ear 2009”, see−http://research.shu.ac.uk/lti/awordinyourear2
009/papers.html
Examples
• Video example−English for Academic purposes course−Russell Stannard, University of Westminster−Video file available from
http://www.russellstannard.com/king/king.html
−For full details, see http://www.english.heacademy.ac.uk/explore/publications/casestudies/technology/camtasia.php
Jing for video feedback
• David Tual
My motivation to use Jing
• The possibility to ‘talk’ the students through their work in an asynchronous manner, giving them the chance to stop and think as the feedback is being given to them.
• It takes no space on my J: drive, and there is no need to email or post big files on DUO.
• It can be accessed from any pc with an internet connection.
• Fast and easy to send to the students.• Provide some extra listening practice for
students.
A quick example
• Set a homework task to be submitted via email
• Record my feedback with Jing• Copy the link to the video into the
email reply and immediately send it to the student
• Video file available from http://screencast.com/t/M2ZjMWZmO
New Options for Feedback
Preview: Future Feedback Tool
in duo
Malcolm Murray
Future Feedback tool
A work in progress...
Future Feedback tool
Select Assignment
File Naming Policy
Feedback Template
Feedback Template - Excel
Feedback Template - Text
Downloading Files
Student Work & Summary
Marking & Feedback
Marking & Feedback
Marking & Feedback
Individual Feedback Files
One per Student
Bulk Upload
Grade Centre
Contact
If you would like to know more or have any questions please contact us via