Personalised feedback vs consistency: the role of electronic marking and feedback.

Personalised feedback vs consistency:the role of electronic marking and feedback.Maureen Readle & Jak RadiceCentre for Educational DevelopmentUniversity of BradfordM.Readle@Bradford.ac.ukJ.Radice@Bradford.ac.uk

Personalised feedback vs consistency the role of electronic marking and feedback.The University of Bradford has been trailing the use of different eMarking tools (Blackboard rubrics and TurnitinUK GradeMark) to provide timely, rich feedback to students on summative coursework. One driver for the use of eMarking is to provide consistent feedback to students across cohorts and programmes through the use of marking criteria presented as a rubric. Students do not appear to value or engage with feedback and yet the Universitys NSS scores for assessment and feedback show the lowest student satisfaction results compared with the other categories. Staff are generally positive about the tools and find them easy to use. Blackboard rubrics are used to mark a range of different types of coursework from essays and reports to presentations and practical work marked using iPads.Many staff want to give students personalised feedback through individual comments and in-text annotations as they perceive this to be of value to the students. However, this can result in students receiving an inconsistent level of feedback from different lecturers.Student surveys show that they like to receive their feedback electronically and find it easy to use. However, it is not clear if feedback is more consistent or of a higher quality than marking on paper as previously done.The university would like to move towards a strategy of providing personalised feedback on formative tasks which benefits the students as they prepare for their summative assessment. More consistent marking of summative assessment could then be based on the use of assessment criteria only.1Drivers for eMarking / eFeedback

20 daysAnonymous Marking

BlackboardPebblePadTurnitinUK

Assessment & FeedbackImprove the quality of feedbackNSS ScoresUniversity of Bradford policies:Electronic submission,Typed feedback,20 day turnaround,Anonymous marking.2eMarking ProcessAnonymous markingMarking hidden from studentsUsing PebblePad and Blackboard rubrics some use of TurnitinUK GradeMark 3How we use rubrics:34+ Blackboard rubrics covering:Audio Visual workEssays / assignmentsExam essaysLab reportsPortfoliosPostersPractical work: OSCEs, NMSK,PresentationsSeminars.iPads used to access Blackboard rubrics to mark:Presentations Practical work (OSCEs) Mock interviews.OSCE Objective Structured Clinical ExaminationNMSK Neuro-Musculoskeletal Practice4Using Blackboard Rubrics

Read the students work here or download the file here

Open the rubric5Using Blackboard Rubrics

Grid View:Select the appropriate level for each criterion and add a comment if appropriate.

Select the mark for each criteria from the range available.6Using Blackboard Rubrics

General comment on the work.Option to change the mark.The rubric adds up the marks selected.7Using Blackboard Rubrics

Add a comment for the student so they know where to find their feedbackOption to add notes for other markers. Grading notes are not seen by the student.Rubric total is displayed

Blackboard calculates statistics and a rubric report click on the column header options to view.8Formative FeedbackInline grading:Annotating students text.

9Staff viewsPersonalised feedbackRubric used as a starting point to tailor individual feedbackConsistent feedbackRubric used to deliver common messages to cohort

Feedback given in a consistent manner

Lecturer 1: I think its allowed me to be more consistent in terms of giving feedback, I think its allowed the common things that crop up to be kind of more accurately recorded so you can get a message across to a cohort as a whole but also make it individual from that point of view. I think you can make it very personal. I dont think that has changed particularly from paper-marking to on-line marking.Lecturer 2: Im finding myself writing quite a lot of things over and over again, just literally changing the wording of it so it feels like Im tailoring it to the students but actually there are things that come up really, really commonly in Masters level work, really commonly, so things about critical analysis, things about integration, things about synthesis so I do try to be consistent now in the way that I give feedback about all of those.Lecturer 3 (ADLT): the benefits for the students are that the same rubrics or similar rubrics are used in the same place at the same time so once theyve got used to where to find the information then theyre easily accessible and continually there throughout the programme for the student.Lecturer 3 (ADLT): There are some challenges now. Having now introduced consistency about how feedbacks given its then an issue about volume because some lecturers will give a large amount of feedback to summative work whereas others say we prefer to give more feed forward about this is what you need to do next time, this is where perhaps you could have done better this is where to go next time and thats actually about marking processes as much as anything else and it doesnt matter what system you use it will be the same discussions.10Student ViewsThe previous system enabled much more comprehensive and detailed feedback within the text. This was far more useful than the new system.Generally more consistent because the feedback was structured.

I have noticed no reduction in the depth/quality of marking, and the fact that I can access it instantly from my phone is excellent.11Student ViewsThe quality and consistency still depends on the individual tutor who gives feedback disregarding the system (paper or electronic).The rubric is too generic, the feedback is not individualised, therefore not at all constructive therefore cannot be used to improve further assignments.Tutors use rubrics as an excuse to not have to give individual feedback, as this allows them to just tick relevant boxes and not comment at all. Found it very impersonal.12DiscussionWhat systems have you used?How does your experience match with ours?What plans do you have for eMarking?Experience using iPads?13Maureen Readle & Jak Radice

Centre for Educational DevelopmentUniversity of Bradford

M.Readle@Bradford.ac.ukJ.Radice@Bradford.ac.uk

Climbing the eMarking mountainPersonalised feedback vs consistency the role of electronic marking and feedback.The University of Bradford has been trailing the use of different eMarking tools (Blackboard rubrics and TurnitinUK GradeMark) to provide timely, rich feedback to students on summative coursework. One driver for the use of eMarking is to provide consistent feedback to students across cohorts and programmes through the use of marking criteria presented as a rubric. Students do not appear to value or engage with feedback and yet the Universitys NSS scores for assessment and feedback show the lowest student satisfaction results compared with the other categories. Staff are generally positive about the tools and find them easy to use. Blackboard rubrics are used to mark a range of different types of coursework from essays and reports to presentations and practical work marked using iPads.Many staff want to give students personalised feedback through individual comments and in-text annotations as they perceive this to be of value to the students. However, this can result in students receiving an inconsistent level of feedback from different lecturers.Student surveys show that they like to receive their feedback electronically and find it easy to use. However, it is not clear if feedback is more consistent or of a higher quality than marking on paper as previously done.The university would like to move towards a strategy of providing personalised feedback on formative tasks which benefits the students as they prepare for their summative assessment. More consistent marking of summative assessment could then be based on the use of assessment criteria only.14nullBlues56057.848nullBlues35030.367