Upload
jaylen-rasch
View
218
Download
1
Tags:
Embed Size (px)
Citation preview
Assessment for learning: the benefits of generating feedback
David Nicol Professor of Higher Education
Centre for Academic Practice and Learning Enhancement (CAPLE]
Director, Peer project (www.reap.ac.uk)University of Strathclyde, Scotland
University of Cadiz, March 9-11, 2011
Plan
BackgroundRe-engineering Assessment Practices (REAP) project www.reap.ac.uk/reap/index.html Concepts and example of practiceInstitutional considerationsPEER project www.reap.ac.uk/peer.aspx Discussion
Background
Departments and faculties: educational improvement projects, including REAP projectPolicy/strategy: development of educational policy and principles of assessment & feedback (based on REAP)Students: ‘Feedback as dialogue’ campaignQuality procedures:– redesigning course documentation and review proceduresPEER project – developing students ability to evaluate the quality of each other’s work
See www.reap.ac.uk
REAP : Re-engineering Assessment Practices
Scottish Funding Council for Universities (£1m)3 Universities - Strathclyde, Glasgow & Glasgow CaledonianLarge 1st year classes (160-600 students)A range of disciplines (19 modules ~6000 students)Many technologies: online tests, simulations, discussion boards, e-portfolios, e-voting, peer/feedback software, VLE, online-offlineLearning quality and teaching efficienciesAssessment for learner self-regulationwww.reap.ac.uk
Background (1)
Gibbs, G. & Simpson, C (2004) Conditions under which assessment supports students’ learning, Learning and Teaching in Higher Education, 1, 3-31.
See:
Formative Assessment in Science Teaching (FAST) project at: http://www.open.ac.uk/science/fdtl
Gibbs and Simpson (2004)
Assessment tasks [Conditions 1-4]1. Capture sufficient study time (in and out of
class)2. Are spread out evenly across timeline of study3. Lead to productive activity (deep vs surface)4. Communicate clear and high expectations
i.e concern here is with ‘time on task’ how much work students do - their active engagement in study
Background (2)Literature Review
Nicol, D. & Macfarlane-Dick, D. (2006). Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Studies in Higher Education, 34 (1), 199-218
BackgroundStudent Enhanced Learning through Effective Feedback [SENLEF] project funded by HE AcademyREAP project: www.reap.ac.uk
Rethinking assessment and feedback1. Consider self and peers as much as the teacher as sources of assessment and feedback
Tap into different qualities than teacher can provide Saves time Provides considerable learning benefits (lifelong
learning)
2. Focus on every step of the cycle: Understanding the task criteriaApplying what was learned in action
3. Not just written feedback: Also verbal, computer, vicarious, formal and informal
Seven principles of good feedback
Good feedback:1. Clarifies what good performance is (goals, criteria,
standards).2. Facilitates the development of reflection and self-
assessment in learning 3. Delivers high quality information to students: that
enables them to self-correct4. Encourages student-teacher and peer dialogue
around learning5. Encourages positive motivational beliefs & self
esteem6. Provides opportunities to act on feedback7. Provides information to teachers that can be used
to help shape their teaching (making learning visible)
Source: Nicol and Macfarlane-Dick (2006)
EMPOWERMENT/SELF-REGULATION
ENGAGEMENT
Principle 1: Clarify what good performance is (the context of dialogue)
• Students create criteria• Students add own criteria • Students identify criteria
from samples of work• Exemplars of different
performance levels provided
• Students rephrase criteria in own words
• Provide document with criteria
EMPOWERMENT/SELF-REGULATION
ENGAGEMENT
Principle 2: Facilitate reflection and self-assessment in learning
• Students create MCQs including feedback for right and wrong (distractors) answers
• Students self-assess using MCQs and confidence ratings
• Students self-assess own performance using online MCQs
Two meta principles
1. Meta-PRINCIPLE 1: time and effort on task (structured engagement) i.e. steers on how much work to do and when – Gibbs and Simpson 4 conditions
2. Meta-PRINCIPLE 2: developing learner self-regulation (empowerment/self-regulation) i.e steers to encourage ownership of learning – the seven principles discussed above.
Key task for teacher is to balance 1 and 2
Example:Psychology
Psychology
560 first year students6 topic areas (e.g. personality, classical conditioning), 48 lectures, 4 tutorials, 12 practicalsAssessment; 2 x MCQs (25%), tutorial attendance (4%), taking part in experiment (5%), essay exam (66%)
Problems identified
No practice in writing skills but required in the examMore detail provided in lectures than mentioned in exams (not enough independent reading)No feedback except on Multiple Choice Questions (percent correct)Didn’t want to increase staff workload
Psychology Redesign
Discussion board in Learning Management SystemStudents in 85 discussion groups of 7-8, same groups throughout yearAlso open discussion board for classFriday lectures cancelled – discover material themselvesSeries of online tasks
Structure of group tasks
6 cycles of 3 weeks (one cycle x major course topic)
• First week: ‘light’ written task (e.g. define terms) = 7 short answers (all answer)
• Second week = guided reading • Week three: ‘deep’ written task: students
collaborate in writing a 700-800 word essay on the same topic.
Within each week:• The Monday lecture – introducing material• Immediately after lecture, task posted online –
for delivery the following Monday• Model answers (selected from students) posted
for previous week’s task
The teaching role
Participation in the discussions was compulsory but not marked (in subsequent years there was 2% mark for participation)Course leader provided general feedback to the whole class – often motivational He encouraged students to give each other feedbackThe group discussions were not moderated but monitored for participation
An example of ‘deep’ task
The Task – 800 word essay:
Assess the strengths and weaknesses of Freud’s and Eysenck’s theories of personality. Are the theories incompatible?readings suggestedquestions provided which all students should try
Relation to the Gibbs & Simpson’s four assessment conditions
1. Tasks require significant study out of class (condition 1)
2. Tasks are distributed across topics and weeks (condition 2)
3. They move students progressively to deeper levels of understanding (condition 3)
4. There are explicit goals and progressive increase in challenge (condition 4)
Relation to 7 feedback principles
1. Standard format and model answers provide progressive clarification of expectations (principle 1)
2. Students encouraged to self-assess against model answer (principle 2)
3. Course leader provides motivational and meta-level feedback and selects model answers (principle 3)
4. Online peer discussion aimed at reaching consensus is core feature of design about response (principle 4)
5. Focus on learning not just marks, sense of control/challenge enhanced motivation (principle5)
6. Repeated cycle of topics and tasks provide opportunities to act on feedback (principle 6)
7. VLE captures all interactions allowing course leader to monitor progress and adapt teaching (principle 7)
Benefits
Students worked exceptionally hard Written responses of exceedingly high standard Students took responsibility for learningHigh levels of motivation: atmosphere in class improvedOnline interactions showed powerful ‘scaffolding’ and community buildingFeedback with 560 students through peer and self-feedback (model answers)Easy for tutors to monitor participationImproved mean exam performance (up from 51-59%, p<0.01) weaker students benefit most
Has it worked?
I read more about Psychology and read it earlier in each semester than I would have done without
the online projects
0
10
20
30
40
50
60
1 2 3 4 5
agreed-disagreed
Nu
mb
ers
I found that reading other people's contributions helped me understand Psychology
010203040
50607080
1 2 3 4 5
agreed-disagreed
nu
mb
ers
Series1
I learned more in Psychology because of online projects than I did in my other subjects
0
10
20
30
40
50
60
1 2 3 4 5
agreed-disagreed
nu
mb
ers
Series1
Guidelines for Implementation
1. A single principle or many?2. Tight-loose – maintain fidelity to the principles
(tight) but encourage disciplines to develop their own techniques of implementation (loose)
3. Balance teacher feedback with peer and self-generated feedback
4. Focus on developing students’ own ability for critical evaluation
5. Create opportunities for ‘learning communities’ to emerge
6. The more actively engaged students are, the better the course design
Developments since REAP
1. Principles of Assessment and Feedback approved by University Senate and embedded in policy (2008)
2. Use of principles to inform curriculum renewal and Quality Assurance processes
3. ‘Feedback as Dialogue’ campaign to gain commitment of students
4. PEER Project (Peer Evaluation in Education Review)5. Work on the way we document our courses and
programmes
Peer Review in Education Evaluation [PEER]
The aims of the PEER project are to:Review evidence base for peer reviewDevelop educational designs for peer review (and self-review)Identify software support for peer reviewPilot implementations of peer review with large student numbersProduce guidelines for higher education – why do it, how to do it, pitfalls and solutions and software possibilities.
see http://www.reap.ac.uk/peer.aspx
PEER Project: Core Ideas
All graduate development in higher education requires that:
1. Students learn to evaluate critically the quality and impact of their own work (e.g. academic texts, problem solutions, designs)
2. Students learn to evaluate critically the work of others (e.g. peer review and peer feedback)
Ref:Nicol, D (2010) The foundation for graduate attributes: developing self-regulation through self and peer assessment. www.reap.ac.uk/peer.aspx
The research on peer feedback
Mainly about peers marking each others workWhere peer feedback is the focus it is usually about peers augmenting teacher feedback by increasing the quantity and variety of feedback
The focus of PEER project
Scenarios where students make evaluative judgements about the work of peers and provide a feedback commentary, usually writtenNot talking about .....collaborative tasks where students give each other informal feedback.....scenarios where the focus is on students evaluating each other’s performance in group working.....scenarios where the focus is on students grading/marking each other’s work although some rating of performance might be part of the peer design
Benefits of feedback construction (1)
Constructivist rather than transmission perspective
Producing is cognitively more demanding than reading feedback from others: students cannot be passiveStudents actively exercise assessment criteria (process/reprocess) from many different perspectivesSee how others tackle assignments and learn that quality can be produced in different waysDevelop writing skills through commentariesDeepens critical thinking in the discipline
Benefits of feedback construction (2)
Learn to assess own work, to monitor and evaluate their own productions - same skills involvedDevelops self-regulation – for life beyond university.Changes the power relationship in class (shares responsibility across students).Can develop learning communities
Giving and receiving feedback
Giving and receiving feedback within same domain enriches the whole process.Students are able to compare the feedback they receive with the feedback they produceAnd learn how different reviewers perceive their work – a single source of feedback is never satisfactoryCan develop a sense of shared responsibility across students for learning
Example: peer feedback
• Students write essay on one topic from three • Each student provides feedback on three essays in another topic
anonymously using rubric • The rubric: write a short summary of the essay, comment on and
rate (four point scale) the structure, arguments, evidence, writing, suggest ways of improving the essay.
• Students receive peer reviews of their own essays• Then review, comment on and rate their own essay using same
rubric.• Graded: for participating in the task, for their own essay and for
their review of it.• Finally students rate 3 reviews (on others’ work) and comment on
how useful they think they would be to author.• Software used to support administration
Six learning opportunities in peer review
1. Producing the target assignment2. Reviewing and producing peer feedback3. Receiving feedback from peers4. Responding to peer reviews (on own reviews of
those given to others)5. Self-reviewing one’s own work (e.g. after peer
review)6. Receiving feedback from the teacher – e.g. on
the assignment, peer review or self-review
1-3 comprise the essentials of peer review while 4-6 are further options.
Principles of effective peer reviewPeer review should
1. Engage students in active use of criteria and standards 2. Involve students in constructing commentaries in relation
to peer judgements, not just marks3. Lead to practice in both analytic (componential) and
holistic (configurational) judgements about quality4. Facilitate dialogue around the object and quality of the
review5. Ensure an atmosphere of mutual trust and respect6. Integrate self-review activities within peer review designs7. Ensure that making peer judgements is a regular activity
and not a one-off event
www.reap.ac.uk/PEER.aspx
Some of my PublicationsNicol, D (2010) The foundation for graduate attributes: developing self-regulation thorugh self and peer assessment, QAA Enhancement Themes, ScotlandNicol, D (2010) From monologue to dialogue: Improving written feedback in mass higher education. Assessment and Evaluation in Higher Education. 35(5), 501 -517 Nicol, D and Draper, S (2010) A blueprint for transformational organisational change in HE: REAP as a case study, Published by the Higher Education Academy, UK (see reap.ac.uk website)Nicol, D (2009) Transforming assessment and feedback: Enhancing integration and empowerment in the first year, Published by Quality Assurance Agency, Scotland (http://www.enhancementthemes.ac.uk/documents/firstyear/FirstYear_TransformingAssess.pdfNicol, D (2009), Assessment for learner self-regulation: Enhancing achievement in the first year using learning technologies, Assessment and Evaluation in Higher Education, 34(3), 335-352Nicol, D (2007), Laying the foundation for lifelong learning: cases studies of technology supported assessment processes in large first year classes, British Journal of Educational Technology, 38(4), 668-678Nicol, D (2007) E-assessment by design: using multiple-choice tests to good effect, Journal of Further and Higher Education.31(1), 53-64.Nicol, D. & Milligan, C. (2006), Rethinking technology-supported assessment in relation to the seven principles of good feedback practice. In C. Bryan and K. Clegg, Innovations in Assessment, Routledge.Nicol, D, J. & Macfarlane-Dick (2006), Formative assessment and self-regulated learning: A model and seven principles of good feedback practice, Studies in Higher Education, 31(2), 199-218.See www.reap.ac.uk/resources.aspx for copies.