Review in Computerized Peer-Assessment

Review in Computerized Review in Computerized Peer-AssessmentPeer-Assessment

Dr Phil DaviesDr Phil DaviesDepartment of ComputingDepartment of ComputingDivision of Computing & Division of Computing & Mathematical SciencesMathematical Sciences

FATFATUniversity of GlamorganUniversity of Glamorgan

AUTOMATICALLY

CREATE A MARK THAT REFLECTS THE QUALITY OF AN ESSAY/PRODUCT

VIA PEER MARKING,

What do we need to provide to have fully Automated Peer –Assessment System?

AUTOMATICALLY

A MARK THAT REFLECTS THE QUALITY OF THE

PEER MARKING PROCESS i.e. A FAIR/REFLECTIVE

MARK FOR MARKING AND COMMENTING

SLIGHT GAP

THE FIRST CAP MARKING INTERFACE

Typical Assignment ProcessTypical Assignment Process Students register to use system - Students register to use system -

CAPCAP Create an essay in an area Create an essay in an area

associated with the module using an associated with the module using an RTF template of headingsRTF template of headings

Submit via Bboard Digital Drop-BoxSubmit via Bboard Digital Drop-Box Anonymous code given to essay Anonymous code given to essay

automatically by systemautomatically by system Use CAP system to markUse CAP system to mark

Self/Peer AssessmentSelf/Peer Assessment Often Self-Assessment stage usedOften Self-Assessment stage used

Set Personal CriteriaSet Personal Criteria Opportunity to identify errorsOpportunity to identify errors Get used to systemGet used to system

Normally peer-mark about 5/6Normally peer-mark about 5/6 Raw peer MEDIAN mark producedRaw peer MEDIAN mark produced Need for student to receive Need for student to receive

Comments + Comments + MarksMarks

CompensationCompensationHigh and Low MarkersHigh and Low Markers

Need to take this into accountNeed to take this into account Each essay has a ‘raw’ peer generated Each essay has a ‘raw’ peer generated

mark - MEDIANmark - MEDIAN Look at each student’s marking and Look at each student’s marking and

ascertain if ‘on average’ they are an under ascertain if ‘on average’ they are an under or over markeror over marker

Offset mark given by this valueOffset mark given by this value Create a Create a COMPENSATED PEER MARKCOMPENSATED PEER MARK

Below are comments given to students.Below are comments given to students.Select the 3 most Select the 3 most ImportantImportant to to YOUYOU

1.1. I think you’ve missed out a big area of the I think you’ve missed out a big area of the researchresearch

2.2. You’ve included a ‘big chunk’ - word for word You’ve included a ‘big chunk’ - word for word that you haven’t cited properlythat you haven’t cited properly

3.3. There aren’t any examples given to help me There aren’t any examples given to help me understandunderstand

4.4. Grammatically it is not what it should be likeGrammatically it is not what it should be like5.5. Your spelling is atroceiousYour spelling is atroceious6.6. You haven’t explained your acronyms to meYou haven’t explained your acronyms to me7.7. You’ve directly copied my notes as your answer You’ve directly copied my notes as your answer

to the questionto the question8.8. 50% of what you’ve said isn’t about the question50% of what you’ve said isn’t about the question9.9. Your answer is not aimed at the correct level of Your answer is not aimed at the correct level of

audienceaudience10.10. All the points you make in the essay lack any All the points you make in the essay lack any

references for supportreferences for support

Each Student is using a different set of weighted comments

Comments databases sent to tutor

First Stage => Self Assess own Work

Second Stage (button on server) => Peer Assess 6 Essays

Comments – Both Positive and Negative

in the various categories. Provides a Subjective Framework

for Commenting & Marking

Feedback IndexFeedback Index

Produced an index that reflects the quality Produced an index that reflects the quality of commentingof commenting

Produced a Weighted Feedback IndexProduced a Weighted Feedback Index

Compare how a marker has performed Compare how a marker has performed against these averages per essay for both against these averages per essay for both Marking + Commenting – Looking for Marking + Commenting – Looking for consistencyconsistency

-5-5 -4-4 -3-3 -2-2 -1-1 -0-0 +0+0 11 22 33 44 55 66 77 88 99

2929 4444 4141 4949 4646 5353 6464 4949 5353 6060 6262 6969 6868 6969 8282

3838 4848 4747 5151 4545 5454 5858 5353 6262 6262 6464 6565 7373

4949 5151 5050 6060 5757 5757 6767 6666

5151 5858 5353 5050 5959

5757 6363

5959 6565

6464

00 4.24.2 5.05.0 1.41.4 3.53.5 4.04.0 6.86.8 4.84.8 3.63.6 3.93.9 4.74.7 2.52.5 3.13.1 2.82.8 00

2929 4141 4545 4848 4949 4949 5656 5252 5656 5858 5959 6767 6464 7171 8282

The Review ElementThe Review Element

Originally in Communications within CAP Originally in Communications within CAP marking process, it requires the owner of marking process, it requires the owner of the file to ‘ask’ questions of the markerthe file to ‘ask’ questions of the marker

Emphasis ‘should’ be on the markerEmphasis ‘should’ be on the marker Marker does NOT see comments of other Marker does NOT see comments of other

markers who’ve marked the essays that markers who’ve marked the essays that they have markedthey have marked

Marker does not really get to reflect on their Marker does not really get to reflect on their own marking – get a reflective 2own marking – get a reflective 2ndnd chance chance

I’ve avoided this in past -> get it right first I’ve avoided this in past -> get it right first timetime

Click on button to get an essay

previously marked +

comments and marks

Click on button to get to view comments of

another marker

Can change any marks

and/or comments they feel

appropriate and submit by clicking button

Trialled with Post-Graduate Trialled with Post-Graduate GroupGroup

13 students 13 students 76 markings76 markings Average time per marking = 42 minutes (range 3-Average time per marking = 42 minutes (range 3-

72)72) Average number of menu comments/marking = Average number of menu comments/marking =

15.715.7 Peer Avge. mark = 59.69% (before review 60.15%)Peer Avge. mark = 59.69% (before review 60.15%) Number of students who did replacements = 10 Number of students who did replacements = 10

(out of 13) (out of 13) 41 ‘Replaced’ markings (54%) 41 ‘Replaced’ markings (54%) Out of 41 Markings ‘Replaced’ –> 26 changed mark Out of 41 Markings ‘Replaced’ –> 26 changed mark

26/76 (34%)26/76 (34%) Only 33 out of 41 REALLY CHANGED ANYTHINGOnly 33 out of 41 REALLY CHANGED ANYTHING 2 students ‘Replaced’ ALL his/her markings2 students ‘Replaced’ ALL his/her markings

-8, -7, -7, -7, -6, -6, , -6, -5, -5, -5, -4, -4, -3, -3, -2, --8, -7, -7, -7, -6, -6, , -6, -5, -5, -5, -4, -4, -3, -3, -2, -2, -2, -2, -1, -12, -2, -2, -1, -1

+1, +2, +6,+7, +8, +9+1, +2, +6,+7, +8, +9

++77

++66

++55

++44

++33

++22

++11

++00

-0-0 -1-1 -2-2 -3-3 -4-4

8181 6868 6262 7272 5353 6161 5252 4343 4343

7373 6666 5151

5151

Mapping of Feedback Indexes to Compensated Peer Essay Marks

# # S/AS/A Ref Ref S/AS/A

Raw Raw PeerPeer

Comp Comp PeerPeer

Raw Raw P/RevP/Rev

Comp Comp Peer Peer P/RevP/Rev

11 8383 8383 7070 7373 6767 7272

22 6060 6060 6969 6969 6969 6868

33 6363 6262 5454 5353 5454 5353

44 6363 6262 4747 5151 4747 5151

55 4545 4343 4545 4343

66 7070 6868 4949 5555 4646 5252

77 5757 5757 6464 6464 6161 6262

88 7171 6060 8686 8181 8686 8181

99 7070 5656 5151 5656 5151

1010 6969 6565 6161 6262 6161 6161

1111 7979 7474 6666 7171 6868 7373

1212 6666 5555 4848 4444 4848 4343

1313 6969 6565 7070 6565 6868 6666

-1

-1

-2

-9

-4

-5

-1

-4


Raw Raw PeerPeer

Comp Comp PeerPeer

Raw Raw P/RevP/Rev


11 8383 8383 7070 7373 6767 7272

22 6060 6060 6969 6969 6969 6868

33 6363 6262 5454 5353 5454 5353

44 6363 6262 4747 5151 4747 5151

55 4545 4343 4545 4343

66 7070 6868 4949 5555 4646 5252

77 5757 5757 6464 6464 6161 6262

88 7171 6060 8686 8181 8686 8181

99 7070 5656 5151 5656 5151

1010 6969 6565 6161 6262 6161 6161

1111 7979 7474 6666 7171 6868 7373

1212 6666 5555 4848 4444 4848 4343

1313 6969 6565 7070 6565 6868 6666

+3

-1

+4-2

+6

-5

-5

+1+5-4

-5


Raw Raw PeerPeer

Comp Comp PeerPeer

Raw Raw P/RevP/Rev


11 8383 8383 7070 7373 6767 7272

22 6060 6060 6969 6969 6969 6868

33 6363 6262 5454 5353 5454 5353

44 6363 6262 4747 5151 4747 5151

55 4545 4343 4545 4343

66 7070 6868 4949 5555 4646 5252

77 5757 5757 6464 6464 6161 6262

88 7171 6060 8686 8181 8686 8181

99 7070 5656 5151 5656 5151

1010 6969 6565 6161 6262 6161 6161

1111 7979 7474 6666 7171 6868 7373

1212 6666 5555 4848 4444 4848 4343

1313 6969 6565 7070 6565 6868 6666

+5-1

-1

+4-2

+6+1-5

-5

0

+5-5

-2

NEGATIVENEGATIVE POSITIVEPOSITIVE

88 77 66 55 44 33 22 11 11 22 33 44 55 66 77 88 99

11 33 22 33 22 22 44 22 11 11 00 00 00 22 11 11 11

AVERAGE MARK CHANGE = -1.69

Student Mark Changes During Review Stage

9 30-39

8 58-66

7 79-86

6 14-20; 40-46

8 67–59

7 73-67; 67-60; 60-53

6 73-67; 52-46

How to work out Mark (& How to work out Mark (& Comment) ConsistencyComment) Consistency

Marker on average Marker on average OVEROVER marks by marks by 1010%% Essay worth Essay worth 6060%% Marker gave it Marker gave it 7575%% Marker is Marker is 1515% over% over Would Would expect 10% overexpect 10% over, therefore Actual , therefore Actual

Consistency index (Difference) = Consistency index (Difference) = 55 If the marker on average had If the marker on average had UNDERUNDER marked by marked by

1010% - Difference would have been 2% - Difference would have been 255 Summing and Averaging these differences Summing and Averaging these differences

produces a Marking Consistency Index (low is produces a Marking Consistency Index (low is good – high is poor)good – high is poor)

This can be done for all marks and commentsThis can be done for all marks and comments

Pre-ReviewPre-Review Post-ReviewPost-Review

Student Student NumberNumber

Average Mark Average Mark DifferenceDifference

Mark Mark ConsistencyConsistency

Average Mark Average Mark DifferenceDifference


11 -5.17-5.17 6.856.85 -6.0-6.0 5.595.59

22 -1.14-1.14 5.495.49 0.140.14 4.554.55

33 0.830.83 2.552.55 0.50.5 2.312.31

44 1212 5.545.54 11.1411.14 4.174.17

5

66 6.676.67 7.467.46 77 7.597.59

77 -19.25-19.25 11.1911.19 -18.25-18.25 10.7810.78

88 -11.67-11.67 8.588.58 -9.83-9.83 4.544.54

9

1010 7.837.83 6.146.14 88 5.995.99

1111 4.674.67 4.164.16 2.172.17 3.313.31

1212 -4.83-4.83 8.418.41 -3.67-3.67 6.676.67

1313 -1-1 2.952.95 0.830.83 3.733.73

-1.26

-0.94

-0.24

-1.37

+0.13

-0.41

-4.04

-0.15

-0.85

-1.74

+0.78

Automatically Generate Automatically Generate Mark for MarkingMark for Marking

Linear scale 0 -100 mapped directly to Linear scale 0 -100 mapped directly to consistency … the way in HE?consistency … the way in HE?

Expectation of Normalised Results within Expectation of Normalised Results within a particular cohort / subject / institution?a particular cohort / subject / institution?

Map to Essay Grade Scale achieved Map to Essay Grade Scale achieved (better reflecting ability of group)?(better reflecting ability of group)?

Current ‘Simple’ methodCurrent ‘Simple’ method Average mark for essay e.g. 55%Average mark for essay e.g. 55% Ranges Highest – Lowest marks Ranges Highest – Lowest marks

achieved for essay e.g. 45% <-> 70%achieved for essay e.g. 45% <-> 70% Average Marking Consistency e.g. 5.0Average Marking Consistency e.g. 5.0 Ranges Highest – Lowest consistency Ranges Highest – Lowest consistency

indexes achieved e.g. 2.5 <-> 8.0 indexes achieved e.g. 2.5 <-> 8.0 Essay 45 <- (10) -> 55% <- (15) -> 70Essay 45 <- (10) -> 55% <- (15) -> 70Mark 8.0 <- (3) -> 5.0 <- (2.5)-> 2.5Mark 8.0 <- (3) -> 5.0 <- (2.5)-> 2.5Essay/Mark 3.33 / 1Essay/Mark 3.33 / 1 6.0 / 1 6.0 / 1e.g. Mark Cons = 6.0 > 1 below averagee.g. Mark Cons = 6.0 > 1 below averageMark for marking = 55% - (1*3.33)= Mark for marking = 55% - (1*3.33)=

51.66%51.66%



Consistency Consistency Difference Difference

from from Average Average 5.375.37

% Mark for % Mark for Marking based Marking based on range diff. on range diff.

(Avge 59.69%)(Avge 59.69%)

Mark they Mark they received for received for

essayessay

11 5.595.59 + 0 .22+ 0 .22 59%59% 72%72%

22 4.554.55 - 0.82- 0.82 65%65% 68%68%

33 2.312.31 - 3.06- 3.06 81%81% 53%53%

44 4.174.17 - 1.20- 1.20 68%68% 51%51%

5

66 7.597.59 + 2 .22+ 2 .22 53%53% 52%52%

77 10.7810.78 + 5.41+ 5.41 43%43% 62%62%

88 4.544.54 - 0.83- 0.83 65%65% 81%81%

9

1010 5.995.99 + 0.62+ 0.62 58%58% 61%61%

1111 3.313.31 - 2.06- 2.06 74%74% 73%73%

1212 6.676.67 + 1.30+ 1.30 56%56% 43%43%

1313 3.733.73 - 1.64- 1.64 71%71% 66%66%

What about the What about the commenting?commenting?

Does not take into account the Quality Does not take into account the Quality of the Commentingof the Commenting

Should look at the Average Feedback Should look at the Average Feedback Differences per marker to get a Differences per marker to get a Commenting Consistency GradeCommenting Consistency Grade

Same as creating Mark ConsistencySame as creating Mark Consistency

Create a Commenting Consistency Create a Commenting Consistency IndexIndex


Average Average Feedback Feedback DifferenceDifference

Feedback Feedback ConsistencyConsistency

Avge 2.80Avge 2.80

Consistency Consistency Difference Difference

from from Average Average

% Mark for % Mark for Comments based Comments based

on range diff. on range diff. (Avge 59.69%)(Avge 59.69%)

Mark they Mark they received for received for

essayessay

11 -2.10-2.10 3.053.05 +0.25+0.25 59%59% 72%72%

22 0.750.75 2.532.53 -0.27-0.27 63%63% 68%68%

33 4.434.43 2.942.94 +0.14+0.14 59%59% 53%53%

44 -1.30-1.30 2.212.21 -0.59-0.59 65%65% 51%51%

5

66 4.384.38 1.591.59 -1.21-1.21 75%75% 52%52%

77 -6.00-6.00 7.237.23 +4.43+4.43 43%43% 62%62%

88 -2.29-2.29 2.612.61 -0.19-0.19 62%62% 81%81%

9

1010 1.891.89 3.473.47 +0.67+0.67 57%57% 61%61%

1111 -0.62-0.62 1.071.07 -1.73-1.73 81%81% 73%73%

1212 0.440.44 2.412.41 -0.39-0.39 64%64% 43%43%

1313 -2.05-2.05 1.721.72 -1.08-1.08 73%73% 66%66%


MARK FOR MARK FOR MARKINGMARKING

MARK FOR MARK FOR COMMENTINGCOMMENTING

MARK FOR ESSAYMARK FOR ESSAY FINAL MARKFINAL MARK

60 / 20 / 2060 / 20 / 20

11 59%59% 59%59% 72%72% 67%67%

22 65%65% 63%63% 68%68% 66%66%

33 81%81% 59%59% 53%53% 60%60%

44 68%68% 65%65% 51%51% 57%57%

5

66 53%53% 75%75% 52%52% 57%57%

77 43%43% 43%43% 62%62% 54%54%

88 65%65% 62%62% 81%81% 74%74%

9

1010 58%58% 57%57% 61%61% 60%60%

1111 74%74% 81%81% 73%73% 75%75%

1212 56%56% 64%64% 43%43% 50%50%

1313 71%71% 73%73% 66%66% 68%68%





60 / 20 / 2060 / 20 / 20

11 59%59% 59%59% 72%72% 67%67%

22 65%65% 63%63% 68%68% 66%66%

33 81%81% 59%59% 53%53% 60%60%

44 68%68% 65%65% 51%51% 57%57%

5

66 53%53% 75%75% 52%52% 57%57%

77 43%43% 43%43% 62%62% 54%54%

88 65%65% 62%62% 81%81% 74%74%

9

1010 58%58% 57%57% 61%61% 60%60%

1111 74%74% 81%81% 73%73% 75%75%

1212 56%56% 64%64% 43%43% 50%50%

1313 71%71% 73%73% 66%66% 68%68%





60 / 20 / 2060 / 20 / 20

11 59%59% 59%59% 72%72% 67%67%

22 65%65% 63%63% 68%68% 66%66%

33 81%81% 59%59% 53%53% 60%60%

44 68%68% 65%65% 51%51% 57%57%

5

66 53%53% 75%75% 52%52% 57%57%

77 43%43% 43%43% 62%62% 54%54%

88 65%65% 62%62% 81%81% 74%74%

9

1010 58%58% 57%57% 61%61% 60%60%

1111 74%74% 81%81% 73%73% 75%75%

1212 56%56% 64%64% 43%43% 50%50%

1313 71%71% 73%73% 66%66% 68%68%

Correlation between

Commenting Consistency and Essay Mark 0.05

Final Grade for

Coursework takes into account Essay

Grade, Mark for Marking and Mark

for CommentingPercentag

es

Split in Marks 60 / 20 / 20Split in Marks 60 / 20 / 20

Is it reasonable?Is it reasonable? Higher Order skills of Marking – worth Higher Order skills of Marking – worth

more?more?

If we’re judging marking process on If we’re judging marking process on consistency – then should be consistency – then should be rewarded for showing consistency rewarded for showing consistency within marking AND commentingwithin marking AND commenting

Revised split 60 / 15 / 15 / 10Revised split 60 / 15 / 15 / 10

StudenStudent#t#

MARK MARK FOR FOR

MARKINMARKINGG

MARK FOR MARK FOR COMMENTINCOMMENTIN

GG

MARK FOR MARK FOR BEING BEING

CONSISTENT CONSISTENT IN MARKING & IN MARKING & COMMENTINGCOMMENTING

MARK FOR MARK FOR ESSAYESSAY

FINAL MARKFINAL MARK

15 / 15 / 10 / 6015 / 15 / 10 / 60

11 59%59% 59%59% 80%80% 72%72% 69%69%

22 65%65% 63%63% 72%72% 68%68% 67%67%

33 81%81% 59%59% 43%43% 53%53% 57%57%

44 68%68% 65%65% 68%68% 51%51% 57%57%

5

66 53%53% 75%75% 43%43% 52%52% 55%55%

77 43%43% 43%43% 81%81% 62%62% 58%58%

88 65%65% 62%62% 68%68% 81%81% 75%75%

9

1010 58%58% 57%57% 79%79% 61%61% 62%62%

1111 74%74% 81%81% 58%58% 73%73% 73%73%

1212 56%56% 64%64% 56%56% 43%43% 49%49%

1313 71%71% 73%73% 74%74% 66%66% 69%69%

Correlation between Final ESSAY grade & Mark for Marking, Commenting &

Consistency is 0.54

Correlation between FINAL MARK & ESSAY

GRADE is 0.85

StudenStudent#t#

MARK MARK FOR FOR

MARKINMARKINGG

MARK FOR MARK FOR COMMENTINCOMMENTIN

GG

MARK FOR MARK FOR BEING BEING

CONSISTENT CONSISTENT IN MARKING & IN MARKING & COMMENTINGCOMMENTING

MARK FOR MARK FOR ESSAYESSAY

FINAL MARKFINAL MARK

15 / 15 / 10 / 6015 / 15 / 10 / 60

11 59%59% 59%59% 80%80% 72%72% 69%69%

22 65%65% 63%63% 72%72% 68%68% 67%67%

33 81%81% 59%59% 43%43% 53%53% 57%57%

44 68%68% 65%65% 68%68% 51%51% 57%57%

5

66 53%53% 75%75% 43%43% 52%52% 55%55%

77 43%43% 43%43% 81%81% 62%62% 58%58%

88 65%65% 62%62% 68%68% 81%81% 75%75%

9

1010 58%58% 57%57% 79%79% 61%61% 62%62%

1111 74%74% 81%81% 58%58% 73%73% 73%73%

1212 56%56% 64%64% 56%56% 43%43% 49%49%

1313 71%71% 73%73% 74%74% 66%66% 69%69%

IS

IT

WORTH

THE

HASSLE??

Student CommentsStudent Comments

Have you used Peer-Assessment in PHave you used Peer-Assessment in Past?ast?

How did you find self-assessment?How did you find self-assessment? Creating the Comments Database?Creating the Comments Database? How did they find using the CAP systHow did they find using the CAP syst

em and peer-assessment?em and peer-assessment? Thoughts on new Review Stage?Thoughts on new Review Stage? Thoughts on Mark for Marking?Thoughts on Mark for Marking?

Two Main Points to ConsiderTwo Main Points to Consider

How do we assess the time required to How do we assess the time required to perform the marking task?perform the marking task?

What split of the marks between What split of the marks between creation & markingcreation & marking

DefinitionDefinition Student or Lecturer CommentsStudent or Lecturer Comments

Contact InformationContact InformationEmail: [email protected]: [email protected]

Phone: 01443 - 482247Phone: 01443 - 482247

Dr Phil Davies Dr Phil Davies

J317J317

Department of Computing & Department of Computing & Mathematical SciencesMathematical Sciences

Faculty of Advanced TechnologyFaculty of Advanced Technology

University of GlamorganUniversity of Glamorgan

THE END

General View of Peer AssessmentGeneral View of Peer AssessmentLecturer or Student?Lecturer or Student?

Lectures getting out of doing their jobs

i.e. marking

Good for developing student skills & employability

How can all students be expected to mark

as ‘good’ as ‘experts’

Why should I mark ‘properly’ and waste my time - I get a fixed mark

for doing it

The feedback given by students is not of the same standard

that I give.

The best thing I’ve ever done to make me

reflect

Defining Peer-AssessmentDefining Peer-Assessment

In describing the teacher ..In describing the teacher ..

A tall b******, so he was. A tall thin, mean A tall b******, so he was. A tall thin, mean b******, with a baldy head like a light bulb. b******, with a baldy head like a light bulb. He’d make us mark each other’s work, He’d make us mark each other’s work, then for every wrong mark we got, we’d then for every wrong mark we got, we’d get a thump. That way – he paused – ‘we get a thump. That way – he paused – ‘we were were

implicated in each other’s painimplicated in each other’s pain’’ McCarthy’s Bar McCarthy’s Bar (Pete McCarthy, 2000, page (Pete McCarthy, 2000, page 68)68)


Used Peer-Assessment in past?Used Peer-Assessment in past?

None to any real degreeNone to any real degree A couple for staff development type A couple for staff development type

activitiesactivities


How did you feel about performing How did you feel about performing Self-assessment?Self-assessment?

Very Difficult. Very Difficult. Helped to promote critical thinking Helped to promote critical thinking

ready for peer-assessment stage.ready for peer-assessment stage. Made me think about how I was going to Made me think about how I was going to

assess othersassess others


Creating Comments Database?Creating Comments Database?

Very difficult – not knowing what Very difficult – not knowing what comments they’d needcomments they’d need

Weighting really helped me create Weighting really helped me create criteria ready for markingcriteria ready for marking

Could have helped to do dummy Could have helped to do dummy marking marking


How did they find using the CAP system How did they find using the CAP system and peer-assessment?and peer-assessment?

Very positive & interestingVery positive & interesting Very time consumingVery time consuming Would do it better next timeWould do it better next time Important to maintain anonymityImportant to maintain anonymity Interesting & complex – thought more about Interesting & complex – thought more about

the assessment processthe assessment process Really helped student developmentReally helped student development


Thoughts on new Review StageThoughts on new Review Stage

Liked 2Liked 2ndnd chance to review own marks chance to review own marks Gained experience going through processGained experience going through process Didn’t really take much note of peers’ Didn’t really take much note of peers’

commentscomments Liked to see that others felt the same Liked to see that others felt the same

about an essayabout an essay


Thoughts on Mark for MarkingThoughts on Mark for Marking

Good rewarded appropriatelyGood rewarded appropriately Difficult to fully understandDifficult to fully understand Let owner of essay provide mark for Let owner of essay provide mark for

markingmarking

Documents

Review in Computerized Peer-Assessment