Expertiza: Rubric-Based Peer Assessment of Team Projects Edward F. Gehringer Dept. of Computer...

Expertiza: Rubric-Based Peer Expertiza: Rubric-Based Peer Assessment of Team ProjectsAssessment of Team Projects

Edward F. GehringerDept. of Computer Science

North Carolina State University

Supported by NSF DUE under a CCLI grantNCSU Faculty Center for Teaching and Learning

NCSU STEM InitiativeCenter for Advanced Computing and Communication

04/20/23 Gehringer, Rubric-based peer assessment of team projects, efg@ncsu.edu

Outline Expertiza Rationale Team Assignments Expertiza “Virtual” Demo Forthcoming Features Workshop

Introduction Expertiza began as a peer-review system for co-

production of course resources. Building resources through electronic peer

review gets students working together to improve others’

learning experiences, helps them learn, by performing tasks that are similar

to real-world responsibilities, gives them experience in writing their ideas up for an

audience of their peers, allows each cohort to “stand on the shoulders” of

students in earlier classes.

Homework, traditionally … For students to demonstrate mastery of

the subject. Every student does the same thing—

redundant effort. Work is graded and thrown away, never

benefiting anyone but the student who did it. In Expertiza, students (or teams) sign up

for different work Now the best work can be reused, to

help others learn.

Major advantages Expertiza discourages plagiarism

Students are not doing the same work. Their work must go through multiple review

cycles Expertiza better utilizes scarce resources

Students do most of the reviewing. Instructor/TA spend less time grading.

With Expertiza, large classes are an advantage!

Why team assignments? Teamwork mimics the work environment Teaming skills are important to success Teams can tackle larger projects,

which diminishes the effort of integration

How does Expertiza support teams? All team members have access to a

common submission area. Any member can revise the submission.

Individuals review teams. This simplifies reviewing, and provides for more reviews.

Teammates assess each other’s contribution to the team. This score can be factored into the student’s

grade.

“Virtual” demo Team formation

By instructor, from the user interface By instructor, by importing a file By students, by issuing invitations

Reviewing Feedback from author to reviewer Teammate review

The login screen

Let’s log in as an instructor …

The instructor’s homepage

Setting up a wiki assignment

Let’s create some teams …

In the beginning, there are no teams

But let’s create one …

Team created; let’s add members

1st member is added; now for the 2nd …

Team now has two members

Importing teams

Here’s the file that’ll be imported

Here’s how we import it

Now there are five teams …

Students can create teams too …

Click on this assignment, Team test 2

Now, click on “Your team”

This user needs to set up a team

Let’s create a team

Invite another member to join …

Invitation has been sent

Now the invited user logs in

… and goes to Team test 2

Then accepts the invitation

… and is placed on the team

Advantages of various approaches Instructor-assigned teams

using class rank using demographics

Student-selected teams Forthcoming: Schedule-based team

formation

Let’s review a wiki page

Click on the assignment …

… then on “Others’ work”

Choose a review

Read the submitted wiki page

Then fill out the review template

Save and view the review

04/20/23 Gehringer, Rubric-based peer assessment of team projects, efg@ncsu.edu10/29/08 Gehringer: Expertiza: A Peer-Review System for Collaborative Learning, efg@ncsu.edu

After the Initial Review

Resubmission phase. 2–7 days to revise work in response to reviewer comments.

Final review. 3–7 days to make final comments and assign scores.

Metareview phase. Students review each other’s reviews.

Now the reviewee logs in …

… and views the scores

Click on “View” to expand review

Notice the link down at the bottom

Use it to give feedback to the reviewer

Then the reviewer can view the feedback

Review of teammates

Answer these questions …

Summary of feedback in Expertiza Reviews Feedback to author Teammate reviews Metareviews (x reviews y’s review of z) Surveys

All but surveys can influence the final grade

Instructor report

New Features Coming to Expertiza Game mechanics …

Achievements system Reputation system Leaderboards Levels

Message boards Students can ask for help; other students can

answer Microtasks/micropayments

Forthcoming: Achievement System Like in computer games. Categories in which achievement is tracked

Quality of submitted work Quality of reviews Contribution to team Helpfulness to other students

Prompt reviews/answers are more helpful, so reward for these …

Forthcoming: Message Boards Student a posts a question Student b responds Student a rates student b’s response Others can rate response too.

TAs can also respond Reliability …

Forthcoming: Microtasks/micropayments Desired enhancements posted

Along with amount of “payment” Students can voluntarily perform the

task. Students can also post tasks. How students should “pay” for these

For more info … http://research.csc.ncsu.edu/efg/expertiza Ed Gehringer, efg@ncsu.edu

Now, for the hands-on demo …

Edit a Wiki Page1. Go to [redacted]

2. You will be redirected to this page:

3. Log in with “unctlt#” and password “unctlt”.

4. Return to UNC_TLT, click on your user # and start editing.

5. Save your page.

Review a Wiki Page

1. After you save the page you were editing, log in at http://expertiza.ncsu.edu with the same username and password.

2. Click on “Others’ work”, then on “Begin” for one of the reviews.

3. Assign scores and give text comments.

Expertiza: Rubric-Based Peer Assessment of Team Projects Edward F. Gehringer Dept. of Computer...

Documents

Expertiza tehnica alimentara si raportul de expertiza pentru lapte

Expertiza Condimentelor.doc

O Melhor de Max Gehringer

expertiza scrisului

expertiza merceologica

Expertiza Contabila

expertiza financiara

referat expertiza

Suport Expertiza

Expertiza Si Doctrina 193 Din 645; Expertiza 2010x

Expertiza: Management and Peer Review of Student Team Projectsexpertizaatncsu.sourceforge.net/files/Gehringer... · Expertiza: Management and Peer Review of Student Team Projects

EXPERTIZA TEHNICA

Expertiza autotehnică

Manual Da Carreira Por Max Gehringer

Expertiza Ceccar

REGISTRUL DE STAT AL EXPERŢILOR JUDICIARI ATESTAŢI LISTA ... · - expertiza grafoscopică - expertiza tehnică ale actelor - expertiza traseologică - expertiza balistică - expertiza

Expertiza marfurilor

0Memoriu expertiza

Expertiza cascavalului

raport expertiza