Expertiza: Improving Course Materials and Learning Outcomes through Peer Review of Student Work

Aug. 9, 2007 Gehringer: Improving Course Materials … Through Peer Review … efg@ncsu.edu

Expertiza: Improving Course Materials Expertiza: Improving Course Materials and Learning Outcomes through Peer and Learning Outcomes through Peer

Review of Student WorkReview of Student Work Edward F. GehringerDept. of Computer Science

North Carolina State University

Supported by NSF DUE under a CCLI grantNCSU Center for Teaching and Learning

NCSU LITRE (Learning in a Technology-Rich Env.)

Outline Introduction Expertiza Rationale Demo Experiment 1: Improving a Textbook Experiment 2: Review Wiki Submissions

Introduction Electronic peer review is students reviewing

other students’ work over the Web. Building resources through electronic peer

review gets students working together to improve others’

learning experiences, helps them learn, by performing tasks that are similar

to real-world responsibilities, gives them experience in writing their ideas up for an

audience of their peers, allows each cohort to “stand on the shoulders” of

students in earlier classes.

Homework, traditionally … For students to demonstrate mastery of

the subject. Every student does the same thing—

redundant effort. Work is graded and thrown away, never

benefiting anyone but the student who did it. Now the best work can be published, to

help others learn.

Outline Introduction Expertiza Rationale Demo Experiment 1: Improving a Textbook Experiment 2: Review Wiki Submissions Conclusion

The Rationale Improve student learning Improve teaching Better utilize resources

Improving student learning The Expertiza platform improves student

learning in these ways: Integrates active and cooperative learning

Active learning allows students to take responsibility for their own learning.

Extends active learning to out-of-classroom activities and distance education

DE has been a roadblock to the use of active learning … students viewing lectures remotely can work only by themselves.

Discourages plagiarism Multiple deadlines and milestones make it impossible

to submit a finished product obtained from an external source.

Improving teaching The Expertiza platform improves teaching in

these ways: Increases the supply of examples/homework

problems/test questions Students are assigned to make up such

examples/questions, and these are peer reviewed.

Focuses students on explaining/understanding the concepts that are hardest to master

“Write an example to clarify the hardest concept in lecture k.”

Improving resource utilization Some work is peer-graded, so teaching

assistants can spend more time working with students and less time grading. Having inadequate TA support no longer limits the

amount and kinds (e.g., design problems) of homework that can be assigned.

Students rely more on their peers for help, less on the course staff.

Makes teaching large classes an advantage! Large classes can produce more and better resources.

Improving an OOD Text Our class

Master’s level course CS & ECE Substantial DE enrollment

In Fall 2005, we used a new object-oriented design text for the first time, Dale Skrien’s, An Intrduction to Object-

Oriented Design and Design Patterns Using Java

Three Homework Assignments

Improve an explanation of a topic covered in the text.

Create a new example of a concept covered in the text.

Write a new exercise for a chapter in the text.

All students did not do these exercises in the same order.

The Review Process: How It Begins

Classroom discussion of the most difficult topics

Students select a topic from a list. Several students are allowed to select the

same topic, But the number of slots is limited.

Signing up for a First-Round Topic

Student Choices

The Review Rubric Students review a submission based on the following

questions (for examples). Does the example fully illustrate the concept being

explained? Is the example easy to understand, i.e., as clear as it

could be and still illustrate the concept? Does the example model the real world, i.e., could it be

implemented in practice? Is the example code elegant? Does the example use up-to-date Java code or UML? Does the text that accompanies the example explain it

well? Is the example more useful than the examples in this

section of the book? Does the example seem to be original? Other: How would you rate the submission on factors

not reflected in the scores on other questions?

After the Initial Review

Resubmission phase. 2–7 days to revise work in response to reviewer comments.

Grading phase. 3–7 days to make final comments and assign scores.

Review of review phase. Students review each other’s reviews.

Results

17 student submissions selected for text!

I learned a lot from doing the peer-reviewed

assignments related to the text.

I enjoyed doing the peer-reviewed assignment

related to the text.

Teaching with Wikis A wiki is, essentially, a Web site that can

be edited by any user. Homework done on wikis promotes

collaboration between students. Problem: How to assess so much writing. Solution: Peer review.

Our Approach Students select from a set of topics for a weekly

wiki assignment. The students are given class time to work on

the topic in groups. Then they take a couple of days to finish up

their submission. Their submission is peer-reviewed by other

students. They have a chance to revise their submission. Students rate the contributions of their

partners. Instructor considers the reviews, revisions, and

partner evals in assigning grades.

Students Seem to Benefit Wiki seems to be preferred as a submission

mechanism. “The wiki was really fantastic for creating pages

because it guarantees a uniform style for everything, putting the major focus on the content created, rather than formatting issues. It also helped format content so that it was more understandable.”

Contributions are more extensive than we have seen when files are submitted and resubmitted.

Peer review is the only scalable solution. Expertiza gives a framework for review.

Other Uses Lecture annotations Code reviews: Contributions to OSS

projects Class “proceedings” in a research course

Survey papers or research papers FAQ in a service-learning course

Future Work “Peer-assisted” review Total Quality Management Automated reviews of reviews

Conclusion Students can do good work—

work that will help their classmates learn the material

work that will help the instructor improve the class

work that allows each class to “stand on the shoulders” of earlier classes.

The Expertiza approach enables faculty to introduce these exercises into their courses.

For more info … http://research.csc.ncsu.edu/efg/expertiza Ed Gehringer, efg@ncsu.edu

Expertiza: Improving Course Materials and Learning Outcomes through Peer Review of Student Work

Documents

expertiza judiciara

Peer Implementing Specialist Peer Support Services …...Peer support provides a wide variety of improved outcomes for Veterans . Research and experience have shown how peer support

expertiza merceologica

expertiza ecologica

REGISTRUL DE STAT AL EXPERŢILOR JUDICIARI ATESTAŢI LISTA ... · - expertiza grafoscopică - expertiza tehnică ale actelor - expertiza traseologică - expertiza balistică - expertiza

How To Guide: Peer Education To Guide... · The role and desired outcomes for the peer education ... Learning outcomes for ... content focused on the FYE target learning areas, planning

Botosani Expertiza

Rasp Expertiza

Expertiza Condimentelor.doc

Peer-to-peer lending and birth outcomes during national ...thew.web.unc.edu/files/2013/08/acquah.pdfPeer-to-peer lending and birth outcomes during national economic crises: Lessons

Curs Expertiza

Peer Outcomes Protocol (POP)Response Card 8 Service Use Module Peer Outcomes Protocol Service Use Peer O Community Services At Peer Support Program Outside of Peer Program At Peer

grile expertiza

Expertiza accidentelor

Aug. 9, 2007Gehringer: Improving Course Materials … Through Peer Review … efg@ncsu.edu Expertiza: Improving Course Materials and Learning Outcomes through

0Memoriu expertiza

referat expertiza

Expertiza contabila.ppt

Measuring Outcomes of Peer Support: What Have We Measured

Expertiza Si Doctrina 193 Din 645; Expertiza 2010x