Learning Beyond the Horizon: Using Peerwise to increase engagement for students in transition

Learning Beyond the Horizon: Using Peerwise to increase engagement for students in transition

Eamon Costello Mark Brown

James Brunton Lorraine Delaney

National Institute for Digital Learning – Dublin City University

BackstoryBackstory

Backstory

Backstory

Learning online

Challenges of Teaching Programing

• Diverse student body• Some new to programing• Online/Flexible learning

Some Key Principles in Learning Programing

Feedback

Reflection

Practice

Kahneman, D. (2011). Thinking, fast and slow. Macmillan. Chicago

The Question

• How can we increase engagement?(and create a virtuous practice circle that includes reflection and feedback)

Feedback

Reflection

Practice

Principles of the Approach Taken“[The] biggest effects on student learning occur when teachers become learners of their own teaching, and when students become their own teachers”.(Hattie 2013, p. 22) • Test enhanced learning (Roediger & Karpickle, 2006): testing itself promotes recall and memorisation of knowledge providing testing includes feedback and is periodic

Hattie, J (2013) Visible learning: A synthesis of over 800 meta-analyses relating to achievement. RoutledgeRoediger, H. L., & Karpicke, J. D. (2006). Test-enhanced learning taking memory tests improves long-term retention. Psychological science, 17(3), 249-255.Chicago

Student Task

• Create Multiple Choice Questions on PHP Web programming topics according to question creation criteria

• Answer questions• Provide feedback• Rate your peers

• Required: number of questions to– create,– answer, – and comment-on/rate

How to ask

• Purchase, H., Hamer, J., Denny, P. and Luxton-Reilly, A. (2010) The quality of a PeerWise MCQ repository. IN: Proceedings of the Twelfth Australasian Conference on Computing Education-Volume 103. Australian Computer Society, Inc.

• Tarrant, M., Knierim, A., Hayes, S. K. and Ware, J. (2006) The frequency of item writing flaws in multiple-choice questions used in high stakes nursing assessments. Nurse Educ.Today. vol. 26, no. 8, pp 662-671.

Many educators are not good question authors (Tarrant et. al., 2006)But well guided students may be (Purchase et. al., 2010)

Student-friendly guide to best practice in authoring MCQ questions:– The question is clearly stated; – The question is error free; – The distractors (incorrect answers) are feasible; – The accompanying explanation is good; – the specified answer is correct.

Peerwise

• Peer assessment• Peerwise - online social learning environment

designed for peer assessment

Results

Requirement: Create 5 questions• 56% of the 27 students created more than 5 questions.

Engagement

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 270

5

10

Questions Created per Student

Required

Requirement: Answer 7 questions• 85% of the 27 students created more than 5 questions.

Engagement

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 270

50

100

7

Questions Answered per Student

Required

Engagement

Denny, P. (2013). The effect of virtual achievements on student engagement. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 763-772). ACM.Chicago

Large-scale (n > 1000) randomized, controlled experiment (Denny, 2013) found a significant positive effect:• on the quantity of student

contributions• the period of time over which

students engaged

Student Feedback

Student Feedback

“Peerwise depends on the knowledge of the person asking the questions. As a secondary teacher a lot of the people asking questions didn't balance the style of the question with the type of information required. People made questions unclear by trying to make them too hard. Also what could be done is if the people have two write questions in the future. Do so as they have to write 2 easy, 2 harder and one in depth question. This will allow them to develop their skills. It was an interesting experience and a nice break from coding. ”

Crowd-Sourced Assessment

Lessons Learned

• Peerwise very easy to use for students and well received

• Promotes engagement• Requires time thought to implement• Tip: Start simply and iterate

Questions?

More about Peerwise: http://peerwise.cs.auckland.ac.nz/

Me: eamon.costello@dcu.ie