37
Scaling up Assessment for Learning CITERS 2017, June 9 Professor David Carless University of Hong Kong The University of Hong Kong

CITERS Keynote

Embed Size (px)

Citation preview

Page 1: CITERS Keynote

Scaling up Assessment for Learning

CITERS 2017, June 9

Professor David Carless

University of Hong Kong

The University of Hong Kong

Page 2: CITERS Keynote

Overview

• Assessment for learning (AfL)

• AfL & MOOCs

• Peer feedback & technology

• Leadership & communities of practice

The University of Hong Kong

Page 3: CITERS Keynote

The University of Hong Kong

Page 4: CITERS Keynote

Chapter 1: Scaling up AfL: Progress & prospects (Carless)

Chapter 14: How does Technology enable Scaling up AfL? (Dawson & Henderson)

The University of Hong Kong

Page 5: CITERS Keynote

AfL definition

Assessment for which the

first priority is promoting students’

learning (Black et al., 2004)

Cf. Formative assessment

Learning-oriented assessment

(Carless, 2007, 2015a)

The University of Hong Kong

Page 6: CITERS Keynote

Interlinked AfL strategies

1. Productive assessment design

2. Students making judgments

3. Effective feedback processes

4. Developing student appreciation of quality

(Carless, 2017)

The University of Hong Kong

Page 7: CITERS Keynote

Scaling up (Coburn, 2003)

Spread

Depth

Sustainability

Shifts in ownership

The University of Hong Kong

Page 8: CITERS Keynote

Why scale-up AfL?

Research evidence

(Black & Wiliam, Hattie)

Dissatisfaction with current assessment & feedback practices

The University of Hong Kong

Page 9: CITERS Keynote

Black & Wiliam (1998)

Formative assessment & learning gains

Effect sizes: 0.4 - 0.7

… But implementation is generally weak

The University of Hong Kong

Page 10: CITERS Keynote

Hattie: Visible Learning The University of Hong Kong

Page 11: CITERS Keynote

MOOCs & AfL

The University of Hong Kong

Page 12: CITERS Keynote

MOOC Assessment design

Alignment of goals, activities & low-stakes assessment

MC quizzes + automated

feedback

Higher order thinking?

The University of Hong Kong

Page 13: CITERS Keynote

Peer assessment

Peer reviewed assignments + detailed rubrics as key feature of MOOCs

(Admiraal et al, 2015; Huisman et al., 2016)

The University of Hong Kong

Page 14: CITERS Keynote

Reliability of PA

Moderate reliability of peer assessment (Admiraal et al., 2015)

Use of multiple peer assessments to mitigate variance in judgments (Hew, 2016)

The University of Hong Kong

Page 15: CITERS Keynote

Peer feedback for learning

Learners gain more from composing than receiving peer feedback (Nicol et al., 2014)

The University of Hong Kong

Page 16: CITERS Keynote

Peer & Self-evaluation

Students did 5 peer reviews then self-evaluated own work

(Hew, 2016)

The University of Hong Kong

Page 17: CITERS Keynote

FEEDBACK PROCESSES & TECHNOLOGY

The University of Hong Kong

Page 18: CITERS Keynote

Audio & Video feedback

Rapport

Nuance

Personalisation

Monologue or Dialogue?

Time saver?

The University of Hong Kong

Page 19: CITERS Keynote

Peer video feedback

Peer-to-peer video feedback

delivered via Facebook

Hung (2016)

The University of Hong Kong

Page 20: CITERS Keynote

Personalised feedback at scale

Using learning analytics to

scale the provision of

personalised feedback

https://www.ontasklearning.org/

The University of Hong Kong

Page 21: CITERS Keynote

Sustainable feedback

Students generating & using feedback from peers or self as part of self-regulated learning (Carless et al., 2011)

The University of Hong Kong

Page 22: CITERS Keynote

DEVELOPING STUDENT APPRECIATION OF

QUALITY

The University of Hong Kong

Page 23: CITERS Keynote

MOOC limitation?

MOOCs do not systematically develop student understanding of quality

(Dawson & Henderson, 2017)

The University of Hong Kong

Page 24: CITERS Keynote

Using exemplars The University of Hong Kong

Exemplars convey messages that nothing else can

(Sadler, 2002)

Page 25: CITERS Keynote

The University of Hong Kong

Page 26: CITERS Keynote

Appreciating quality

Students need to debate nature of quality & develop capacities in making judgments

The University of Hong Kong

Page 27: CITERS Keynote

SCALING UP POSSIBILITIES

The University of Hong Kong

Page 28: CITERS Keynote

Enhancing design

Adding technology to existing design & expecting improvements is flawed

Invest in improved assessment & feedback designs to leverage gains from technology

(Dawson & Henderson, 2017)

The University of Hong Kong

Page 29: CITERS Keynote

Leadership

Resourcing & support

Incentives & rewards

The University of Hong Kong

Page 30: CITERS Keynote

TELI

Technology-Enhanced Learning Initiative

The University of Hong Kong

Associate Vice-President (Teaching and Learning) Ricky Kwok

Page 31: CITERS Keynote

Incentives

Teaching Innovation Award

Teaching Feedback Award

The University of Hong Kong

Page 32: CITERS Keynote

Communities of practice (CoPs)

Surfacing and sharing AfL practices (Hounsell & Zou, 2017)

The University of Hong Kong

Page 33: CITERS Keynote

Staff development

Good CoPs

Dialogue & communication

The University of Hong Kong

Page 34: CITERS Keynote
Page 35: CITERS Keynote

References

Admiraal, W., Huisman, B., & Pilli, O. (2015). Assessment in massive open online courses. The Electronic Journal of e-Learning, 13(4), 207-216.

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black box: Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 8-21.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education, 5(1), 7-74.

Carless, D. (2007). Learning-oriented assessment: Conceptual basis and practical implications. Innovations in Education and Teaching International, 44(1), 57-66.

Carless, D. (2015a). Exploring learning-oriented assessment processes. Higher Education, 69(6), 963-976.

Carless, D. (2015b). Excellence in University Assessment: learning from award-winning teachers. London: Routledge.

Carless, D. & K.K.H. Chan (2016). Managing dialogic use of exemplars. Assessment and Evaluation in Higher Education, http://dx.doi.org/10.1080/02602938.2016.1211246

Carless, D., Salter, D., Yang, M., & Lam, J. (2011). Developing sustainable feedback practices. Studies in Higher Education, 36(4), 395-407.

Coburn, C. (2003). Rethinking scale: Moving beyond numbers to deep and lasting change. Educational Researcher, 32(6), 3-12.

The University of Hong Kong

Page 36: CITERS Keynote

References

Hattie, J. (2009). Visible learning. London: Routledge.Hew, K. F. (2016). Promoting engagement in online courses: What strategies can we

learn from three highly rated MOOCS. British Journal of Educational Technology, 47(2), 320-341.

Hounsell, D. & Zou, T. (2017). Surfacing and Sharing Advances in Assessment: A Communities-of-practice approach. In D. Carless, S. Bridges, C.K.W. Chan & R. Glofcheski (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer.

Huisman, B., Admiraal, W., Pilli, L., van de Ven, M., & Saab, N. (2016). Peer assessment in MOOCs: The relationship between peer reviewers’ ability and authors’ essay performance. British Journal of Educational Technology. Doi: 10.1111/bjet.12520.

Hung, S.-T. A. (2016). Enhancing feedback provision through multimodal video technology. Computers & Education, 98, 90-101.

Nicol, D., Thomson, A., & Breslin, C. (2014). Rethinking feedback practices in higher education: a peer review perspective. Assessment & Evaluation in Higher Education, 39(1), 102–122.

Sadler, D. R. (2002). Ah! … So that’s ‘quality’. In P. Schwartz & G. Webb (Eds.), Assessment: Case Studies, Experience and Practice from Higher Education (p.130-136). London: Kogan Page.

The University of Hong Kong

Page 37: CITERS Keynote

THANK YOU

The University of Hong Kong