AHE Exemplars

Managing dialogue around exemplars

David Carless, Masterclass

AHE Manchester,

June 28, 2017

http://davidcarless.edu.hku.hk/

Twitter: @CarlessDavid

The University of Hong Kong

Overview

1. Exemplars: rationale and benefits

2. Challenges and how they might be tackled

3. Managing dialogue

4. Implementation suggestions

The University of Hong Kong

Teaching Enhancement Project

Enhancing dialogic use of exemplars amongst 10 teachers in a Faculty of Education

The University of Hong Kong

What are exemplars?

Samples used to illustrate dimensions of quality

Usually assignments from a previous cohort

The University of Hong Kong

Why used?

Exemplars convey messages that nothing else can (Sadler, 2002)

The University of Hong Kong

Promotes self-evaluation

Illustrate what good work looks like

Benchmark for comparison with own performance

The University of Hong Kong

Benefits for students

Gain experience in making judgments

Apply insights to own work & improve learning outcomes

The University of Hong Kong

Hendry et al., (2012)

Students positive about exemplars

Balanced teacher-led discussion of exemplars as crucial

The University of Hong Kong

Student views

Rubrics: vague, unclear and do teachers really use them?

Exemplars: useful & concrete; students want more of them

(Carless, 2015, 2017)

The University of Hong Kong

The University of Hong Kong

‘Model answer syndrome’

May reduce student creativity

May lead to copying

(Handley & Williams,

2011)

The University of Hong Kong

Scaffolded use of exemplars

Students work on assessment task prior to exposure to exemplars

The University of Hong Kong

Scaffolding steps

Instructional sequences combining:

- Student work in progress;

- Interaction with peers and teachers;

- Analysis of exemplars.

(Carless et al., 2018)

The University of Hong Kong

Time issues

• Collecting them

• Obtaining consent

• Time taken from instruction

• How many exemplars?

The University of Hong Kong

Content vs process

Too much time spent teaching content, insufficient attention to learning processes

The University of Hong Kong

It’s challenging!

Difficulties for students in evaluating exemplars accurately

The University of Hong Kong

Have I addressed challenges satisfactorily?

Any other thoughts or queries?

The University of Hong Kong

DIALOGIC USE OF EXEMPLARS

The University of Hong Kong

Key assumption

The quality of dialogue about exemplars is crucial

The University of Hong Kong

The University of Hong Kong

Research question

What are the main features of a specific exemplars dialogue and how is it orchestrated?

The University of Hong Kong

Context of the case

Participants:

• Trainee Science teachers

• BEd/BSc double degree

Teacher-researcher:

Dr Kennedy Chan

The University of Hong Kong

Data collection

• Classroom observation

• Open-ended student survey

• 2 focus group interviews

• Interview with teacher-researcher

• Teacher journal

• Student artefacts: ‘exit slips’, assignments etc.

The University of Hong Kong

Coding scheme

Main teacher moves:

- Eliciting student views

- Summarizing student views

- Elaborating student views

The University of Hong Kong

Exemplars provided

2 high quality exemplars

– Same format (i.e. reflective essays) but different

content focus

– Design aimed at reducing ‘copying’

The University of Hong Kong

Management of dialogue

1. Prior to class, students analyzed exemplars;

2. Students discussed exemplars in pairs;

3. Students elicited views from peers and made mini-presentations;

4. Teacher-orchestrated dialogue;

5. Students submitted exit slip of reflections.

The University of Hong Kong

Dilemmas

Students

constructing

notion of quality

VS

Telling students

about quality

Time for students

to talk with

peers

VS

Time for

developing shared

understandings

Exemplars as

guide

VS

Exemplars as

model

Using students’

voices

VS

Making

teacher’s voice

explicit

The University of Hong Kong

Good Exemplars dialogue

• Airs multiple & divergent viewpoints

• Shows linkages between peer talk & whole-class discussion

• Evidences development of student views

• Makes explicit some key qualities of exemplars

(Carless & Chan, 2016)

The University of Hong Kong

Implications

The University of Hong Kong

Implementation blueprint

Assessment task

Students devise or engage with criteria

for good task response

Two samples read before class

Peer discussionTeacher-led

dialogueStudent ownership

of insights

The University of Hong Kong

Exemplars & feedback

By developing capacities to make judgments,students are learning to decode feedback(Sadler, 2010)

Analysis of exemplars facilitates student uptakeof feedback (Carless et al., 2018; To & Carless,2016)

The University of Hong Kong

Engaging with quality

Student need to engage with what quality looks like and develop capacities in making judgments

The University of Hong Kong

COMMENTS,

QUESTIONS

The University of Hong Kong

References

Carless, D. (2015). Excellence in University Assessment: learning from award-winning teachers. London: Routledge.

Carless, D. (2017). Students’ experiences of Assessment for Learning. In D. Carless, S. Bridges, C.K.W. Chan & R. Glofcheski (Eds.), Scaling up Assessment for learning in Higher Education. Singapore: Springer.

Carless, D. & K.K.H. Chan (2016). Managing dialogic use of exemplars. Assessment and Evaluation in Higher Education,http://dx.doi.org/10.1080/02602938.2016.1211246

Carless, D., Chan, K.K.H., To, J., Lo, M. & E. Barrett (2018). Developing students’ capacities for evaluative judgement through analysing exemplars. In D. Boud, R. Ajjawi, P. Dawson & J. Tai (Eds), Developing Evaluative Judgement in Higher Education: Assessment for knowing and producing quality work. London: Routledge.

Handley, K. & Williams, L. (2011). From copying to learning: Using exemplars to engage students with assessment criteria and feedback. Assessment and Evaluation in Higher Education, 36(1), 95-108.

The University of Hong Kong

References

Hendry, G., Armstrong, S. & Bromberger, N. (2012). Implementing standards‐based assessment effectively: Incorporating discussion of exemplars into classroom teaching. Assessment and Evaluation in Higher Education, 37(2), 149-161.

Sadler, D. R. (2002). Ah! … So that’s ‘quality’. In P. Schwartz & G. Webb (Eds.), Assessment: Case Studies, Experience and Practice from Higher Education (p.130-136). London: Kogan Page.

Sadler, D. R. (2010). Beyond feedback: Developing student capability in complex appraisal. Assessment and Evaluation in Higher Education, 35(5), 535-550.

To, J. & Carless, D. (2016). Making productive use of exemplars: Peer discussion and teacher guidance for positive transfer of strategies. Journal of Further and Higher Education, 40(6), 746-764.

The University of Hong Kong

The University of Hong Kong

The University of Hong Kong

Productive assessment task design

Appreciating the nature of quality work

Student engagement with feedback

Learning-oriented assessment framework

The University of Hong Kong