27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 1 Contextualising assessment: the lecturers’ perspective Lee Shannon, Bill Norton,

27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 1

Contextualising assessment: the lecturers’ perspective

Lee Shannon, Bill Norton, Lin Norton, Frances Phillips and Katherine Harrington*

Liverpool Hope University, UK.*London Metropolitan University, UK


Outline

• Context and research literature

• Aims of the research

• Research process and methodology

• Analysis of findings

• Conclusions and further research


Context of the research• Widely held view that assessment should be for

rather than of learning (Birenbaum et al, 2005)

• View that assessment in HE manifests many poor practices (Rust, 2007)

• Relationship between lecturers’ pedagogical beliefs and assessment practice remains under-researched area (exception Samuelowicz & Bain, 2002)

• Weston Manor Assessment manifesto (ASKe November 2007)


Theoretical background

• ‘Ideal’ versus ‘Actual’: disjunction between beliefs and practices is one of the mysteries of HE.

(Murray and MacDonald, 1997)

• ‘Espoused Theory’ versus ‘Theory in Use’

(Argyris and Schon,1974)

• Cognitive Dissonance

(Festinger 1957)


Research aimsTo elicit lecturers’ perceptions of assessment within the broader context of their philosophy of learning and teaching, specifically:

• ideal conditions for student learning• marking• feedback• emotional issues related to assessment• relationship between lecturers’ past

experiences and current practices


Methodology• In depth semi-structured interviews

• 29 lecturers in 18 disciplines at four universities in the UK

• Assessment experience ranged from 1-22 years.

• Interviews were transcribed verbatim

• Thematic analysis using QSR NVivo 7


Interviewees’ disciplines• Archive and Record Management

• Biology• Business• Cognitive Neuroscience

• Computer Studies• Education• English• Film Studies• Geography

•History•Human Resource Management

•Media Studies•Medicine•Political Science•Psychology•Religious Studies•Sociology•Theatre Studies


Representation of the themes• Next five slides represent our findings

visually• Not a model but a colour coded

representation of the main themes where:– Yellow is the main focus of the interview– Blue represents themes where there is a

dichotomy– Green represents the ‘ideal’ assessment

practices– Red represents the ‘actual’ assessment

practices


Relationship between pedagogical philosophy & assessment practices

Autonomy to change

delivery only

No autonomyto change

assessment

Institutionalrequirements Changing

withexperience

Autonomy to change

assessment

Completealignment

In an ideal world

Personal philosophies

Relationship


Strategic

Signposting

Range of students

Motivational

Fit for purpose

Pedagogically appropriate

Aids learning

Focus on grades

Assessmentfor learning

Assessment for learning


Mental models

Marking processes

Marking &assessment

criteria Highly regulated

Students’understanding

of criteria

Criteria useful for lecturers

Criteria usefulfor students

Connoisseurship

Marking

Marking


Feedback

Negativeaspects forlecturers

Theunmotivated

Timewith

students

Timely feedback

Allowsself

reflection

Motivates

Powerrelations

Feedback


Emotional issues related to assessment

Self doubt

Non supportive colleagues

Negativeaspects for

lecturer

Positivetwo-way

relationship

Rewarding forlecturer

Supportive colleagues

Powerrelations

Emotional issues


Development of Assessment, Marking & Feedback Skills

Models of good practice

ExperienceInformalTraining

A.M.F


Conclusions• Lecturers in this study articulated a personal

pedagogical philosophy of learning, teaching and assessment, but ….

• they gave competing accounts of the processes associated with assessment, marking and feedback.

• This finding supports the ‘ideal’ versus ‘actual’ dichotomy of Murray & Macdonald (1997) and the ‘espoused theory’ versus ‘theory in use’ of Argyris and Schon (1974).


Implications• For researchers:

– The interview process required lecturers to articulate what is often tacit, so may have engendered some cognitive dissonance which will be further researched.

• For staff developers:– Despite recognition by lecturers that

assessment can be problematic, there was still a collective resistance to formal training in assessment design, marking and feedback.


References• Argyris, C. and Schön, D. (1974) Theory in practice: Increasing

professional effectiveness, San Francisco: Jossey-Bass.• Assessment Standards Knowledge Exchange (ASKe)

http://www.brookes.ac.uk/aske/ (accessed 14 february 2011)• Birenbaum, M., Breuer K., Cascallar E, Dochy, F. Dori, Y. & Ridgway ,

J. in Wiesemes, R & Nickmans, G.(eds) (2006) position paper; A learning integrated system Educational Research Review 1 (2006) 61–67. Avaialble electronically: http://edu.technion.ac.il/chemical-education/judy/publications/no12_Assessment%20for%20learning%202006.pdf (accessed 14 February 2011)

• Festinger, L. (1957) Theory of cognitive dissonance. Stanford University Press

• Murray, K. & Macdonald, R. (1997) The disjunction between lecturers' conceptions of teaching and their claimed educational practice. Higher Education, 33, 331 - 349.

• Rust, C (2007) "Towards a scholarship of assessment" Assessment & Evaluation in Higher Education 32, 2, 229-237

• Samuelowicz, K & Bain,J.D. (2002) Identifying academics’ orientations to assessment practice, Higher Education, 43,2, 173-201

http://www.brookes.ac.uk/aske/

http://edu.technion.ac.il/chemical-education/judy/publications/no12_Assessment%20for%20learning%202006.pdf

http://edu.technion.ac.il/chemical-education/judy/publications/no12_Assessment%20for%20learning%202006.pdf

Documents

27-29 August 2008 Fourth Biennial EARLI/Northumbria Assessment Conference 1 Contextualising assessment: the lecturers’ perspective Lee Shannon, Bill Norton,