Assessing to Support New Learning Outcomes:

Assessing for Graduate Capability Acquisition

Evaluations & Assessment 2003

Sally Kift, Assistant Dean QUT Law Faculty

QUT

Overview

Acknowledge input of large team on Large Grant.What’s the assessment issue?What are the areas on which we are focussing and why?Developing an assessment framework Later session on this today

Look briefly at the areas of concern.What we have achieved/still to do.Discussion.

QUT

What’s it all about?

In embedding graduate capabilities, questions arose re Validity and reliability of traditional assessment

practices and their transferability to the new imperatives of assessing authentic learning tasks.

How can we assure the quality of our assessment of students’ capability development?

The (tentative) answer is that - we should be able to assure the quality of assessment if it satisfies certain criteria (as distilled from the educational literature)

QUT

In four project areas: -

Identified 4 areas of capability development that have proved extremely challenging to assess –

1. Embedding Indigenous content and perspectives;

2. Development of oral communication (esp negotiation, advocacy, tutorial participation & interviewing for internal and external cohorts);

3. Infusion of ethical values and knowledge;4. Teamwork in large classes (internal and

external cohorts)

QUT

What does that mean?

Developed assessment framework to provide theoretical basis for identifying models of best practice Found assessment practices in the project areas both w/i and o/s QUT.

THEN – evaluate these assessment tasks against draft framework to distill relative strengths and weaknesses (some egs of evaluation on website)

This with a view to developing best practice models for trialling in 2nd stage Re Indigenous area, content reviewed as well

Session later today

QUT

Eg: Oral communication

5 specific communication skills being addressed: Interviewing Tutorial performance Negotiation Advocacy

OK

Oral presentations

•New criteria sheet; and

•Instructions to tutors & students (in core 1st yr)

Improved negotiation exercise in core 2nd yr

•New assessment criteria (core 1st yr);

•‘Student Assessment Handbook” for tutors & instructions for students;

•Student video (o/s grant)

QUT

Eg: Teamwork

Best practice models being developed for – In the classroom Outside the classroom (live interactive groups) In the virtual classroom (predominately electronic

contact)

Consider whether assessing process or productIf process models could include: Formative or summative; optional or compulsory Self, peer or teacher assessment Allocating individual or group marks

QUT

Eg: Ethics – Very challenging Connections with Indigenous project areaLegal ed currently utilises Bloom taxonomy to measure cognitive domain achievements (ie “professional” aspect of ethical knowledge) Audit revealed only one unit in Law (cf Justice) as having structured form of assessment that includes both cognitive and affective components.

Have had to devise a matrix of criteria and standards …2/

QUT

Ethics – Reloading the Matrix

The matrix of criteria and standards – referenced to observable characteristics or qualities in assessment materials which are themselves coupled to accepted taxonomies of achievement Attempt to maximise the dependability and reliability of assessment.Have used Facione & Facione (1994) and Bloom et al (1964)

QUT

The Matrix

Criteria mapped to increasing levels of achievement (Bloom 1 to 6) - ie leftmost criteria reflect “low-level” Bloom achievement (eg knowedge, comprehension), while rightmost criteria reflect highest level of Bloom achievement (synthesis, evaluation)

Standards mapped to increasing levels of achievement from Facione & Facione 1 –4 & referenced to observable & objective descriptors reflecting differences across Facione– eg “locates ALL rules’”(L4); “locates MOST rules” (L3); “locates SOME rules” (L2) & “does not locate rules” (L1)

CRITERIA

WarnerBros

QUT

Re Indigenous Project Area(With ors across QUT)

What is an Indigenous perspective (cf content): Access understanding through – the cultural construction of whiteness (eg explore

own racial position) & consultation with Indigenous persons (latter to move

beyond problematising and essentialising)Assessment: eg reflective assessment to chart attitudinal change informed by knowledge development and incorporating higher order analysis.Also Indigenous employment and staff development

QUT

Anything else?

Grant website a major dissemination strategy https://olt.qut.edu.au/law/ASSESSMENT/Ethics approval for the research undertaken by Project team members Staff survey and student focus groupPleasing no of pubs: eg 4xE1, 4xC1,15xATNIncluding Casual staff (2 RAs and training)Links across Faculty and across UniversityPositively received 2003 Law Discipline Review

QUT

Still TO DO:

A LOT!!!Trial the new materials Survey their success with students and staff and analyseRefine assessment frameworkProduce some training videos to assist in both teaching oral communication skills and in modelling application of assessment criteria.

QUT

The Pretty Website –

Assessing to Support New Learning Outcomes:

Assessing for Graduate Capability Acquisition

Evaluations & Assessment 2003

Sally Kift, Assistant Dean QUT Law Faculty