Professor Beverley Oliver

Assuring Graduate CapabilitiesSetting the scene for

employability, engagement and quality education

http://tiny.cc/boliver

This PDF includes key slides from the Forum presentation: many pages contain clickable links so

you can find further information.

What got you to the table?

Academic transcript

Jack JamesBachelor of XYZ

Psychology 100: High distinctionScience 120: CreditFrench 134: PassMathematics 156: DistinctionBlah blahBlah blahBlah blah Blah blahBlah blahBlah blahBlah blah

This is to certify that

Jack Jameshas just scraped through the

Doctor of Philosophy

ResumeA creative portrait of

Jack James

your ticket to ride

Selection CriteriaJack James

...semi-fiction...

your digital footprintyour three best friends

Your last job interview

educatedengaged

employable

University of Anywhere Graduate of Anything

educated

engaged

employable

Key to graduate employability:

How can we assure that?

at the appropriate standard

Achievement of the capabilities that count

for early professional success

Inputs/Outputs(must haves)

Outcomes(how high)

Standards

the rise of the evidence-based culture

generic capabilitiesgraduate attributes

competencies

graduate outcomesessential learning outcomes

1. generic capability frameworks

Universities K-12 VET Business Council

COMMUNICATION

THINKING

PROBLEM SOLVING

INFORMATION LITERACY

SELF-MANAGEMENT

TEAMWORK

CIVIC ENGAGEMENT

Capabilities

COMMUNICATION

THINKING

PROBLEM SOLVING INFORMATION LITERACY  SELF-MANAGEMENT

TEAMWORK

CIVIC ENGAGEMENT See

ALTC Good Practice Guide:

Assuring Graduate Outcomes

VALUE

Subject benchmark statements

The Academic Infrastructure has four elements and all are interrelated:The Code of practice for the assurance of academic quality and standards in UK higher educationThe frameworks for higher education qualificationsSubject benchmark statementsProgramme specifications

Typical and threshold standards: 'Typical' = level attained by the majority of 'Threshold' = in positive terms the minimally acceptable level of attainment of an honours graduate.

Australian Qualifications Framework

Sample Performance TaskYou are the assistant to Pat Williams, the president of DynaTech, a company that makes precision electronic instruments and navigational equipment. Sally Evans, a member of DynaTech's sales force, recommended that DynaTech buy a small private plane (a SwiftAir 235) that she and other members of the sales force could use to visit customers. Pat was about to approve the purchase when there was an accident involving a SwiftAir 235. You are provided with the following documentation:1: Newspaper articles about the accident2: Federal Accident Report on in-flight breakups in single engine planes3: Pat's e-mail to you & Sally's e-mail to Pat4: Charts on SwiftAir's performance characteristics5: Amateur Pilot article comparing SwiftAir 235 to similar planes6: Pictures and description of SwiftAir Models 180 and 235

Please prepare a memo that addresses several questions, including what data support or refute the claim that the type of wing on the SwiftAir 235 leads to more in-flight breakups, what other factors might have contributed to the accident and should be taken into account, and your overall recommendation about whether or not DynaTech should purchase the plane.

just want the answers?

The following initiatives are currently underway in Australian higher education. Most are cross-sectoral, or multi-institution:• A sector-wide model for assuring final year subject and program achievement standards through inter-university

moderation; Professor Kerri-Lee Krause (Project Leader) k.krause@griffith.edu.au; Professor Geoff Scott g.scott@uws.edu.au

• Quality Verification System, Group of Eight; Professor Pip Patterson, pepatt@unimelb.edu.au• Achievement Matters: External Peer Review of Learning and Teaching Academic Standards for Accounting; Phil Hancock

(UWA) and Mark Freeman (Sydney) mark.freeman@sydney.edu.au• Teaching standards framework; Professor Judyth Sachs; provost@mq.edu.au• Hunters and gatherers: strategies for curriculum mapping and data collection for assuring learning; Dr Romy

Lawson Romy.Lawson@uts.edu.au• Harmonising higher education and professional quality assurance processes for the assessment of learning outcomes in

health; Professor Maree O'Keefe (Project Leader), maree.okeefe@adelaide.edu.au• Assessing and assuring Australian graduate learning outcomes: principles and practices within and across the

disciplines; Associate Professor Simon Barrie (Project Leader), simon.barrie@sydney.edu.au• Curriculum renewal and interprofessional health education: establishing capabilities, outcomes and

standards; Professor Alison Lee (Project Leader), Alison.Lee@uts.edu.au; Marie Manidis• Navigating a pathway between the academic standards and a framework for authentic, collaborative, outcomes-

focused thinking in Engineering Education Professor Wageeh Boles 2011 ALTC National Teaching Fellow, w.boles@qut.edu.au

• Developing the foundation for a national assessment of medical student learning outcomes; Professor David Wilkinson (Project Leader), david.wilkinson@uq.edu.au

• After standards: engaging and embedding history, Associate Professor Sean Brawley (Project Leader),s.brawley@unsw.edu.au

• Curriculum renewal in public health nutrition, Associate Professor Heather Yeatman (Project Leader),heather_yeatman@uow.edu.au

• Strengthening Australian legal education by integrating clinical experiences: identifying and supporting effective practices; Associate Professor Adrian Evans (Project Leader), adrian.evans@law.monash.edu.au

• National standards for psychological literacy and global citizenship, Associate Professor Jacquelyn Cranney 2010 ALTC National Teaching Fellow J.Cranney@unsw.edu.au

• Quantitative skills in science: curriculum models for the future, www.qsinscience.com.au  Contact: k.ma2hews1@uq.edu.au    

Map inputsEvaluate outcomes

Plan enhancementsBenchmarking portfolios: Benchmark, plan enhancements--and start again

Student portfolios: Evaluate achievement of capabilities in

portfolio approaches to teacher, self and peer

assessment

Map intended development and assessment of capabilities throughout the curriculum

Support student achievement of the capabilities through work-integrated learning

Determine the capabilities and standards for early professional success

Assurance of learning for graduate employability

Determine capabilities

Course review portfolios: Analyse indicators of

course quality including perceptions of graduate

achievement of capabilities

6.

5.

4.

1.

2.

3.

Expectations

Evidence

Assuring Graduate CapabilitiesAn Approach to Determining and Evidencing Standards

Graduate Employability

IndicatorsGRADUATES

EMPLOYERS

TEACHING TEAMS

14 capabilities (NSSE)

how important?

to what extent?

G

Oliver, B., Whelan, B., Hunt, L., & Hammer, S. (2011). Accounting graduates and the capabilities that count: Perceptions of graduates, employers and Accounting academics in four Australian Universities. Journal of Teaching and Learning for Graduate Employability, 2(1), 2 - 27.

316

99

51

!

Triangulated: what’s expected?

!developed

Triangulated: what’s evidenced?

But how well?

Learning standards

and provide examplars

1. Expectations

minimum standard

This is a description. This is not a marking rubric.

COMMUNICATION

THINKING

PROBLEM SOLVING INFORMATION LITERACY  SELF-MANAGEMENT

TEAMWORK

CIVIC ENGAGEMENT

Written and oral communication Critical and analytical and creative and reflective thinking Problem-solving, generating ideas and innovative solutions Use technology to find, evaluate and create information  Learning and working independently Learning and working collaboratively Ethical and inclusive engagement with communities, cultures and nations

Written Communication/ Oral Communication

Critical Thinking/Inquiry and Analysis

Problem Solving/ Creative Thinking/ Quantitative Literacy

Information Literacy/ Reading

Lifelong Learning/Integrative Learning

Teamwork

Civic Engagement/Ethical Reasoning/Intercultural

Fifteen templates available...

Work underway:143 rubrics51 coursesonline soon

The key question to students:

How have you satisfied, through your work, the aims [and standards] stated for your particular program of study?

Mantz Yorke

the student makes a case using evidence:

•marks or grades•qualitative assessments of performance (work placements)

•claims not formally assessable but supported by evidence

This requires a portfolio of achievements

Yorke, M. (2008). Grading Student Achievement in Higher Education: Signals and Shortcomings. Abingdon: Routledge. Available as Kindle edition @ Amazon

50%

50%

50% 50%

measurement

bottom up

marksgrades credits

Yr 1 Yr 2 Yr 3 Capstone Portfolio

Assessment

Capability 1: synthesis of evidence

Capability 2: synthesis of evidence

For marks or credit or notFormative or summative or bothTeacher or self or peer assessed or all

• Valid• Reliable• Robust• Efficient and

affordableAuthentic Auditable

Pathway portfolio

evidence of prior learning

evidence for entry

evidence for employment

eg AQF level 7

AQF level 9

AQF level 5

evidencing expectations in the

capabilities that count

at entry and exit

• marks or grades, artefacts• qualitative assessments of

performance (work placements)• claims not formally assessable but

supported by evidence • service learning• volunteering...