Standards for learning outcomes and course design Michael ... · TEQSA’s Evidence-based approach to regulation | AHEEF 29 October 2013 | Slide Slide 66 . Provider Registration Stds

TEQSA’s Evidence-based approach to regulation | Slide 1 AHEEF 29 October 2013 | Slide 1

Standards for learning outcomes and course design Michael Wells Commissioner 2013 Australasian Higher Education Evaluation Forum – 29 October 2013


Outline

What are the national standards? What do they mean in practice across

a diverse sector, including for TEQSA?

The role of indicators, benchmarking and expert review

Future developments in this area Note: Self-accredited / TEQSA

accredited


Introduction

This is at the heart of higher education

It will have the most direct impact on students, graduates and society

We are way past content It is about effective learning Effective measurement Community of scholarship and innovation System-wide opportunity and recognition


The national standards

The current Threshold Standards Provider Registration Standards Provider Category Standards Course Accreditation Standards Qualifications Standards

Australian Qualifications Framework Revision of standards by HESP


Provider Registration Stds

3.8 Corporate and academic governance arrangements demonstrate: the maintenance of academic standards, with

… external input, in accordance with international conventions for good academic practice; and,

effective quality assurance arrangements …, encompassing systematic monitoring, review and improvement.


Provider Registration Stds [2]

4.1 Objectives for higher education operations include the cultivation in students of critical and independent thought and the capacity for learning throughout life.

4.2 …promotes and protects free intellectual inquiry and expression in its higher education learning, teaching, and research activities.



5.2 …has the necessary staff positions, filled by appropriately qualified and experienced personnel …to achieve …expected student learning outcomes.



5.6 …compares its performance on teaching, student learning outcomes, graduate outcomes, and research with other higher education providers, and uses regular, valid and reliable feedback from internal and external stakeholders to improve …



6.5 …identifies and adequately meets the varying learning needs of all its students, including: the provision of orientation courses and

transition support; and, ongoing academic language and learning

support.


Provider Category Standards HEPs

1.3 …delivers teaching and learning that engage with advanced knowledge and inquiry.

1.4 …academic staff are active in scholarship that informs their teaching, and are active in research when engaged in research student supervision.


Provider Category Standards Australian University

2.4 …demonstrates the commitment of teachers, researchers, course designers and assessors to the systematic advancement and dissemination of knowledge.

2.5 …demonstrates sustained scholarship that informs teaching and learning in all fields in which courses of study are offered.


Provider Category Standards Australian University

2.6 …identifies and implements good practices in student teaching and learning, including …dissemination nationally.

2.9 …has systematic, mature internal processes for quality assurance and the maintenance of academic standards and academic integrity.


Course Accreditation Standards

CAS 1: Course design is appropriate − Takes account of external standards and requirements − Draws from substantial, coherent and current body − Emergent / specialised field to draw on more established bodies − Appropriate consideration of pathways − Coherence and engagement in intellectual inquiry at that level − Equivalent learning outcomes regardless of mode or place

CAS 2: Course resourcing and information is adequate

CAS 3: Admission criteria are appropriate

CAS 4: Teaching and learning are of high quality


Course Accreditation Standards [2]

CAS 5: Assessment is effective and expected student learning outcomes are achieved − Tasks provide opportunities to demonstrate outcomes − Assessment by appropriate qualified academics − Moderation etc ensures consistent and appropriate

assessment − Analyses and acts on comparative performance data − External benchmarking of intended/actual standards − Appropriate attrition, progression and graduate attributes

CAS 6: Course monitoring, review, updating and termination are appropriately managed − Systematic updated through internal revision and external

reviews − Protections for teachout or course transition


Qualifications Standards

QS 1: Higher education awards delivered meet the appropriate criteria

QS 2: Certification documentation issued is accurate and protects against fraudulent use

QS 3: Articulation, recognition of prior learning and credit arrangements meet the appropriate criteria


Australian Qualifications Framework

Learning outcomes of each qualification level and type, non-discipline-specific

Taxonomy focusses on Complexity and depth of achievement Autonomy required of graduates What graduates are expected to know,

understand and do Volume of learning (part of complexity of

type), as notional duration (‘normally’), expressed in equivalent full-time years


Australian Qualifications Framework [2]

AQF Issuance policy Nationally consistent and correct use of

AQF qualification titles Titles of AQF qualifications are the

representation of the qualification type, level and field of study / discipline of the qualification and provide the basis for national and international recognition.

AQF qualifications will have titles that unambiguously identify the qualification type, level and field of study/discipline


Australian Qualifications Framework [3]

AQF Pathways policy supports students’ lifelong learning explicitly about maximising credit builds pathways in to qualifications design Pathways are:

− clear and systematic − Flexible, including horizontal and vertical − credit for entry into as well as credit towards

Providers must have policies and processes to provide qualifications pathways and credit arrangements for students


Review of the Threshold Standards by HESP

Organising framework – a map of characteristics of higher education provision

Reference points (including learning outcomes for particular disciplines, professional accreditation)

HES should not operate at the level of specificity of individual disciplines / fields (discipline-free as AQF says)

Participation and attainment standards a critical focus


Review of the Threshold Standards by HESP [2]

Course design (cw) standards Defined process for design and QA Approval and oversight of peak academic

body Course design encompasses … Course and expected student learning is

consistent with qualification Content drawn from …, includes … Outcomes regardless of place / mode 5 yearly reviews auspiced by peak

academic body

http://www.hestandards.gov.au/sites/default/files/HESDraftStandardsforCourseDesign(Coursework)-March2013.pdf


Review of the Threshold Standards by HESP [2]

Learning outcomes (cw) standards LO specified and consistent with qualification Nationally comparable, internationally informed Taxonomy of LO

− Disciplinary knowledge and skills of field/s − Generic skills and attributes − Application of generic skills & attributes in field/s − Employment requirements related to field/s

Relationship between − unit LOs and overall LOs − types of LOs and assessment methods

5y reviews of LO, assessment, benchmarked grades

http://www.hestandards.gov.au/sites/default/files/HESDraftStandardsforLearningOutcomes(Coursework)-March2013.pdf


Other sources of standards

Professional accreditation ALTC / OLT projects Standards for Online Education

(draft)(Prof Mitch Parsell et al, OLT funded project)


In practice – course design

External input: academic experts; professional bodies; industry, market research

Coherence, body of knowledge, advanced inquiry

Appropriate learning outcomes mapped to appropriate assessment frameworks and tasks

Pathways – the student located in a system

Academic governance and quality assurance


In practice – learning outcomes

Direct Indirect Preconditions Assessment Employment Diversity - mapped Higher study Progression - opportunities Pathways Completion - QA techniques Recall Equivalence across

mode/place/segment Testing eg AHELO

QA Review

Tailoring for students at risk

Self evaluation – quant and qual

Learning analytics


Benchmarking

Similar course at same level In variety of institutions Comparing, analysing and

understanding Structure Cohorts Assessment tasks and grading Outcomes (direct, indirect, preconditions)

Taking action as a result


Roles of Expert Review

Design Structural design Learning outcomes design Assessment design Innovation

Resourcing Recruitment Learning resources

Review Moderation and post-hoc grading analyses Overall quality review Benchmarking


How TEQSA approaches a course accreditation

Case management

Application form, guide, RFI

Typically two experts in discipline

Site visit if necessary

New courses focus on plans, capabilities

Renewals focus on effectiveness of implementation and outcomes

Findings put to applicant if adverse

Commission decision


Future developments

Streamlining Greater reliance on track record More reliance on external reviews Evidence scoped with case manager Single commissioner delegate

HESP Revised standards Transition

Indicator development Direct / indirect / preconditions


Closing remarks

Standards debate is improving the quality of design and outcomes

Revealing and spreading good practices Rapid developments in some areas (some

disciplines, online, assessment, analytics) But some challenging issues: Market-ready products and lifecycles Diversity and innovation – ‘normally’ Measuring the ‘actual’ outcomes for graduates National and global recognition


Workshop themes

1. What are the best current measures of learning outcomes?

2. What would the best measures be in an ideal world, allowing for development?

3. How can course design be optimised to balance established knowledge, innovation, market need and qualifications recognition?

4. What processes are most critical in ensuring high quality learning outcomes?


Questions and Answers

Documents

Standards for learning outcomes and course design Michael ... · TEQSA’s Evidence-based approach to regulation | AHEEF 29 October 2013 | Slide Slide 66 . Provider Registration Stds