Outcomes-Based Programme Development and Quality Assurance at Masaryk University1 Outcomes-Based Programme Development and Quality Assurance at Masaryk

Outcomes-Based Programme Development and Quality Assurance at Masaryk University

1


Soňa BasovníkováJiří Nantl

DECOWE International Conference24 – 26 September 2009, Ljubljana, Slovenia


2

Trends in Tertiary Education: Why do we introduce Qualifications Framework and Learning Outcomes?

overall move towards universal higher educationchanges in students structurechanges in structure of higher education study programmesvarious perceptions on purposes and content of higher education in general growing demands of public authorities on accountability and possibility to assess and measure higher education‘ s outcomesinternationalization and mobility


3

Qualification, qualification frameworks, learning outcomes:

Qualification - a formal certification of learning outcomes achieved. Qualification frameworks – generic description of competencies/learning outcomes that a student must demonstrate in order to reach a qualification. learning outcomes - statements of what a learner is expected to know, understand and/or be able to demonstrate after a completion of a process of learning (e.g. course, module, programme)

Adelman C. Bologna Process For US Eyes. 2009


4

Qualifications framework

A „warranty“ of higher education (C. Adelman):Students must demonstrate to the HEI that they have achied the intended learning outcomesInstitutions must demonstrate to the government that their students have demonstrated the achievement of intended learning outcomesIt is designed not only for HE institutions, academic staff, and students, but also the government and general publicA tool for understanding what the higher education is meant to be about


5

Role of academic staff in the use of qualification frameworks

Academic staff has an irreplaceable responsibility for creation, development and assessment of study programmes:they formulate objectives of the study programme and translate them into measurable learning outcomesthey choose corresponding teaching methods and they assess students‘ performance


6

Programme‘s objectives/aims vs. learning outcomes (1/2)

Programme‘s objectives are broad general statements comprising:

philosophy, values, and theoretical-methodological background of the discipline;identification of those students for which study programme is suitable;graduates profile and their main characteristics; possibilities of employment

Programme‘s objectives should be formulated in language

comprehensible for prospective students and their parents. Usually their are not measurable.


7

Programme‘s objectives/aims vs. learning outcomes (2/2)

Learning outcomes (LOs) represent operationalization of programme‘s objectives/aims They are formulated with „active verbs“ (Bloom‘s taxonomy) LOs are measurableThere is an alignment among LOs, teaching and leaning methods, and assessment Intended LOs refer to minimum threshold requirements on students (how well student fulfil these requirement is reflected in assessment criteria and grading scale)


8

Advantages of learning outcomes:

help students to reflect their own learning and therefore learn more effectivelyhelp teachers to tell students more precisely what is expected from themprovide information to prospective students and employers about key graduates competenciesfacilitate comparison of individual study programmes for purposes of international recognition together with the criterion of workload enable to determine credit value of a given period of study clearly define study programme for purposes of accreditation and evaluation


9

Formulating learning outcomes (1/4)

They should be formulated as a result of an explicit agreement of concerned academic community and their idea about study content and typical graduate They should take into consideration trends in a given discipline and expectations of the societyDecision on what won‘t be included in the curriculum is as much important as a decisions on what will be the content


10

Formulating learning outcomes(2/4)

LOs represent measurable knowledge, understanding or abilities which are the object of assessmentLOs are designed on two levels: programme and courseLOs do not have to (and cannot) cover everything what student will learn during period of study Each compulsory course should contribute to one of

study programme‘ s learning outcome


11

How formulate learning outcomes (3/4)

Begin each LO with an action verb, followed by the object of the verb followed by a phrase that gives the context.Use only one verb per LO, avoid complicated sentences.Avoid vague terms like know, understand, learn… Ensure that the LOs of the course relate to the overall outcomes of the programme.LOs must be observable and measurable and capable of being assessed.Ask yourself if it is realistic to achieve the LOs within the time and resources available.Before finalising the LOs, ask your colleagues and possibly former students if the learning outcomes make sense to them.

Kennedy, Hyland, Ryan: "Writing and Using Learning Outcomes: a Practical Guide"


12

Alignment of the study programme‘s objectives and course‘s structure:

Formulation of study programme‘s

objectives

Formulation of learning outcomes

Choose appropriate teaching and assessment

methods

Alignment to the structure of courses


13

Study programme‘s assessment at Masaryk University:

Study programme‘s assessment is part of student-centered approach Programme level assessment covers not only academic quality but also sustainability of the programme, and its contribution to the profile and mission of the universityPart of the university strategy since 2005, pilot projects in 2008, a university-wide approach since 2009A continual process involving all the academic staff (no bureaucratic exercise)


14

Principles of study programme‘s assessment

Formative assessment – each programme should be given feedback so as to enable improvement. There should be no „ranking“ or „rating“ of programmes within university.Non-bureaucratic approach – quality assurance and monitoring are an integral part of academic staff job description (under university regulations since 2008)Balanced assessment – the self assessment report by the programme is reviewed by a group of external examiners (graduates, employers, and academic colleagues)


15

Process of study programme evaluation (1/5)

Self-evaluation report contains:Strategic goals: strategy of study programmes‘ s development for next 5 years (human and material resources, competitiveness, etc.)Educational goals: what a graduate is expected to know, understand and/or be able to demonstrate after a completion of a process of learning

Self-assessment report is to reflect strenghts, weaknesses,

opportunities and threats to the programme, using the SWOT analysis technique


16

Process of study programme evaluation(4/5)

Self-assessment report is subject to a review by group of external examiners:An academic colleague outside the university (or, exceptionally, from outside the faculty involved) A graduate (or, exceptionally, a student)An employerA representative of the university leadership (vice rector, or a member of the Study Programmes Board)


17

Process of study programme evaluation (5/5)

At the end of the assessment procedure, a joint meeting of the self-assessment team and external examiner enables the fair exchange of viewsPilot projects in 2008 showed that many academic staff sought support from external examiners views to foster curricular and didactic changes unpopular with the rest of the faculty Final recommendation submitted to the faculty management should be, as far as possible, based upon understanding between the self-assessment team and external examiner


18

Benefits of study programme`s assesment

Increased transparency of the study programme: helps students to follow learning process and academic staff to clear up their aims Verification of study programme‘s consistency and logics – identification of possible gaps or overlaps Setting strategic goals for study programmesNot a ranking but focus on reflection, recommendations for development and improvement


19

Thank you for your attention

Soňa Basovníková Head, Office for Strategy

[email protected]

Jiří NantlChief Legal and Policy Officer

[email protected]

Documents

Outcomes-Based Programme Development and Quality Assurance at Masaryk University1 Outcomes-Based Programme Development and Quality Assurance at Masaryk