Learning Outcomes

Learning Outcomes

Learning Theory: How to Teach, Learn, and Assess?

Some Basic Concepts

Methodology: How? – Theory and ProceduresGuidelines for Educational Design

2

Starting Point: What are Learning Outcomes? Motivation and Considerations

Overview

Moscow

Assessment: How? – Different Types of

Assessments

Concordance & Alignment

Learning Outcomes 3

Starting Point:

Introduction: What are Learning Outcomes?

Moscow

Learning Outcomes 4

Learning outcomes are important for recognition… The principal question asked of the student or the graduate will therefore no longer be “what did you do to obtain your degree?” but rather “what can you do now that you have obtained your degree?” This approach is of relevance to the labour market and is certainly more flexible when taking into account issues of lifelong learning, non-traditional learning, and other forms of nonformal educational experiences.

(Purser, Council of Europe, 2003)

Main Idea

Moscow

Learning Outcomes 5

From a teacher-centred approach… …to a student-centred approach

Teacher-centred approach -focuses on the teacher’s input and on assessment in terms of how well the students absorbed the material taught.

Student-centred approach -focuses on what the students are expected to be able to do at the end of the module or program.

Moscow

Learning Outcomes 6

Working Definition

Learning outcomes are statements of what a learner is expected to know, understand and/or be able to demonstrate after completion of a process of learning.

Moscow

Learning Outcomes 7

The curriculum should be redesigned to reflect learning outcomes, rather than number of credits and number of hours of study.

We have to describe qualifications in terms of • Workload• Level• Learning Outcomes• Competences• Profile

Moscow

Learning Outcomes 8

Learning Theory:

Learning Processes and Learning Outcome

Moscow

Learning Outcomes 9Moscow

Transfer Tutor Coach

• factual knowledge, “know-that”

• Transfer of propositional knowledge

• to know, to remember

• Production of correct answers

• Verbal knowledge, Memorisation

• to teach, to explain

• procedural knowledge, "know-how"

• Presentation of pre-determined problems

• to do, to practice

• Selection of correct method and its use

• Skill, Ability

• to observe, to help, to demonstrate

• social Practice, "knowing-in-action"

• Action in real (complex and social) situations

• to cope, to master

• Realisation of adequate action strategies

• Social Responsibilty

• to cooperate, to support

Learning I Learning II Learning III

Learning Outcomes 10

Knowledge Mastery

knowing thatDeclarative KnowledgeStatic KnowledgeFactual Knowledge

knowing howProcedural KnowledgeDynamic KnowledgeProcess Knowledge

to be able toCompetencesAbilitiesSkillsProficiency

knowing whereSocial KnowledgeOrganisational KnowledgeOrientational Knowledge

Gap

Knowledge is not Mastery!

Moscow

Learning Outcomes Page 11

Learning as a social process (H. Dreyfus)

to discoverto construct

”Skill 3"Action(intuitive)Expertness

Project

Experience,Practice

to imitate

"know how"

Beginner

Drill &Tests (e.g.

MultipleChoice)

to decideto choose

”Skill 1"Understanding(detached)Competence

ExerciseApplication

to developto act

”Skill 2"Understanding(implicit)Dexterity

Demonstration(e.g. Timed Stations)

1 2 3 4 5to remember

"know that"

Novice

InformationPresentation

Learning Outcomes 12

Taxonomy (Bloom)

Moscow

Learning Outcomes 13/37

Know-ledge

Cognitive Processes

Remember (1)

Under-stand (2)

Apply (3)

Analyze (4)

Evaluate (5)

Create (6)

Facts (A)

Concepts (B)

Proce-dures (C)

Meta-cognitive (D)

Moscow

Learning Outcomes 14

Comparing Taxonomies

Moscow

1. Remember2. Understand

3. Apply4. Analyse5. Evaluate6. Create

Bloom Anderson/Krathwohl

Learning Outcomes 15

Gestaltungsebenen komplettCognitive Processes

Create

Evaluate

Apply

Understand

Remember

Analyse

Moscow

Learning Outcomes 16/37

Know-ledge

Cognitive Processes

Remember (1)

Under-stand (2)

Apply (3)

Analyze (4)

Evaluate (5)

Create (6)

Facts (A)

Concepts (B)

Proce-dures (C)

Meta-cognitive (D)

Transfer(Knowledge)

Tutor(Practice)

Coach(Construction)

Learning I(Teaching I)

Learning II(Teaching

II)

Learning III(Teaching III)

Baumgartner/Payr 1999 & Baumgartner 2004

Moscow

Learning Outcomes 17/37

Know-ledge

Cognitive Processes

Remember (1)

Under-stand (2)

Apply (3)

Analyze (4)

Evaluate (5)

Create (6)

Facts (A)

Concepts (B)

Proce-dures (C)

Meta-cognitive (D)

Mastery ofConcepts

Mastery ofSkills

Accommodateto relationships

CognitiveLearning

OperationalLearning

ExperientialLearning

Barnett 1999: Higher Education

Moscow

Learning Outcomes 18/37

Know-ledge

Cognitive Processes

Remember (1)

Under-stand (2)

Apply (3)

Analyze (4)

Evaluate (5)

Create (6)

Facts (A)

Concepts (B)

Proce-dures (C)

Meta-cognitive (D)

Knowledge Skills Competence

TheoreticalFactual

CognitivePractical

ResponsibilityAutonomy

European Qualifications Framework (EQF)for Lifelong Learning

Moscow

Learning Outcomes 19

Construction

Blended Learning

Special Design of Blended Learning Scenarios

Use of special tools: Wikis, Podcasts, Vodcasts, E-Portfolios

Develop tasks and exercises for virtual settings

LMS is an educational tool for learning and teaching

Self directed learning (projects) with personal responsibilities

Transfer

Repository

Upload Material 1:1 to the LMS platform (Word, PDF)

Face-to-face study is supported by distribution of materials

Up-/Download changes responsibility structure

LMS is a tool for administrative support

Implementation step by step but organisation-wide

Communication

Enrichment

Material prepared for the learning process („Study-Guide“)

Asynchronous Communication (Forum, email)

Integrate tasks and exercises

LMS is a tool for the communication process

E-Moderation, E-Tutoring necessary

Moscow

Learning Outcomes 20

Student Workload:1 ECTS ~ 25-30 Hours of Workload

For Example in our Department:

1 Modul = 3 ECTS = 1 face-to-face Seminar = 4 Months

Month-1 Month-2 Month 1 Month 2

Start of Module Contact Timeface2face

End of Module

Blended Learning Arrangement (Example)

Moscow

Learning Outcomes 21

Learning Style: Overview & Critique

Coffield, F., D. Moseley, E. Hall, and K. Ecclestone. 2004. ‘Learning Styles and Pedagogy in Post-16 Learning: A Systematic and Critical Review’. Learning and Skills Research Centre. http://sxills.nl/lerenlerennu/bronnen/Learning%20styles%20by%20Coffield%20e.a..pdf.

http://snipurl.com/learning_styles

Moscow

Perceive & Do (Debug)

Produce &Deposit

Practice &Discuss

Interactingwith Object

Interacting with Self & other Subject (Human)

InteractingSelf & with Society

Knowing-in-action

Reflecting-in-practice Reflecting-in-action

ArtefactEnvironment

ReflectiveLearning

RelationalLearning

ConventionalLearning

Work Based LearningExperiential Learning

Peter Baumgartner

David Major

Donald Schön

Moscow Learning Outcomes 22

Learning Outcomes 23

Methodology:

Guidelines for Educational & Curriculum Design

Moscow

Learning Outcomes 24

Three Basic Tasks (1)

1. Clearly defining the learning outcomes

2. Selecting teaching and learning methods that are likely to ensure that the learning outcomes are achieved.

3. Assessing the student learning outcomes and checking to see how well they match with what was intended

Moscow

Learning Outcomes 25

Teaching and Learning Concept

• Which teaching and learning concept determines a module?

• What forms of assessment apply to a teaching concept?

Moscow

Learning Outcomes 26

Gestaltungsebenen komplettTime

InstitutionSince Foundation (Years)

Programme/CurriculumMany ECTS

Educational Ensemble(Several Hours)

Educational Scenario(Minutes – Hours)

Educational Interaction(Seconds – Minutes)Microlevel

Mesolevel

MacrolevelModule

Few ECTS

nat. + int. PoliticsLegislations Periods – Year

Moscow

Learning Outcomes 27

Level, Scope, Time of Educational Design

Global objectives / broad / 1 year or more (often many years): providing a vision

Educational objectives / moderate / weeks or months: designing the curriculum

Instructional objectives / narrow / hours or days: lesson planning

Moscow

Learning Outcomes 28

Three Basic Tasks (2)

1. Clearly defining the learning outcomes

2. Selecting teaching and learning methods that are likely to ensure that the learning outcomes are achieved.

3. Assessing the student learning outcomes and checking to see how well they match with what was intended

Moscow

Learning Outcomes 29

Example

Learning Objective:

„Students are able to apply the concepts of learning outcomes to their curricula.“

Moscow

Learning Outcomes 30

Example

Learning Objective:

„Students are able to apply the concepts of learning outcomes to their curricula.“

Moscow

„Apply“ is mentioned = 3

„Concepts“ is

conceptual knowledge = B

Taxonomy = B3

Learning Outcomes 31/37

Know-ledge

Cognitive Processes

Remember (1)

Under-stand (2)

Apply (3)

Analyze (4)

Evaluate (5)

Create (6)

Facts (A) 15

Concepts (B) 7 3

Proce-dures (C) 7

Meta-cognitive (D)

38

Moscow

Learning Outcomes 32

Assessment:

Matching Learning Outcome with Assessments

Moscow

Learning Outcomes 33

Alignment: Assessment mirrors Learning Outcomes

As far as the students are concerned, the assessment is the curriculum:

Moscow

Learning Outcomes 34/37

Know-ledge

Cognitive Processes

Remember (1)

Under-stand (2)

Apply (3)

Analyze (4)

Evaluate (5)

Create (6)

Facts (A)

Concepts (B)

Proce-dures (C)

Meta-cognitive (D)

Mul

tiple

Cho

ice

Written ExaminationOral Examination

P R O J E C T

Written Examination

EssayOral Defensio

(E-) P O R T F O L I O

Moscow

Learning Outcomes 35

Learning Objectives and Assessment

• What are the learning objectives of the module?

• Is the chosen form of assessment applicable/appropriate for measuring defined learning objectives?

• What knowledge, skills & competencies should the student acquire? (Use descriptors of EQF & NQF, e.g. for Austria/Germany: Professional competencies? Methodological competencies? Self competencies? Social competencies?)

• Does the selected form of assessment provide information regarding the type of acquired competence?

Moscow

Learning Outcomes 36

Three Basic Tasks (3)

1. Clearly defining the learning outcomes

2. Selecting teaching and learning methods that are likely to ensure that the learning outcomes are achieved.

3. Assessing the student learning outcomes and checking to see how well they match with what was intended

Moscow

Learning Outcomes 37

Benchmark Orientation

Moscow

Individual benchmark: allows for assessing new knowledge acquisition and learning gains, also contributes to the student’s self assessment ability

Social benchmark: allows for comparing individual performance to average performance delivered by control or peer group.

Learning objective benchmark: allows for assessing the level of student learning and achievement. The benchmark seems applicable with regard to validity and reliability as results are measured according to dimensions that are defined prior to assessment and that are aligned with the competences required.

Learning Outcomes 38

Learning Outcomes and Assessment Criteria - Example

Learing outcomes

Assessment criteriaExample for a National Grading System (Russia)

Grade 5Excellent

Grade 4Good

Grade 3Satisfactory

Grade 2Unsatisfactory

Grade 1Very Poor

On successfulcompletion of this module,Students should be able to: Summariseevidence fromthe scienceeducation literature to support develop-ment of a line of argument.

Outstanding use of literature showing excellent ability toSynthesise evidence in analytical wayto formulate clear conclusions.

Very good useof literatureshowing highability to synthesiseevidence in analytical way toformulate clearconclusions.

Good use ofLiterature showing goodability to synthesiseevidence in analytical way toformulate clearconclusions.

Limited use ofliteratureshowing fairability to synthesiseevidence to formulate conclusions.

Poor use ofliteratureshowing lack ofability to synthesiseevidence to formulateconclu-sions.

Moscow

Learning Outcomes 39

Problem: Many grading systems co-exist in Europe and the interpretation of grades varies considerably from one country to another one needs a common measure to facilitate the transfer of students and their grades between European higher education institutions

Solution: The national grading system is supported and complemented by the ECTS grading system. The ECTS grading scale classifies results according to normal distribution - hence following a social benchmark - regardless of the level of knowledge and skills achieved.

Moscow

ECTS grading system

Learning Outcomes 40

A B C D E Fx F

highest performing10 %

following25 %(35% accum.)

following30 %(65 %accum.)

following25 %(90 %accum.)

lowest performing10 %(100 %)

Fail 1 Fail 2

Out-standing, only minor errors

Above the average, but with some errors

Generally sound but with notable errors

Fair but with significant short-comings

Meets the minimum criteria

Some more work required

Consi-derable further work required

Moscow

ECTS grading system

Learning Outcomes 41

Identify the reference group for which the grade distribution will be calculated (usually a degree programme, but in some cases a wider or different grouping of students such as a Faculty or sector—e.g. Humanities). Collect all grades awarded over a period of (at least) two academic years for the reference group identified.Calculate the grade distribution in terms of percentages for the reference group. Include the grading percentage table of your degree programme in every Transcript of Records/Diploma Supplement. For transfer, compare the percentage table of the other institution’s degree programme with your own. On the basis of this comparison individual grades can be converted.

Moscow

ECTS grading system - Requirements

Learning Outcomes 42

Grading and Feedback

What is the adequate benchmark for student

achievements?

Shall the achievements be graded or not?

Shall one provide/not provide feedback to students? If

yes, how?

Are self assessments/peer assessments applicable?

Moscow

Learning Outcomes 43

Difficulties with Learning Outcomes

1. Academic study should be open-ended and that learning outcomes do not fit in with this liberal view of learning (Instead of Education: Industrialisation of Learning).

2. There is a danger of an assessment-driven curriculum if learning outcomes are too confined (Teaching/Learning to the Test.)

3. Learning outcomes could give rise to confusion among students and staff if guidelines are not followed and integrated seamlessly into the curriculum (System Change).

Moscow

Learning Outcomes 44

Advantages of Learning Outcomes I

Help teachers to tell students more precisely what is

expected of them.

Help students to learn more effectively: students know

where they stand and the curriculum is made more open

to them.

Help teachers to design their materials more effectively by

acting as a template for them.

Make it clear what students can hope to gain from

following a particular course or lecture.

Moscow

Learning Outcomes 45

Advantages of Learning Outcomes II

Help teachers select the appropriate teaching strategy matched

to the intended learning outcome, e.g. lecture, seminar, group

work, tutorial, discussion, peer group presentation or laboratory

class.

Help teachers to tell their colleagues more precisely what a

particular activity is designed to achieve.

Assist in setting examinations based on the materials delivered.

Ensure that appropriate teaching and assessment strategies are

employed.

Moscow

Thank you for your attention!

Спасибо за Ваше внимание

46