Learning Outcomes
Learning Outcomes
Learning Theory: How to Teach, Learn, and Assess?
Some Basic Concepts
Methodology: How? – Theory and ProceduresGuidelines for Educational Design
2
Starting Point: What are Learning Outcomes? Motivation and Considerations
Overview
Moscow
Assessment: How? – Different Types of
Assessments
Concordance & Alignment
Learning Outcomes 3
Starting Point:
Introduction: What are Learning Outcomes?
Moscow
Learning Outcomes 4
Learning outcomes are important for recognition… The principal question asked of the student or the graduate will therefore no longer be “what did you do to obtain your degree?” but rather “what can you do now that you have obtained your degree?” This approach is of relevance to the labour market and is certainly more flexible when taking into account issues of lifelong learning, non-traditional learning, and other forms of nonformal educational experiences.
(Purser, Council of Europe, 2003)
Main Idea
Moscow
Learning Outcomes 5
From a teacher-centred approach… …to a student-centred approach
Teacher-centred approach -focuses on the teacher’s input and on assessment in terms of how well the students absorbed the material taught.
Student-centred approach -focuses on what the students are expected to be able to do at the end of the module or program.
Moscow
Learning Outcomes 6
Working Definition
Learning outcomes are statements of what a learner is expected to know, understand and/or be able to demonstrate after completion of a process of learning.
Moscow
Learning Outcomes 7
The curriculum should be redesigned to reflect learning outcomes, rather than number of credits and number of hours of study.
We have to describe qualifications in terms of • Workload• Level• Learning Outcomes• Competences• Profile
Moscow
Learning Outcomes 8
Learning Theory:
Learning Processes and Learning Outcome
Moscow
Learning Outcomes 9Moscow
Transfer Tutor Coach
• factual knowledge, “know-that”
• Transfer of propositional knowledge
• to know, to remember
• Production of correct answers
• Verbal knowledge, Memorisation
• to teach, to explain
• procedural knowledge, "know-how"
• Presentation of pre-determined problems
• to do, to practice
• Selection of correct method and its use
• Skill, Ability
• to observe, to help, to demonstrate
• social Practice, "knowing-in-action"
• Action in real (complex and social) situations
• to cope, to master
• Realisation of adequate action strategies
• Social Responsibilty
• to cooperate, to support
Learning I Learning II Learning III
Learning Outcomes 10
Knowledge Mastery
knowing thatDeclarative KnowledgeStatic KnowledgeFactual Knowledge
knowing howProcedural KnowledgeDynamic KnowledgeProcess Knowledge
to be able toCompetencesAbilitiesSkillsProficiency
knowing whereSocial KnowledgeOrganisational KnowledgeOrientational Knowledge
Gap
Knowledge is not Mastery!
Moscow
Learning Outcomes Page 11
Learning as a social process (H. Dreyfus)
to discoverto construct
”Skill 3"Action(intuitive)Expertness
Project
Experience,Practice
to imitate
"know how"
Beginner
Drill &Tests (e.g.
MultipleChoice)
to decideto choose
”Skill 1"Understanding(detached)Competence
ExerciseApplication
to developto act
”Skill 2"Understanding(implicit)Dexterity
Demonstration(e.g. Timed Stations)
1 2 3 4 5to remember
"know that"
Novice
InformationPresentation
Learning Outcomes 12
Taxonomy (Bloom)
Moscow
Learning Outcomes 13/37
Know-ledge
Cognitive Processes
Remember (1)
Under-stand (2)
Apply (3)
Analyze (4)
Evaluate (5)
Create (6)
Facts (A)
Concepts (B)
Proce-dures (C)
Meta-cognitive (D)
Moscow
Learning Outcomes 14
Comparing Taxonomies
Moscow
1. Remember2. Understand
3. Apply4. Analyse5. Evaluate6. Create
Bloom Anderson/Krathwohl
Learning Outcomes 15
Gestaltungsebenen komplettCognitive Processes
Create
Evaluate
Apply
Understand
Remember
Analyse
Moscow
Learning Outcomes 16/37
Know-ledge
Cognitive Processes
Remember (1)
Under-stand (2)
Apply (3)
Analyze (4)
Evaluate (5)
Create (6)
Facts (A)
Concepts (B)
Proce-dures (C)
Meta-cognitive (D)
Transfer(Knowledge)
Tutor(Practice)
Coach(Construction)
Learning I(Teaching I)
Learning II(Teaching
II)
Learning III(Teaching III)
Baumgartner/Payr 1999 & Baumgartner 2004
Moscow
Learning Outcomes 17/37
Know-ledge
Cognitive Processes
Remember (1)
Under-stand (2)
Apply (3)
Analyze (4)
Evaluate (5)
Create (6)
Facts (A)
Concepts (B)
Proce-dures (C)
Meta-cognitive (D)
Mastery ofConcepts
Mastery ofSkills
Accommodateto relationships
CognitiveLearning
OperationalLearning
ExperientialLearning
Barnett 1999: Higher Education
Moscow
Learning Outcomes 18/37
Know-ledge
Cognitive Processes
Remember (1)
Under-stand (2)
Apply (3)
Analyze (4)
Evaluate (5)
Create (6)
Facts (A)
Concepts (B)
Proce-dures (C)
Meta-cognitive (D)
Knowledge Skills Competence
TheoreticalFactual
CognitivePractical
ResponsibilityAutonomy
European Qualifications Framework (EQF)for Lifelong Learning
Moscow
Learning Outcomes 19
Construction
Blended Learning
Special Design of Blended Learning Scenarios
Use of special tools: Wikis, Podcasts, Vodcasts, E-Portfolios
Develop tasks and exercises for virtual settings
LMS is an educational tool for learning and teaching
Self directed learning (projects) with personal responsibilities
Transfer
Repository
Upload Material 1:1 to the LMS platform (Word, PDF)
Face-to-face study is supported by distribution of materials
Up-/Download changes responsibility structure
LMS is a tool for administrative support
Implementation step by step but organisation-wide
Communication
Enrichment
Material prepared for the learning process („Study-Guide“)
Asynchronous Communication (Forum, email)
Integrate tasks and exercises
LMS is a tool for the communication process
E-Moderation, E-Tutoring necessary
Moscow
Learning Outcomes 20
Student Workload:1 ECTS ~ 25-30 Hours of Workload
For Example in our Department:
1 Modul = 3 ECTS = 1 face-to-face Seminar = 4 Months
Month-1 Month-2 Month 1 Month 2
Start of Module Contact Timeface2face
End of Module
Blended Learning Arrangement (Example)
Moscow
Learning Outcomes 21
Learning Style: Overview & Critique
Coffield, F., D. Moseley, E. Hall, and K. Ecclestone. 2004. ‘Learning Styles and Pedagogy in Post-16 Learning: A Systematic and Critical Review’. Learning and Skills Research Centre. http://sxills.nl/lerenlerennu/bronnen/Learning%20styles%20by%20Coffield%20e.a..pdf.
http://snipurl.com/learning_styles
Moscow
Perceive & Do (Debug)
Produce &Deposit
Practice &Discuss
Interactingwith Object
Interacting with Self & other Subject (Human)
InteractingSelf & with Society
Knowing-in-action
Reflecting-in-practice Reflecting-in-action
ArtefactEnvironment
ReflectiveLearning
RelationalLearning
ConventionalLearning
Work Based LearningExperiential Learning
Peter Baumgartner
David Major
Donald Schön
Moscow Learning Outcomes 22
Learning Outcomes 23
Methodology:
Guidelines for Educational & Curriculum Design
Moscow
Learning Outcomes 24
Three Basic Tasks (1)
1. Clearly defining the learning outcomes
2. Selecting teaching and learning methods that are likely to ensure that the learning outcomes are achieved.
3. Assessing the student learning outcomes and checking to see how well they match with what was intended
Moscow
Learning Outcomes 25
Teaching and Learning Concept
• Which teaching and learning concept determines a module?
• What forms of assessment apply to a teaching concept?
Moscow
Learning Outcomes 26
Gestaltungsebenen komplettTime
InstitutionSince Foundation (Years)
Programme/CurriculumMany ECTS
Educational Ensemble(Several Hours)
Educational Scenario(Minutes – Hours)
Educational Interaction(Seconds – Minutes)Microlevel
Mesolevel
MacrolevelModule
Few ECTS
nat. + int. PoliticsLegislations Periods – Year
Moscow
Learning Outcomes 27
Level, Scope, Time of Educational Design
Global objectives / broad / 1 year or more (often many years): providing a vision
Educational objectives / moderate / weeks or months: designing the curriculum
Instructional objectives / narrow / hours or days: lesson planning
Moscow
Learning Outcomes 28
Three Basic Tasks (2)
1. Clearly defining the learning outcomes
2. Selecting teaching and learning methods that are likely to ensure that the learning outcomes are achieved.
3. Assessing the student learning outcomes and checking to see how well they match with what was intended
Moscow
Learning Outcomes 29
Example
Learning Objective:
„Students are able to apply the concepts of learning outcomes to their curricula.“
Moscow
Learning Outcomes 30
Example
Learning Objective:
„Students are able to apply the concepts of learning outcomes to their curricula.“
Moscow
„Apply“ is mentioned = 3
„Concepts“ is
conceptual knowledge = B
Taxonomy = B3
Learning Outcomes 31/37
Know-ledge
Cognitive Processes
Remember (1)
Under-stand (2)
Apply (3)
Analyze (4)
Evaluate (5)
Create (6)
Facts (A) 15
Concepts (B) 7 3
Proce-dures (C) 7
Meta-cognitive (D)
38
Moscow
Learning Outcomes 32
Assessment:
Matching Learning Outcome with Assessments
Moscow
Learning Outcomes 33
Alignment: Assessment mirrors Learning Outcomes
As far as the students are concerned, the assessment is the curriculum:
Moscow
Learning Outcomes 34/37
Know-ledge
Cognitive Processes
Remember (1)
Under-stand (2)
Apply (3)
Analyze (4)
Evaluate (5)
Create (6)
Facts (A)
Concepts (B)
Proce-dures (C)
Meta-cognitive (D)
Mul
tiple
Cho
ice
Written ExaminationOral Examination
P R O J E C T
Written Examination
EssayOral Defensio
(E-) P O R T F O L I O
Moscow
Learning Outcomes 35
Learning Objectives and Assessment
• What are the learning objectives of the module?
• Is the chosen form of assessment applicable/appropriate for measuring defined learning objectives?
• What knowledge, skills & competencies should the student acquire? (Use descriptors of EQF & NQF, e.g. for Austria/Germany: Professional competencies? Methodological competencies? Self competencies? Social competencies?)
• Does the selected form of assessment provide information regarding the type of acquired competence?
Moscow
Learning Outcomes 36
Three Basic Tasks (3)
1. Clearly defining the learning outcomes
2. Selecting teaching and learning methods that are likely to ensure that the learning outcomes are achieved.
3. Assessing the student learning outcomes and checking to see how well they match with what was intended
Moscow
Learning Outcomes 37
Benchmark Orientation
Moscow
Individual benchmark: allows for assessing new knowledge acquisition and learning gains, also contributes to the student’s self assessment ability
Social benchmark: allows for comparing individual performance to average performance delivered by control or peer group.
Learning objective benchmark: allows for assessing the level of student learning and achievement. The benchmark seems applicable with regard to validity and reliability as results are measured according to dimensions that are defined prior to assessment and that are aligned with the competences required.
Learning Outcomes 38
Learning Outcomes and Assessment Criteria - Example
Learing outcomes
Assessment criteriaExample for a National Grading System (Russia)
Grade 5Excellent
Grade 4Good
Grade 3Satisfactory
Grade 2Unsatisfactory
Grade 1Very Poor
On successfulcompletion of this module,Students should be able to: Summariseevidence fromthe scienceeducation literature to support develop-ment of a line of argument.
Outstanding use of literature showing excellent ability toSynthesise evidence in analytical wayto formulate clear conclusions.
Very good useof literatureshowing highability to synthesiseevidence in analytical way toformulate clearconclusions.
Good use ofLiterature showing goodability to synthesiseevidence in analytical way toformulate clearconclusions.
Limited use ofliteratureshowing fairability to synthesiseevidence to formulate conclusions.
Poor use ofliteratureshowing lack ofability to synthesiseevidence to formulateconclu-sions.
Moscow
Learning Outcomes 39
Problem: Many grading systems co-exist in Europe and the interpretation of grades varies considerably from one country to another one needs a common measure to facilitate the transfer of students and their grades between European higher education institutions
Solution: The national grading system is supported and complemented by the ECTS grading system. The ECTS grading scale classifies results according to normal distribution - hence following a social benchmark - regardless of the level of knowledge and skills achieved.
Moscow
ECTS grading system
Learning Outcomes 40
A B C D E Fx F
highest performing10 %
following25 %(35% accum.)
following30 %(65 %accum.)
following25 %(90 %accum.)
lowest performing10 %(100 %)
Fail 1 Fail 2
Out-standing, only minor errors
Above the average, but with some errors
Generally sound but with notable errors
Fair but with significant short-comings
Meets the minimum criteria
Some more work required
Consi-derable further work required
Moscow
ECTS grading system
Learning Outcomes 41
Identify the reference group for which the grade distribution will be calculated (usually a degree programme, but in some cases a wider or different grouping of students such as a Faculty or sector—e.g. Humanities). Collect all grades awarded over a period of (at least) two academic years for the reference group identified.Calculate the grade distribution in terms of percentages for the reference group. Include the grading percentage table of your degree programme in every Transcript of Records/Diploma Supplement. For transfer, compare the percentage table of the other institution’s degree programme with your own. On the basis of this comparison individual grades can be converted.
Moscow
ECTS grading system - Requirements
Learning Outcomes 42
Grading and Feedback
What is the adequate benchmark for student
achievements?
Shall the achievements be graded or not?
Shall one provide/not provide feedback to students? If
yes, how?
Are self assessments/peer assessments applicable?
Moscow
Learning Outcomes 43
Difficulties with Learning Outcomes
1. Academic study should be open-ended and that learning outcomes do not fit in with this liberal view of learning (Instead of Education: Industrialisation of Learning).
2. There is a danger of an assessment-driven curriculum if learning outcomes are too confined (Teaching/Learning to the Test.)
3. Learning outcomes could give rise to confusion among students and staff if guidelines are not followed and integrated seamlessly into the curriculum (System Change).
Moscow
Learning Outcomes 44
Advantages of Learning Outcomes I
Help teachers to tell students more precisely what is
expected of them.
Help students to learn more effectively: students know
where they stand and the curriculum is made more open
to them.
Help teachers to design their materials more effectively by
acting as a template for them.
Make it clear what students can hope to gain from
following a particular course or lecture.
Moscow
Learning Outcomes 45
Advantages of Learning Outcomes II
Help teachers select the appropriate teaching strategy matched
to the intended learning outcome, e.g. lecture, seminar, group
work, tutorial, discussion, peer group presentation or laboratory
class.
Help teachers to tell their colleagues more precisely what a
particular activity is designed to achieve.
Assist in setting examinations based on the materials delivered.
Ensure that appropriate teaching and assessment strategies are
employed.
Moscow
Thank you for your attention!
Спасибо за Ваше внимание
46