Claus Brabrand “Department Seminar on Teaching & Learning”May 28, 2008 “Department Seminar on Teaching & Learning” IT University of Copenhagen ((( May

Claus Brabrand “Department Seminar on Teaching & Learning” May 28, 2008

“Department Seminar on Teaching & Learning”

IT University of Copenhagen((( May 28, 2008 )))


“Intro to Constructive Alignment & The SOLO Taxonomy”

IT University of Copenhagen((( May 28, 2008 )))

– Theory & Practice –

[ 3 ]Claus Brabrand “Department Seminar on Teaching & Learning” May 28, 2008

Agenda (Morning)

Introduction to Teaching & Learning: “Constructive Alignment & The SOLO Taxonomy”

From Theory to Practice: “From Content to Competence”

09:15

10:10

--- 15’ break ---

--- 15’ break ---

Group Exercise (“heterogeneous groups”): “Discussion of Theory in Practice”

11:05

12:00 Lunch: “One hour lunch break”

Welcome & Presentation of Programme: by Mads Tofte & Claus Brabrand

09:00


Agenda (Afternoon)

Main Group Exercise (“homogeneous groups”): “Application of Theory in Practice”

Short presentations of Good Examples

13:30

16:10

--- short (10’) break ---

Poster Session (Tour I + Tour II)16:40

18:30 Dinner + Bar: “Dinner + Bar”

Course Descriptions: “Good advice on how to write course descriptions”

13:00

+ short breaks

End Note: by Mads Tofte & Claus Brabrand

18:20


+ short break


Agenda (Morning)



09:15

10:10

--- 15’ break ---

--- 15’ break ---


11:05



09:00


First: Exercise

Before we start; “Focus exercise”:

1) Write down answer to: "what is good teaching?"2) Swap Post-Its...

T


“Teaching for Quality Learning at University

- What the student does”

“Teaching for Quality Learning at University

- What the student does”

Constructive Alignment & SOLO Taxonomy:

Teaching & Learning (theory)

“Teaching Teaching & Understanding Understanding”

“Teaching Teaching & Understanding Understanding”19 minute short-film on Teaching & Learning(available on DVD in 7 languages, epilogue by John Biggs)

John Biggs’ world famous ”Bible of Teaching & Learning”:

(see compendium in your ”registration bag” + paper)


Exercise

Buzz Session:

1) Discuss w/ neighbour: "which of the messages of the film did you find particularly relevant ?"

2) Write it on a Post-It3) Swap Post-Its…

T


Alignment

Make explicit ILO’s (Intended Learning Outcomes):

(…and tell this to students)

Exam = ILO’s = Teaching


Student Motivation

Susan: (”intrinsic motivation”) - wants to…: learn ! Robert: (”extrinsic motivation”) - to…: pass exams !


Constructivism

”Transmission is Dead…” : (lectures = ) Knowledge is… Actively Constructed !

active teacher &passive students

!

risk


SOLO Taxonomy

Hierarchy for Competences:

Deep learning (not surface) !

5: generalize, theorize, predict, …4: explain, analyze, compare, …3: describe, combine, classify, …2: recite, identify, calculate, …


Stud Learning Focus

Focus on Student Learning ! (instead of ”what teacher does” & labelling students: ’good/bad’) Student activitation learning


Break (until 10:10)

Please put the Post-Its on the wall

"What is good teaching?" “Film message(s)?"


Agenda (Morning)



09:15

10:10

--- 15’ break ---

--- 15’ break ---


11:05



09:00


From Content to Competence: ”advocate a shift in perspective”

From Theory to Practice…

Elaborate on SOLO (for Competences): ”advocate SOLO for competences”

New Danish Grade Scale: ”show relation to new grade scale”

1

2

3

How to write Course Descriptions: ”concrete advice”

--- After lunch: ---

AFTERLUNCH


From Content to Competence

“Old way”: my ‘2004 Concurrency’ course descr: Given in terms of a 'content description':

Essentially:

This is a bad ideafor 2 reasons...!

Goal is...:

To understand: deadlock interference synchronization ...


1) Problem with 'content' as aim

Problem with 'content‘ as learning goals ?!?

Goal is…:


analyze ...theorize ...

define deadlockdescribe solutions

name solutionsrecite conditons

Stud. C

Stud. A

Stud. B

analyze systemsexplain causes

Censor

Teacher

BUT, even if it werepossible to agree, we know that the

exam will dictate thelearning anyways.

agreement

analyze systemsexplain causes

tacit knowledge

from research-based tradition

(not known by stud.)


2) Problem with 'understanding'

Problem with 'understanding'as learning goal ?!?

The answer is simple:

concept of deadlock ?!

Goal is…:


It cannot be measured (!)


'Competence' as objectives !

'Competence' as learning objectives ! Evaluation = Have the student do something,

and then measure product and/or process

'SOLO' = Structure of the Observed Learning Outcome

Note': inherently operational (~ verbs)

Objective !

To learn to: analyze systems for... explain cause/effects... prove properties of... compare methods of... ...

Note:'understanding' is (of course) pre-requisitional (!)

Competence := knowledge + capacity to act upon it


Neighbour Discussion

Discuss with neighbour:"does this make sense ?!?" (content competence)

T





New Danish Grade Scale: ”relation to new grade scale”

1

2

3



AFTERLUNCH


Advantages of 'SOLO'

Advantages of 'SOLO': Constructed for research-based university teaching Converges on research (at SOLO 5)

SOLO 1 no understanding irrelevant information misses point ...

SOLO 2

SOLO 3

SOLO 4

SOLO 5

"pre-structural"

to identify to do procedure to recite ...

"uni-structural"

to classify to combine to enumerate ...

"multi-structural"

to relate to compare to analyze ...

"relational"

to generalize to hypothesize to theorize ...

"extended abstract" Production of new knowledge


SOLO (elaborated)Note: the list is non-exhaustive

SOLO 2”uni-structural”

SOLO 3“multi-structural”

SOLO 4“relational”

SOLO 5“extended abstract”

theorize generalize hypothesize predict judge reflect transfer theory

(to new domain) …

analyze compare contrast integrate relate explain causes apply theory

(to its domain) …

combine structure describe classify enumerate list do algorithm apply method …

define identify count name recite paraphrase follow (simple)

instructions …

Graphic Legend

problem / question / cue known related issue - given! hypothetical related issue - not given! student response

Q

R

QUANTITATIVE QUALITATIVE

R

R'Q

RQR2

R3

R1

Q

RQ

RQ


Post-It exercise

Write down 1-2 key competences (i.e., verbs)

(for your course)

T

Concurrency:

analyze for deadlock

compare models

Concurrency:


compare models





New Danish Grade Scale: ”relation to new grade scale”

1

2

3



AFTERLUNCH


“The Danish 7 Step Scale”

Unacceptable-3 For a performance which is unacceptable in all respects F

Inadequate00 For an insufficient performance which does not meet the course objectives Fx

Adequate02 For a sufficient performance which barely meets the course objectives E

Fair4For a fair performance which adequately meets the course objectives but also displays several major weaknesses D

Good7 For a good performance which meets the course objectives but also displays some weaknesses C

Very good10 For a very good performance which meets the course objectives, with only minor weaknesses B

Excellent 12 For an excellent performance which completely meets the course objectives, with no or only a few insignificant weaknesses . A

Grade := Degree of realization

of course objectives!


Intended Learning Outcomes !

Consequence: Every course has to explicitly define…:

Intended Learning Outcomes (!)


Break (until 11:05)

“Key competences for your course”?

Please put the Post-Its on the wall

After break:Heterogeneous group exercise(meet here for instructions)

Concurrency:


compare models

Concurrency:


compare models


Agenda (Morning)



09:15

10:10

--- 15’ break ---

--- 15’ break ---


11:05



09:00


Discuss how ‘main messages’of the film relate to your ownteaching (either given/received)

Group Exercise

Group Exercise (in ”heterogeneous groups”):

TNote: Groups arein the programme


LUNCH!

We resume here at 13:00


Agenda (Afternoon)



13:30

16:10




Course Descriptions: “Good advice” & “How to…”

13:00

+ short breaks


18:20


+ short break


Example: Genetics 101 (at AU)

Old course description:

New course description:

Purpose:To give the students a basic understanding of the fundamental laws of genetics and populational genetics and elementary knowledge of …:Content:- evolution, selection, mutation, variation, in-breeding, equibrilia, recombinations, …

After the course, the students are expected to be able to: locate genes on chromosomes do simple calculations: (eg, recombination frequencies, evolutionary equilibria, ..) describe and perform connexion-analysis describe fundamental genetic concepts: (e.g., mutation, variation, selection, …) describe and analyze simple inheritancies analyze inheritance of multiple genes simultaneously


Concrete Recommendations (4x)

Intended Learning Outcomes [Genetics 101]

After the course, the students are expected to be able to: locate genes on chromosomes do simple calculations : (e.g., recombination frequencies,

in-breeding coefficients, Hardy-Weinberg, evolutionary equilibria).

describe and perform connexion-analysis describe fundamental genetic concepts : (e.g., mutation

variation, in-breeding, natural selection). describe and analyze simple inheritancies analyze inheritance of multiple genes simultaneously

2) List sub-goals as 'bullets': Clearer than text

1) Use 'standard formulation':

a) puts learning focus on the student

b) competence formulation: "to be able to"

3) Use 'Verb + Noun' formulation:

What the student is expected to

do with a given matter .V N

V

V

VV

V

V

V

V

N

N

N

N

4) Avoid 'understanding-goals':

"To understand X", "Be familiar with Y", "Have a notion of Z", ...!

N

N


Old course description:

New course description:After the course, the students are expected to be able to:

describe the meaning of a wide range of programming constructsexplain fundamental concepts, techniques, and results within semantics (…)analyze the meaning of a wide range of programming constructscompare semantic descriptionsreason about semantic descriptionsprove consequences of semantic descriptionsimplement semantic descriptions (in familiar programming languages)

Another Example: Semantics

Purpose:To give the students an understanding of :- transition systems, big-step vs. small-step semantics, side-effects, laziness, termination, types, static semantics, dynamic semantics, equivalence, bisimulation, environment-store-model, and structural induction.

Applicable to all courses:

”Quantum Mechanics 101”

”Aesthetics & History of Dancing”

”Migration Patterns of Flock Birds”


Alignment Implementation Process

Process (course specific):

1) Think carefully about: overall goal of course (what are the stud. to learn?)

2) Operationalize these goals: and express them as intended learning outcomes

3) Choose carefully the form(s) of examination (~ intended learning outcomes)

4) Choose carefully the form(s) of teaching (~ intended learning outcomes)

alignmentlearning incentive learning support


Starting Point

Content description (Concurrency '04+'05):

What is the overall goal of the course...?(what are the students to learn)

“think big picture”


Overall Course Philosophy

Example: ‘Concurrency’ Philosophy: “Model-Based Design for Concurrency”:


Implementation Process






alignment


Learning Goals

Model-based design

for Concurrency

.. .

SM

I

#1#2

#3

Intended Learning Outcomes(based on The SOLO Taxonomy):

Note:explicitlyincludedas a non-goal


Alignment Impl.






alignmentlearning incentive

ILO’s: construct models apply methods relate specsmodels test models define properties verify models~props analyze models compare models implement models relate models~impl


On the Role of the Exam

Exam is a "necessary evil"

Exam is a powerful motivational & learning-guiding pedagogical tool (for the teacher) !!!

“Conceptual change”

"To the teacher, assessment is at the end of the teaching-learning sequence of events, but to the student it is at the beginning"

-- John Biggs (2003)

From:

To:


Alignment Impl.






alignmentlearning support

ILO’s: construct models apply methods relate specsmodels test models define properties verify models~props analyze models compare models implement models relate models~impl


Questions?

"What is good teaching?"

R’

xR

R’’

x

The Short-Film

Cognition structures

The SOLO Taxonomy

Association

new ~ old

analyzeexplain

'The Book'

John Biggs

"understanding"

content competence

Student activation

Student models

Susan & RobertTeacher models

levels 1 - 2 - 3

Intended learning outcomes (ILO)

Constructive AlignmentExam

SatisfactionExperiences

Pre vs. Post

Students at Uni

Model-based designfor Concurrency

. ..

My researchand teaching

'TLA'Teaching/Learning Activities

Tips'n'Tricks ???

S M

I

?

SOLO Analyses


Tips'n'Tricks (activation)

Neighbour discussions:

Frequent breaks:

Post-It exercise: focus: zoom in anonymous (!) swap'able everyone will engage empathetic control shared knowledge pool

pu

lse

re

ad

er

me

asu

rem

en

ts:

more questions (students dare ask them)

better questions (students had a chance to discuss)

1-2 min timeout [Phil Race]

Form variation:

lecturing blended with in-class activation exercises


Tips'n'Tricks (cont'd)

"Less-is-more":

Use many examples:(build on student pre-knowledge)

Explicit structure:

analyze compare relate

common deadlock, uncommon deadlock, A-synchronization, B-synchronization, hand-shake, multi-party synchronization, multi-party hand-shake, binary semaphores, generalized semaphores, blocking semaphores, recursive locks, ...

vs.

Emphasize depth over breadth (coverage)

NEWOLD

1. xxxxxxxxxx

2. yyyyyyyyyy

3. zzzzzzzzzz

4. wwwwwww

1. xxxxxxxxxx

2. yyyyyyyyyy

3. zzzzzzzzzz

4. wwwwwww

1. xxxxxxxxxx

2. yyyyyyyyyy

3. zzzzzzzzzz

4. wwwwwww

1. xxxxxxxxxx

2. yyyyyyyyyy

3. zzzzzzzzzz

4. wwwwwww

self evident to you [ teacher ] not to a learner [ student ] (esp. during learning process)

Student 'recap' at end:

after 1 dayafter 1 week

after 3 weeks

after 2 weeks

now


Now, please: "3-minute recap"

Please spend 3' on thinking about and writing down the most important points from the talk – now!:

After 1 dayAfter 1 week

After 3 weeksAfter 2 weeks

Immediately


Agenda (Afternoon)



13:30

16:10





13:00

+ short breaks


18:20


+ short break


Common SOLO Competences

From a study of632x courses at NAT/(AU+SDU)


Group Exercise

Group Exercise (in ”homogeneous groups”):

Rewrite course description (for “your group’s course”) &incorporate recommendations

(Anna will distribute ’old course descriptions’ for your convenience)

T

Do NOT (under ANY circumstances) [in fact ever] use the verbs: to ’understand’ … to ’know’ … to ’be familiar with’ … to ’have a notion of’ …

internal cognitive structure ! non-operational ! cannot be measured !

Note: Groups arein the programme


Object-Oriented Prog. – IntroAfter the course, the student is expected to be able to™: relate a given Java model to the real world phenomena that it models explain, use, and combine in programming: variables, types,

expressions, loops, … design and implement class hierarchies using single inheritance

comprising 3-5 classes. design and implement programs on the order of 500 lines of Java code. use and design generic classes and generic methods. explain and use recursive methods. construct classes that implement a given Java interface. identify and use packages. construct classes that adhere to given class invariants. describe and reason about simple class invariants. explain which methods are being called during execution of a given

program. explain the memory evolution during execution of programs. compare expected execution times (of simple algorithms).


Digital Culture

After the course, the student is expected to be able to: explain the use of the concept of culture in a digital context relate new media culture to a broader historical context apply the concepts introduced on the course in a critical analysis of

cultural og social phenomena related to the use of the internet and information technology in everyday life

reflect on different methodological perspectives on the study of digital culture and be able to select and then use them in their own practice

use and compare relevant sociological and cultural theories on a broad selection of topics within the areas of online media, digital culture and digital communication

independently identify and formulate a current and original problem/question related to the use of IT and digital communication, choose a relevant method to apply in the study, hypothesise the expected results, and finally discuss and perspectivate the actual results of the study in the final assignment


Agenda (Afternoon)



13:30

16:10





13:00

+ short breaks


18:20


+ short break


Key References ”Teaching for Quality Learning at University (what the student does)”

John BiggsThe Society for Higher Education and Open University Press, 2003(Note: 3rd edition available)

”Teaching Teaching & Understanding Understanding”Claus Brabrand & Jacob Andersen19 minute award-winning short-film (DVD)Aarhus University Press, Faculty of Science, University of Aarhus, Denmark

”Constructive Alignment and the SOLO Taxonomy: A Comparative Study of University Competences in Computer Science vs. Mathematics”Claus Brabrand & Bettina DahlKeynote Paper for Koli 2007 Conf. on Computing Education ResearchKoli National Park, Finland

”Evaluating the Quality of Learning: The SOLO Taxonomy”John B. Biggs & Kevin F. CollisNew York: Academic Press, 1982


Thank You!

((( http://www.daimi.au.dk/~brabrand/short-film/ )))

Film's homepage:


BONUS SLIDES


The New Danish Grade Scale

ECTSSCALE

A, B, C, D, E, Fx, F

...

4 steps

8 steps

10 steps

21 steps

...

4 steps

8 steps

10 steps

21 steps

7 steps:

...

... ...

Conversion (between EU countries):

Problems (comparability ~ EU nations): Information loss (10 steps 7 steps):

(13,11) A; (9,8) C; …

The “13” (“exception grade”); doesn’t exist in other scales! Some places only access if you have top grade (~ 13)

…and a number of other motivations

pigeon hole principle


The New Danish Grade Scale-- ++

passed

A: 10%B: 25%C: 30%D: 25%E: 10%

ECTS Scale

10% 25% 30% 25% 10%

Not passed

0(minimum)

100(maximum)

Degree of realization of course objectives

22,5(interval-mid)

50(interval-mid)

77,5(interval-mid)

Divide by 10& round off

0 102 5 8

-2-5

Symmetric transplacement

02 124 7 10

00-3

Avoid using “negative grades”

Add 2


Grading ?

Unacceptable-3 For a performance which is unacceptable in all respects F

Inadequate00 For an insufficient performance which does not meet the course objectives

Fx

Adequate02 For a sufficient performance which barely meets the course objectives

E

Fair4 For a fair performance which adequately meets the course objectives but also displays several major weaknesses

D

Good7 For a good performance which meets the course objectives

but also displays some weaknesses C

Very good10 For a very good performance which meets the course objectives, with only minor weaknesses

B

Excellent12 For an excellent performance which completely meets the courseobjectives, with no or only a few insignificant weaknesses.

A

Learning GoalsAfter the course, the students are expected to be able to:

locate genes on chromosomes do simple calculations: (e.g., …). describe and perform …-analysis describe fundamental genetic concepts describe and analyze … inheritancies analyze inheritance of multiple genes …

§10: Grade = degree of realization of course objectives (overall evaluation) §9: Absolute grading only (i.e., no relative grading)

Karaktergivning:12 = 'Udtømmende' 10 = 'Omfattende' (med nogle mindre væsentlige mangler) 7 = 'Omfattende' (med en del mangler)osv... ...set gennem en forskers øjne (forsknings-baseret undervisning)


”Top 10 Competences”

”Top 10 Competences” for: Computer Science vs. Natural Science (= PHYS+CHEM+BIO+MOL.BIO) vs. Mathematics


Notes on Skill Acquisition

From the world of psychoanalysis: Skill acquisition progresses according to the following

stages of learning: 1. Unconscious incompetence 2. Conscious incompetence 3. Conscious competence 4. Unconscious competence

5. Capacity for moving consciously between stages 3. and 4.: [ required by a teacher ]


A taxonomy / language for teaching impersonalizes teaching

Emotional detachment (aka. “dissociation”) The teacher is good/bad

identity: good/bad teacher The methods are good/bad

behavior: good/bad method knowledge: good/bad method

With dissociation: more capable of dealing with critique better to listen

to constructive advice (…just like with our research)

Impersonalization

identitymoralknowledgebehavior

“Neutological levels”

[model of the mind, “NLP”]

ethics

experience

reactions

convictions

capabilities

interaction


Teacher’sintention

Student’sactivity

Exam’sassessment

e.g.- explain- relate- prove- apply

e.g.- memorize- describe

UNALIGNED COURSE

e.g.- memorize- describe

"Dealing with the test"


Teacher’sintention

Student’sactivity

Exam’sassessment


ALIGNED COURSE






Definition: “Good Teaching”

Definition:

Good news: We now know how to do this:

Alignment!!! Explicitly defined course objectives (as verbs)! Discourage surface-learning! Encourage depth-learning! “Less-is-more”: depth rather than breadth of coverage!

”Good teaching is getting most students to use the higher cognitive level processes that the more academic students use spontaneously”

-- “Teaching for Quality Learning at University”, John Biggs, 2003


The BLOOM Taxonomy (1956)

The BLOOM Taxonomy:

Knowledge

Comprehension

Application

Analysis EvaluationSynthesis

Qualitative

Quantitative

SO

LO 4

+5

SO

LO 2

+3

”[…] really intended to guide the selection of items for a test rather than to evaluate the quality of a student’s response to a particular item”

-- (Biggs & Collis, 1982)”

Documents

Claus Brabrand “Department Seminar on Teaching & Learning”May 28, 2008 “Department Seminar on Teaching & Learning” IT University of Copenhagen ((( May