CTD Sp14 Weekly Workshop: Learning Outcomes

Peter Newbury, Ph.D.

Center for Teaching Development,

University of California, San Diego

pnewbury@ucsd.edu

@polarisdotca #ctducsd

ctd.ucsd.edu

resources: ctd.ucsd.edu/programs/weekly-workshops-spring-2014/

April 16, 2014 12:00 – 12:50 pm Center Hall, Rm 316

Unless otherwise noted, content

is licensed under a Creative Commons

Attribution-Non Commercial 3.0 License.

CTD WEEKLY WORKSHOP:

LEARNING OUTCOMES

please sign in

Scholarly approach to teaching:

Learning Outcomes 2

Carl Wieman

Science Education Initiative

cwsei.ubc.ca

What should

students

learn?

What are

students

learning?

What instructional

approaches

help students

learn?

learning

outcomes

(goals, objectives)

assessment

(Apr 30)

alt to lecture

(Apr 23)

peer instruction,

(May 7, 14)

(Image: NASA)

Learning Outcomes

3

Introductory “Astro 101”

Learning Outcomes 4

This course covers Chapters

1. Mercury

2. Venus

…

8. Neptune

9. other objects

10. Formation of the Solar System

deduce from patterns in the

properties of the planets, moons,

asteroids and other bodies that the

Solar System had single formation

event.

reconstruct the formation and

evolution of various bodies in the

Solar System by interpreting the

presence (and their appearance)

or absence of craters

provide notable examples of how

comets influenced history, art and

science

Course with Learning Outcomes Traditional Course Syllabus

Learning outcomes

Learning Outcomes 5

completes the sentence, “By this end of this

lesson/unit/course, you will be able to…”

begins with an action verb (“deduce”) (more below)

tells the students what they must do to demonstrate

they “understand” the concept

deduce from patterns in the

properties of the planets, moons,

asteroids and other bodies that the

Solar System had single formation

event.

What is the Value of Course-Specific

Learning Outcomes?

Learning Outcomes 6

Simon & Taylor [1] asked students to complete this

sentence:

For me, the use of learning goals in this course is…

They received 597 responses from students in computer

science and microbiology. Responses were put into

categories that emerged from the responses.

Learning Outcomes

7 Simon & Taylor (2009)

Learning outcomes are valuable to…

Learning Outcomes 8

the students

reveals what the instructor is looking for (no guessing what “understand” means.)

big picture of the next part of the course

allows student to check that s/he has mastered the concept (especially when studying later)

the instructor

crystallizes what the instructor actually cares about

helps the instructor

choose clicker questions for peer instruction in class

write the final exam

…choose clicker questions for PI

Learning Outcomes 9 ClassAction http://astro.unl.edu/classaction/

…write the final exam

Learning Outcomes 10

(10 marks) List 3 patterns of the Solar System as a

whole. Then, outline in some detail the current model for

the formation of the Solar System. In particular, make

sure you explain how the observed patterns and

regularities are related to this theory of formation.

several LOs giving big

picture, attitudes,

behaviors

(likely) can’t be

assessed with a single

exam question

supported by many

topic-level LOs

(if not, why not?)

many LOs defining

what it means to

“understand” at this

level (freshman, etc.)

can be (should be)

repeatedly assessed on

homework, exams

support one or more

course-level LOs

(if not, why not?)

11

Learning Outcomes

Course-level LOs Topic-level LOs

Topic-level

LO

Topic-level

LO

Topic-

level LO

Course-level LO #4

Learning Outcomes

12

Course-level LO #2

Course-level LO #3 Course-level

learning outcome (LO) #1

Topic-level

LO Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO Topic-level

LO

Topic-

level LO

Topic-

level LO

Topic-

level LO

Topic-

level LO

Topic-

level LO

Topic-level

learning outcome

Topic-level

LO

Topic-level

LO

Topic-

level LO

Course-level LO #4

Learning Outcomes

13

Course-level LO #2

Course-level LO #3 Course-level

learning outcome (LO) #1

Topic-level

LO Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO Topic-level

LO

Topic-

level LO

Topic-

level LO

Topic-

level LO

Topic-

level LO

Topic-

level LO Topic-

level LO

Topic-

level LO

Topic-level

LO Topic-level

learning outcome

Topic-level

LO

Topic-level

LO

Topic-

level LO

Course-level LO #4

Learning Outcomes

14

Course-level LO #2

Course-level LO #3 Course-level

learning outcome (LO) #1

Topic-level

LO Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO Topic-level

LO

Topic-

level LO

Topic-

level LO

Topic-

level LO

Topic-

level LO

Topic-

level LO Topic-

level LO

Topic-

level LO

Topic-level

LO Topic-level

learning outcome

Topic-level

LO

Topic-level

LO

Topic-

level LO

Course-level LO #4

Learning Outcomes

15

Course-level LO #2

Course-level LO #3 Course-level

learning outcome (LO) #1

Topic-level

LO Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO

Topic-level

LO Topic-level

LO

Topic-

level LO

Topic-

level LO

Topic-

level LO

Topic-

level LO

Topic-

level LO Topic-

level LO

Topic-

level LO

Topic-level

LO Topic-level

learning outcome

sync your LOs see ASTR 310 handout

Writing topic-level LOs

Learning Outcomes 16

Writing learning outcomes is hard because you have to

recognize

declare

(admit)

what you want your students to be capable of doing.

A good start is picking the verb describing the action

the students will perform to demonstrate their mastery

of the concept.

Bloom’s Taxonomy [2,3]

Learning Outcomes 17

transform or combine ideas to create something new

think critically about and defend a position

break down concepts into parts

apply comprehension to unfamiliar situations

demonstrate understanding of ideas and concepts

remember and recall factual information

6 Create

5 Evaluate

4 Analyze

3 Apply

2 Understand

1 Remember

Bloom’s Taxonomy [2,3]

Learning Outcomes 18

6 Create

5 Evaluate

4 Analyze

3 Apply

2 Understand

1 Remember

higher order thinking

lower order thinking

Bloom’s Taxonomy Verbs

Learning Outcomes 19

develop, create, propose, formulate, design, invent

judge, appraise, recommend, justify, defend, criticize, evaluate

compare, contrast, categorize, distinguish, identify, infer

apply, demonstrate, use, compute, solve, predict, construct, modify

describe, explain, summarize, interpret, illustrate

define, list, state, label, name, describe

6 Create

5 Evaluate

4 Analyze

3 Apply

2 Understand

1 Remember

Driver’s Ed 101: How to Drive in CA

The whiteboards are labelled Q1, Q2,… Your group

will concentrate on the DMV Test Question matching

your board’s question.

Task: Write a learning outcome

that your group’s question assesses.

(refer to Wieman handout

for Bloom’s Taxonomy verbs)

Learning Outcomes 20

1. “Back-engineer” LOs from exams

Learning Outcomes 21

Use last year’s (or several years’) final exam. For each

good question, ask yourself

What is this question assessing? What is the learning

outcome I want students to demonstrate to properly

answer this question?

Is that the outcome I want, or is it too low (or high)?

When you have a list of LOs,

Does it cover everything I want for this course?

Have I over- or under-represented any concepts?

2. Draft LOs from course outline

Learning Outcomes 22

Work your way through the list of topics. For each topic,

decide

What do I want students to be able to do, to

demonstrate they “get” this topic?

Don’t worry about drafting many low-level LOs.

When you revise, you’ll start grouping them into

higher-level LOs.

Share your LOs with your students

Learning Outcomes 23

(good) publish them as a document along side your syllabus

(better) publish them with your syllabus AND include relevant learning goals in your lecture slides at the beginning of each topic, even each class.

Be wary of reading them aloud: the students may not yet have the knowledge (or jargon) to appreciate the LOs. The LOs will be there when they study.

Don’t worry about “spoon-feeding” them – help the students do exactly what you feel demonstrates understanding

Scholarly approach to teaching:

Learning Outcomes 24

Carl Wieman

Science Education Initiative

cwsei.ubc.ca

What should

students

learn?

What are

students

learning?

What instructional

approaches

help students

learn?

learning

outcomes

(goals, objectives)

assessment

(Apr 30)

alt to lecture

(Apr 23)

peer instruction,

(May 7, 14)

Peter Newbury, Ph.D.

Center for Teaching Development,

University of California, San Diego

pnewbury@ucsd.edu

@polarisdotca #ctducsd

ctd.ucsd.edu

resources: ctd.ucsd.edu/programs/weekly-workshops-spring-2014/

April 16, 2014 12:00 – 12:50 pm Center Hall, Rm 316

Unless otherwise noted, content

is licensed under a Creative Commons

Attribution-Non Commercial 3.0 License.

CTD WEEKLY WORKSHOP:

LEARNING OUTCOMES

please sign in

References

Learning Outcomes 26

1. Simon, B., & Taylor, J. (2009). What is the Value of Course-Specific Learning Goals? Journal

of College Science Teaching, 39, 2, 52-57. PDF available at

www.cwsei.ubc.ca/SEI_research/files/LifeSci/Simon_Taylor_ValueOfCourseSpecificLG.pdf

2. Bloom B. S. (1956). Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain.

New York: David McKay Co Inc.

Adapted from Anderson, L. W., & Krathwohl, D. R. (2001). A Taxonomy for Learning.

Teaching, and assessing: A revision of bloom's taxonomy of educational objectives.

Revised Bloom’s Taxonomy www.celt.iastate.edu/teaching/RevisedBlooms1.html

3. Excerpt from Wieman, C. (2007). Slides from the Wieman Learning Goals Workshop.

www.cwsei.ubc.ca/resources/learn_goals.htm

4. California DMV Sample Class C Written Test 5

www.dmv.ca.gov/pubs/interactive/tdrive/clc6written.htm

Bloom’s Taxonomy of the Cognitive Domain

(Levels of Learning)

Learning Outcomes Center for Teaching Development ctd.ucsd.edu

Adapted from Carl Wieman (2007) www.cwsei.ubc.ca/resources/learn_goals.htm

6 Create: transform and combine ideas to create something new

develop, create, propose, formulate, design, invent

5 Evaluate: think critically about and defend a position

judge, appraise, recommend, justify, defend, criticize, evaluate

4 Analyze:: break down concepts into parts

compare, contrast, categorize, distinguish, identify, infer

3 Apply: apply comprehension to unfamiliar situations

apply, demonstrate, use, compute, solve, predict, construct, modify

2 Understand: demonstrate understanding of ideas, concepts

describe, explain, summarize, interpret, illustrate

1 Remember: remember and recall factual knowledge

define, list, state, label, name, describe