This work is supported by the National Science Foundation’s Transforming Undergraduate Education in STEM program within the Directorate for Education and Human Resources (DUE-1245025).

GETSI/INTEGRATE DEVELOPMENT APPROACH

& WRITING LEARNING GOALS

MODIFIED FROM PRESENTATIONS BY

JOSHUA CAULKINS (U OF RI) AND DAVID STEER (U OF AKRON)

The webinar begins at:1 pm PT | 2 pm MT | 3 pm CT | 4 pm ET3 pm PT | 4 pm MT | 5 pm CT | 6 pm ET

For audio, call: 1-877-668-4490(or 1-408-792-6300)

Press *6 to mute and unmute(but hopefully we won’t need any muting)

Headphones give less feedback than speakerphone.

GOALS FOR THIS WEBINAR

By the end of the webinar you will be able to:

1. Describe how the InTeGrate/GETSI design approach aligns to the Materials Development Rubric

2. Develop and evaluate learning goals and outcomes1. Give examples of an overarching module-level goal and a more specific

unit-level learning outcome.

2. Describe the difference between the cognitive levels in Bloom’s Taxonomy and why curriculum authors should know this difference.

3. List at least two characteristics of well-designed learning goals and state why they are important.

4. Defend why learning goals should align with assessments for a given module or activity.

3. Access the Serckit Content Management System (CMS)

THOUGHTS/IDEAS FROM ACTIVITIES SINCE LAST

WEBINAR

--Working with your partner – questions/ideas?--Implementing interdisciplinary societal aspect? Did the article or others like it spur your thinking in any way?• Kaatje – brainstormed a whole bunch of ideas; cultural relevancy of lava

flows; native population interacts with and sees lava compared to recent residents; logging industry; parks and messaging

• Venkatesh – decided to include place-based approach; considering demography; flood hazard and how it impacts local communities; with some work, they can include tips on adopting to local instructors area of interest

• Stephen – both liked connecting landslides to societal infrastructure but also sediment budgets and delivery into reservoir systems (ex. Puerto Rico is getting reservoirs filled up); also beaches suffer from reservoir capture

• Jon – thinking since last session has been focusing on places based too; droughts to floods can mean very different things in different parts of the country – wrestling with ideas still; challenge of making broadly applicable but also some local data sets if possible; how do they keep track of water in their water sheds

• Jessica – using GPS is variety of settings; what is a summary for different topics? Ideas: Policy makers; advice to artists? – maybe try to capture a variety of audiences so different students relate to something more

THOUGHTS/IDEAS FROM ACTIVITIES SINCE LAST

WEBINAR

--Working with your partner – questions/ideas?

--Implementing interdisciplinary societal aspect? Did the article or others like it spur your thinking in any way?

• Bruce: game plan for meeting; divided up tasks; in good shape; divide up existing units to become most familiar and then expand from there

• Susan: highlight uncertainty; more original data and data manipulation; consider existing units reference to Blooms

• Bobby: had a lot opportunity to consider societal aspects related to the recent hurricanes; tying in political or social

• Rachel: talked and got really excited about societal impacts of monitoring volcanoes – Hawaii, Mt St Helens, Yellowstone; parks & tourism; management of resources, animals, tourists, forestry, orchid industry; accurate communication to the public that will be trusted; govt may be downplaying actual Yellowstone (suggestions: consider atmosphere and climate too)

LINKING GOALS AND PROCESS: THE MATERIALS DESIGN RUBRIC

1. Guiding Principles

2. Learning Goals and Outcomes

3. Assessment and Measurement

4. Resources and Materials

5. Instructional Strategies

6. Alignment

A. Grand ChallengesB. Interdisciplinary problems

(geoscience & social science tied together)

C. Nature and methods of scienceD. Authentic geodesy data and

inquiryE. Quantitative skills

Identify Module

Learning Goals

Identify teaching &

learning outcomes for

individual units

Determine how to assess and measure

student success on goals and outcomes

Design teaching

resources and materials to

match assessments

Plan Instructional Strategies to implement

teaching resources

THE APPROACH

Guiding Principles

Identify Module

Learning Goals

Identify teaching &

learning outcomes for

individual units

Determine how to assess and measure

student success on goals and outcomes

Design teaching

resources and materials to

match assessments

Plan Instructional Strategies to implement

teaching resources

THE APPROACH

Guiding Principles

Identify Module

Learning Goals

Identify teaching &

learning outcomes for

individual units

Determine how to assess and measure

student success on goals and outcomes

Design teaching

resources and materials to

match assessments

Plan Instructional Strategies to implement

teaching resources

THE APPROACH

LEARNING GOALS/OUTCOMES

Learning goals identify significant and essential learning that students can demonstrate by the end of the module or course.

– Student centered

– Measurable

– Align with one more assessment elements

LEARNING GOALS: WHAT AND WHY

• What is a “level” for a learning goal?

– How are module-level goals (also called “overarching goals”) different from unit-level or activity-level goals/outcomes?

• A module-level goal is still measurable, but has a broader focus. Unit-level or activity-level goals will be more specific and will mention specific tasks or practices that students will master or least become proficient in.

EXAMPLE OF MODULE GOALS

UNIT-LEVEL LEARNING GOALS

BLOOM’S TAXONOMY – COGNITIVE LEVELS

Three (3) Domains of learning:

1. Cognitive (Knowledge)

• Creating

• Evaluating

• Analyzing

• Applying

• Understanding

• Remembering

2. Affective (Feelings, Emotion)

--Attitude, Sense of Self

Higher CognitiveOrders (HOCs)

Lower CognitiveOrders (LOCs)

3. Behavioral (Skills)

--Manual or Physical Skills

BLOOM’S TAXONOMY: ACTION VERBS

– Creating - Create, Invent, Compose, Predict, Plan, Construct, Design, Propose, Devise, Formulate, Combine, Hypothesize, Synthesize, Forecast.

– Evaluating - Judge, Select, Choose, Decide, Justify, Debate, Verify, Argue, Recommend, Assess, Discuss, Rate, Prioritize, Determine, Critique, Evaluate, Criticize, Weigh, Value, Estimate, Defend.

– Analyzing - Analyze, Distinguish, Examine, Compare, Contrast, Investigate, Identify, Explain, Deduce.

– Applying - Solve, Show, Use, Illustrate, Construct, Complete, Examine, Classify, Choose, Interpret, Make, Apply, Produce, Translate, Calculate.

– Understanding - Explain, Interpret, Outline, Discuss, Distinguish, Predict, Restate, Translate, Compare, Describe, Relate, Generalize, Summarize.

– Remembering - List, Describe, Relate, Locate, Write, Find, State, Name, Identify, Label, Recall, Define, Recognize, Match, Reproduce, Memorize.

Note: There is some overlap in the verbs, it depends on the level of the activity you are attempting to capture with the goal.

Low

er-O

rde

rH

igh

er-O

rde

r

WRITING LEARNING GOALS

• An action verb identifying performance

• A learning statement specifying the learning

• A standard for acceptable performance

Students will be able to solve interdisciplinary problems.

Students will be able to describe how economic, social and political issues bear on scientific problems.

At the end of the course, students will be able to convincingly make the case that economic, social and political issues are relevant when addressing problems related to climate change.

For

Exam

ple

EVALUATING LEARNING GOALS (ADAPTED FROM ACTIVITY BY B.

TEWKSBURY AND H. MACDONALD)

Understand the fundamental concepts of fluid dynamics.How would you evaluate this goal?

1. Student-centered? (rather than teacher-centered)

2. Concrete? (or vague and abstract)

3. What Bloom’s Taxonomy level? (1-Remember, 2- Understand, 3-Apply, 4-Analyze, 5-Evaluate, 6-Create)

4. What level is it? (module-level? unit-level?)

5. Make this goal more challenging? (Hint: think “verb”)

6. Measureable? How? (Suggested activity/exercise to measure?)

EVALUATING LEARNING GOALS

Describe the seven major disasters covered in the course and explain the geologic process associated with the disasters.

1. Student-centered? (rather than teacher-centered)

2. Concrete? (or vague and abstract)

3. What Bloom’s Taxonomy level? (1-Remember, 2- Understand, 3-Apply, 4-Analyze, 5-Evaluate, 6-Create)

4. What level is it? (module-level? unit-level?)

5. Make this goal more challenging? (Hint: think “verb”)

6. Measureable? How? (Suggested activity/exercise to measure?)

EVALUATING LEARNING GOALS

Examine three seismograms from different stations, locate the arrival of P-waves and S-waves and, given a uniform time axis and a map of the three stations, be able to estimate the location of the epicenter.

1. Student-centered? (rather than teacher-centered)2. Concrete? (or vague and abstract)3. What Bloom’s Taxonomy level? (1-Remember, 2- Understand, 3-

Apply, 4-Analyze, 5-Evaluate, 6-Create)4. What level is it? (module-level? unit-level?)5. Make this goal more challenging? (Hint: think “verb”)6. Measureable? How? (Suggested activity/exercise to measure?)

OVERARCHING GOALS & LEARNING OUTCOMES

Goal: At the end of the course, students be able to convincingly make the case that economic, social and political issues are relevant when addressing problems related to climate change.

Learning Outcomes: At the end of this unit, students will be able to correctly:• Describe five economic impacts of climate change• Describe three social impacts of climate change• Compare and contrast two major political positions related to

climate change

Identify Module

Learning Goals

Identify teaching &

learning outcomes for

individual units

Determine how to assess and measure

student success on goals and outcomes

Design teaching

resources and materials to

match assessments

Plan Instructional Strategies to implement

teaching resources

The Approach

Measure the learning outcomes Criterion referenced Consistent with course activities and resources Sequenced, varied and appropriate Address various cognitive levels

• Compare your predictive map that you made to the 2012 GRACE map.

• For each of the study areas, decide whether your prediction for ice mass change corresponds to the actual ice mass change measured by GRACE.

• If there is a difference between your prediction and the GRACE data, briefly describe how your prediction was different from the GRACE data.

ASSESSMENT AND MEASUREMENT

ASSESSMENT AND MEASUREMENT

Learning Outcome: Compare and contrast two major political positions related to climate change

Assessment: You just accepted a position as a Congressional intern for the Speaker of the House, John Boehner (R), Ohio. As Speaker, he is responsible for maintaining Republican Party unity. He is preparing for hearings on proposed EPA regulations related to CO2 emissions. Write a one page summary comparing and contrasting the two main political views related to climate change that he will have to contend with during these hearings.

Identify Module

Learning Goals

Identify teaching &

learning outcomes for

individual units

Determine how to assess and measure

student success on goals and outcomes

Design teaching

resources and materials to

match assessments

Plan Instructional Strategies to implement

teaching resources

THE APPROACH

TEACHING MATERIALS & RESOURCES

Identify Module

Learning Goals

Identify teaching &

learning outcomes for

individual units

Determine how to assess and measure

student success on goals and outcomes

Design teaching

resources and materials to

match assessments

Plan Instructional Strategies to implement

teaching resources

THE APPROACH

INSTRUCTIONAL STRATEGIES

Support learning goals Promote student engagement Develop student metacognition Opportunities to practice

communicating geoscience Scaffold learning

INSTRUCTIONAL STRATEGIES

--What are instructional strategies you find most effective?--Implementation challenges?--Intentional reflection (metacognition)?• Kaatje: Uses metacognition throughout the course;

journal – what do you know, what did you learn, what helps you to learn

• Venkatesh: Have activities throughout the semester; giving data, looking at it, related to what they are learning in class; if you use method on one data set apply to another; applying different methods in different contexts

• Jon: try things individually, talk to neighbors; group work

INSTRUCTIONAL STRATEGIES

--What are instructional strategies you find most effective?--Implementation challenges?--Intentional reflection (metacognition)?• Susan: give students blank graph and have them predict

what the graph “should” or “will” look so then it makes them more invested or aware when they see the actual data unfold; own it a lot better

• Rachel: look at data using jigsaw activities; work together in first group to become “expert” in some aspect of the data, then in the second group they have to explain it and learn from others to solve new problem

• Bobby: different strategies are important for different courses; ex. in structural geology – critique each other’s work;

Identify Module

Learning Goals

Identify teaching &

learning outcomes for

individual units

Determine how to assess and measure

student success on goals and outcomes

Design teaching

resources and materials to

match assessments

Plan Instructional Strategies to implement

teaching resources

THE APPROACH:PHASE 2

LINKING GOALS AND PROCESS: THE MATERIALS DESIGN RUBRIC

1. Guiding Principles

2. Learning Goals and Outcomes

3. Assessment and Measurement

4. Resources and Materials

5. Instructional Strategies

6. Alignment

SERC CONTENT MANAGEMENT SYSTEM

• Great tutorialsare available

https://serc.carleton.edu/serc/cms

FOLLOW UP WORK BEFORE FEBRUARY 5 WEBINAR

• Navigate to “Reports” page in your module and fill in “Team Member Information” for you (near the bottom of the page)

• Navigate to “Checkpoint 1 Workpage” and copy in the notes related to Alignment with Guiding Principles

• Do a call with your author partner to discuss module goals. Input ideas to “Checkpoint 1 Workpage”

• Complete Jan 22 Roadcheck