Formative Assessment: What STEM professors think they’re teaching & what

students think they’re learning. Katherine Yngve, Purdue University

Elizabeth Brewer, Beloit College

Types of Assessment

Summative Purpose is to gather feedback which will be helpful to both student & instructor in the learning process.

• Learning Style Inventories • Quiz • Homework • Reflection Essay or Activity • Interview/Observation w/ Feedback • Portfolio • Rubric

Formative

Purpose is to measure the level of proficiency which has been gained due to an instructional unit or intervention.

• Final Exam

• Pre/Post Survey

• Capstone Project

• Term Paper; Senior Recital, etc.

• Portfolio

• Rubric

Session Overview

Beloit College What the Curriculum & Students Say

• Life-long Learning – Study Abroad Learning Goals

• Sequential, Iterative Assessment

• Essay Prompts

• What the Research Shows

• Hands-On Activity: Assessing Essays

Purdue University What Profs Think They’re Teaching

• Embedded Learning & the UG Core Curriculum

• Intercultural Learning Rubric

• What the Research Shows

• Hands-On Activity: Assessment Tools to Move the Learning Needle

Debriefs & Discussion as time allows!

Beloit Curriculum: Life-Long Learning Skills

Experiences outside the classroom that allow students to:

“connect, adapt, and apply knowledge and skills gained in one setting to new contexts”

Preparation for Life-Long Learning Beloit First Year Seminars

Encounter and Exploration – campus, mission, curriculum

Agency – putting students in charge of their path through college

Higher-Order Communication Skills – writing and speaking

Social Identities – self, others, communities

Reflection

Life-Long Learning: Beloit Study Abroad Learning Goals

new perspectives on studies

intercultural competencies and communication skills

engage with situations and questions that challenge assumptions and values,

articulate cultural experience

learn about and from host environments

First-Year seminar

Study Abroad

Post-SA Essays

Study Abroad Plan

1. Students Develop Agency & Communication Skills

2. Students Identify Learning Goals &

Connections to Studies

3. Students Execute & Adjust Plans 4.

Students Reflect: What, Why, How?

So What? What Next?

Stages in

Formative Assessment

Rubrics used by

• Applicants: To prepare a study abroad plan

• Faculty: To evaluate the plans

• Faculty/staff/student teams: To assess the development of life-long learning skills as evidenced in post-study abroad reflective essays

Study Abroad Application Prompt 1:

Learning Goals

A. State three to five things you hope to learn abroad.

B. How will your choice of university/program and location help you achieve your goals?

C. How might differences in educational philosophy and/or delivery affect your ability to achieve your goals?

Study Abroad Application Prompt 2:

Connection to Studies

Connection to Studies

How will the study abroad you propose build on your education to date as well as connect to your education when you return to campus?

Note: There are four prompts. The others focus on preparation and social identities

What Research Shows

• Life-long learning skills advance more during study abroad than intercultural competency

Analysis of post-study abroad reflective essays

• Asking SA applicants to identify their own learning goals and make connections to studies has helped them make stronger connections between study abroad and their studies on campus

Analysis of presentations on study abroad experiences in an annual international symposium

• Students are better able to articulate their learning if given chances post-study abroad for reflection and dialogue

Analysis of reflective writing

Impacts of Formative Assessment

International Symposium content

Connections to Studies

Cultural / Intercultural Learning

Questioning own Assumptions / Values

Host Country / Site

2014 96% 60% 36% 92%

2002 32% 68% 4% 96%

Analysis of 387 abstracts

Impacts of Formative Assessment

International Symposium content sources

Field / Independent Research or Course Project

Informal Observation

Internship

2014 68% 28% 8%

2002 44% 48% 8%

Group-work Task A 1. Read and score application essays using the rubric

labeled “original.” What feedback would you give the student?

2. Compare the original and new prompts and rubrics. Do the changes make sense? Are the new prompts clearer?

3. Brainstorm ideas for how your organization/institution could help students exercise agency around study abroad and articulate their plans.

Purdue Core Curriculum (2012 - 2015)

Assessing Faculty-Led Programs

What will your short-term program teach?

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

AG (11) CLA (18) EDU (8) ENG (10) HHS (5) MGMT (5) TECH (12)

World

Comm

Curious

Open

Self-Aware

Empathy

What will your short-term program teach?

College of Agriculture Teamwork Civic Engagement

College of Liberal Arts Comfort with Difference Knowledge of Intercultural

Theory Social Justice Orientation

College of Education Social Justice Orientation Comfort with Difference

College of Engineering Teamwork Knowledge of Intercultural

Theory Civic Engagement

College of Health & Human Sciences Teamwork

College of Technology Teamwork

College of Management Teamwork

Selected Formative Assessment Tools

MAXSA Handbook

• Language (Learning) Strategy Survey

• Culture (Learning) Strategy Survey

• Culture Mapping Survey

• Stress Factors Inventory

• You as a Culturally Diverse Person

Other sources

• Kolb Learning Style Inventory

• Cross-Cultural Adaptability Index

• Intercultural Effectiveness Scale

• Intercultural Conflict Style Inventory

• Hofstede Centre Culture Compass

• My Cultural Awareness Profile (for teachers; by NAFSA)

• Rubin’s Behavioral Observation Rubric

• Foreign Language Classroom Anxiety Scale

• Liebowitz’ Social Anxiety Scale

• Multi-factor Leadership Questionnaire

• Scale of Ethno-cultural Empathy

Group-Work Task B

1) Select a program description & review the 6

elements of both AAC&U rubrics in packet.

2) Identify 2-3 rubric items that this particular course

addresses (e.g. “Curiosity” or “Action & Reflection”)

3) Review the formative assessment tools on offer

4) Jointly select at least two assessment tools that

your group believes will move the students forward on at least one objective

5) Be prepared to defend your selection of the tools in

open forum debrief!