Diane Ebert-MayLyman Briggs School

Department of Plant BiologyMichigan State University

Assessment - What Evidence do You Accept?

HHMI

Our Team at MSU

Doug Luckie - Physiology Janet Batzli - Plant Biology Scott Harrison - Microbiology Tammy Long - Ecology Heejun Lim - Chemistry Education Duncan Sibley - Geology Joyce Parker - Biochemistry

“Consensogram” Directions“Consensogram” Directions

1. Take one color-coded post-it for each question, write the question # in the corner.

2. Write a number between 0-100 on each post-it in increments of 10.3. Do not share responses

“Consensogram” Questions“Consensogram” QuestionsPlease respond on a scale of 0 -100 in increments of 10:1. To what degree is your course based on active, inquiry-based

learning? 2. To what degree are your course learning objectives,

instructional design and assessment aligned?3. How important is it to use multiple kinds of data to assess your

students?4. How often do I use data to make instructional decisions?5. In my department, teaching is as important as research for

graduate students (100 agree - 0 disagree).6. In my department, effective teaching is rewarded.

(100 agree - 0 disagree)

Goals for TodayGoals for Today

As a result of your participation in this workshop, you will...

Participate in analysis of learning: constructive, inquiry, discovery, active, problem-based, cooperative, outcomes based, project-based.

Examine course goals and predicted learning outcomes.

Use data to identify student understanding and misconceptions.

Use assessment techniques

True or False?

Faculty really are very interested in assessing their students’ learning better, but just don’t know how to?

True or False?

Lack of meaningful assessment in undergraduate education occurs because faculty are satisfied to be less accountable in their teaching than they are in their research.

True or False?

Assessing student learning in science is more closely related to what scientists actually do as research than they realize.

Assessment in ‘Teaching’Parallels Assessment in ‘Research’

We collect data with a purpose. Data we collect are aligned with a question about a

problem Questions we ask are meaningful, interesting, fundable. Research methods and designs appropriate for question. Instruments/techniques we use are calibrated. We explain results in the context of our questions. Results drive our next questions. Our ideas are peer reviewed for publication/funding.

What are 3 central questions about learning?

1. What do we want our students to know and be able to do?

1.5. What evidence will we accept that students know and can do?

2. How will we help students get there?

Cognitive Theory

“Learners are not simply passive recipients of information; they actively construct their own understanding.”

Svinicki 1991

Ultimate goal of teaching:

1. Improved student learning.2. Improved student learning.3. Improved student learning.

What Type of Learning?

Bloom (1956) described major categories in

CognitiveDomain of Educational Objectives

Convergent Thinking

Knowledge - remember material Comprehension - grasp the

meaning of material Application - use learned

material in new concrete situations

– Adapted from Grolund (1970)

Divergent Thinking

Analysis - break down material to understand organizational structure

Synthesis - put parts together to form a new whole

Evaluation - judge value of material for a purpose

– Adapted from Grolund (1970)

Classroom Environment

Teacher inspires students to struggle with the discipline - both within and outside the classroom.

Teacher needs evidence from students about their progress in learning

What is assessment?

Data collection with a purpose Courses: gather data about our

students’ learning.

What type of data do we gather?

Depends on the evidence we will accept that students have learned what we want them to learn.

Data must be aligned with the course goals. Measures of knowledge, attitudes, and skills.

»tests, extended responses, concept maps, »research papers, teamwork, communication

Basic Objectives Biology Department: Hope College

…students to Be active learners - that is to learn

biology by doing biology. Learn the basic set of principles

and factual knowledge about each of the major areas of biology.

Write a Learning GoalWrite a Learning Goal

Individually, write a learning goal for one of your courses (in the context of the departmental objectives)

e.g., »students will demonstrate…»students will be able to …

Next

Share your goal with a partner in your group

Write both goals on large post-its Beneath, write possible

performance expectations

Functions of Assessment Data

Formative: diagnostic feedback to students/instructor

Summative: description of students’ level of attainment

Evaluative: curricular feedback to instructor » (e.g., effectiveness of field trip, lab

investigation) Educative: students engaged in interesting,

challenging experiences to develop further insight and understanding (Hodson 1992)

In effect...

Assessment IS a form of learning.

Goal => Assessment

Students will be able to demonstrate their understanding of photosynthesis and respiration in a variety of problems.

Tools: multiple forms of assessment

Common Misconceptions: Photosynthesis & Respiration

• Photosynthesis as Energy: Photosynthesis provides energy for uptake of nutrients through roots which builds biomass. No biomass built through photosynthesis alone.

• Plant Altruism: CO2 is converted to O2 in plant leaves so that all organisms can ‘breathe’.

• All Green: Plants have chloroplasts instead of mitochondria so they can not respire.

•Thin Air: CO2 and O2 are gases therefore, do not have mass and therefore, can not add or take away mass from an organism.

Multiple choice question (pre-post)

Plants gain a tremendous amount of weight (dry biomass) as they grow from seed to adult. Which of the following substances contributes most to that weight gain

a. compounds dissolved in soil water that are take up by plant roots

b. waterc. molecules in the air that enter through holes in the plant

leavesd. organic material in the soil taken up directly by plant rootse. solar radiation

Carbon Cycle Problem (post) Two fundamental concepts in ecology are “energy flows” and

“matter cycles”. In an Antarctic ecosystem with the food web given above, how could a carbon atom in the blubber of the Minke whale become part of a crabeater seal? Note: crabeater seals do not eat Minke whales. In your response include a drawing with arrows showing the movement of the C atom. In addition to your drawing, provide a written description of the steps the carbon atom must take through each component of the ecosystem Describe which biological processes are involved in the carbon cycle.

Antarctic Food Web

Radish Problem

Experimental Setup: Weighed out 3 batches of radish seeds each weighing

1.5 g. Experimental treatments:

»1. Seeds not moistened (dry) placed in LIGHT »2. Seeds placed on moistened paper towels in

LIGHT»3. Seeds placed on moistened paper towels in DARK

Problem (cont)

After 1 week, all plant material was dried in an oven overnight (no water left) and plant biomass was measured in grams.

Predict the biomass of the plant material in the various treatments (use think-pair-share). » Light, No Water» Light, Water» Dark, Water

Results: Weight of Radish Plants

1.46 g 1.63 g 1.20 gWrite an explanation about the

results.(Remember all treatments started as 1.5g).

Misconceptions => Assessment => Instruction

What data do you want from the assessment?

What do you do when you identify student misconceptions?

How will the data influence your instructional design?

Gene-DNA-Chromosome Students could explain

transcription & translation but not the relation...

“Gene-DNA-Chromosome.”

Concept mapping forces students to “Think different” and confront their (mis) understanding.

Concept Maps

Concept Maps

Visual Diagramsor Models

are

Concept Maps

Visual Diagramsor Models

are

Reflection &Learning

promotes

Assessment

Used for

Organization

Concept Maps

Visual Diagramsor Models

are

Concepts

display

connectedwith

Linking Words

Reflection &Learning

promotes

Assessment

Used for

Organization

Concept Maps

Visual Diagramsor Models

are

Knowledge or Understanding

represent

Concepts

display

connectedwith

Linking Words

Hierarchy

has

Structure

has

Reflection &Learning

promotes

Assessment

Used for

Organization

Concept Maps

Visual Diagramsor Models

are

Knowledge or Understanding

represent

Concepts

display

connectedwith

Linking Words

Hierarchy

has

Structure

has

Reflection &Learning

promotes

Assessment

Used for

OrganizationContext

is constructedwith

NewInformation

PriorKnowledge

Multiple Choice … … Concept Maps … … Essay … … Interview

high Ease of Assessment low

low Potential for Assessment of Learning high

Theoretical Framework•Ausubel 1968; meaningful learning•Novak 1998; visual representations•King and Kitchner 1994; reflective judgement•National Research Council 1999; theoretical frameworks for assessment

Assessment Gradient

Assessment and Research

Assessment answers the “what” questions about student learning.

Research provides explanations about the “why” and “how” of student understanding.

Open-ended questions Open-ended questions

Align with learning goalsAlign with learning goals What thinking skills do you wish to What thinking skills do you wish to

assess, choose one questioning formatassess, choose one questioning format» interpret data?interpret data?»write conclusions from previous work?write conclusions from previous work?»describe?describe?»solve a problem?solve a problem?

Writing Open-ended Questions

Write a description of the situation.Write a description of the situation. Write the directions for writing.Write the directions for writing. Develop a simple rubricDevelop a simple rubric

» Conceptual understandingConceptual understanding» Content knowledgeContent knowledge» Critical-thinking processesCritical-thinking processes» Communication skillsCommunication skills

Goal: explain evolution by natural selection

Individual Problem

Explain the phenotypic changes in the tree and the animal. Use your understanding of evolution by natural selection.

How do we develop rubrics?

Describe the goals for the activity, problem, task

Select the assessment tasks aligned with goals Develop performance standards Differentiate levels of responses based on

clearly described criteria Rate (assign value) the categories

Level of Achievement General Approach ComprehensionExemplary(5 pts)

• Addresses thequestion.• States a relevant,justifiable answer.• Presents arguments ina logical order.• Uses acceptable styleand grammar (noerrors).

• Demonstrates an accurate andcomplete understanding of thequestion.• Backs conclusions with dataand warrants.• Uses 2 or more ideas,examples and/or arguments thatsupport the answer.

Adequate(3 pts)

• Does not address thequestion explicitly,although does sotangentially.• States a relevant andjustifiable answer.• Presents arguments ina logical order.• Uses acceptable styleand grammar (oneerror).

• Demonstrates accurate but onlyadequate understanding ofquestion because does not backconclusions with warrants anddata.• Uses only one idea to supportthe answer.• Less thorough than above.

Needs Improvement(1 pt)

• Does not address thequestion.• States no relevantanswers• indicatesmisconceptions.• Is not clearly orlogically organized.• Fails to use acceptablestyle and grammar (twoor more errors).

• Does not demonstrate accurateunderstanding of the question.• Does not provide evidence tosupport their answer to thequestion.

No Answer (0 pts)

Scoring Rubric for Quizzes and Homework

Advantages of Scoring Rubrics

Improve the reliability of scoring written assignments and oral presentations

Convey goals and performance expectations of students in an unambiguous way

Convey “grading standards” or “point values” and relate them to performance goals

Engage students in critical evaluation of their own performance

Save time but spend it well

Limitations of Scoring Rubrics

Problem of criteria Problem of practice and regular use

Scoring Rubric website:» http://www.wcer.wisc.edu/nise/cl1/flag/

Sample Rubrics for Organismal Biology http://www.msu.edu/course/lbs/144/f01

Proposal Assessment Plan - Essentials

Agree on goals and objectives for learning Design and implement a thoughtful approach

to planning Involve individuals from on/off campus Select/design data collection approaches Examine, share, act on assessment findings Regularly examine assessment process