Smarter Balanced 202: Evidence-Centered Design (Modern Approach to Designing Items and Tasks) Holly...

Smarter Balanced 202:

Evidence-Centered Design

(Modern Approach to Designing Items and

Tasks)

Holly Carter, Assessment Policy AnalystEmail: holly.carter@state.or.us

Ken Hermens, ELA Assessment SpecialistEmail: ken.hermens@state.or.us

Introduction to Evidence-Centered Design (Modern Approach to Designing Items and Tasks)

Traditional Item Development

Evidence-Centered Design

Versus

Claims Assessment Targets

Task Models Evidence

Traditional Approach to Item Development

1.2.1 Apply reference skills to determine word meanings.1.2.2 Apply vocabulary strategies in grade level text.1.3.1 Understand and apply new vocabulary.1.3.2 Understand and apply content/academic vocabulary.1.4.1 Know common sight words appropriate to grade-level.1.4.2 Apply fluency to enhance comprehension.1.4.3 Apply different reading rates to match text.2.1.1 Understand how to use questioning when reading.2.1.2 Understand how to create mental imagery.2.1.3 Understand and determine important or main ideas and important details in

text.2.1.4 Apply comprehension monitoring strategies before, during, and after reading:

use prior knowledge/schema.2.1.5 Apply comprehension monitoring strategies before, during, and after reading:

predict and infer.2.1.6 Apply comprehension monitoring strategies before, during, and after reading:

monitor for meaning, create mental images.2.1.7 Apply comprehension monitoring strategies during and after reading:

summarize informational/expository text and literary/narrative text.

Traditional Approach to Item Development

1.2.1 Apply reference skills to determine word meanings.1.2.2 Apply vocabulary strategies in grade level text.1.3.1 Understand and apply new vocabulary.1.3.2 Understand and apply content/academic vocabulary.1.4.1 Know common sight words appropriate to grade-level.1.4.2 Apply fluency to enhance comprehension.1.4.3 Apply different reading rates to match text.2.1.1 Understand how to use questioning when reading.2.1.2 Understand how to create mental imagery.2.1.3 Understand and determine important or main ideas and important details in text.2.1.4 Apply comprehension monitoring strategies before, during, and after reading:

use prior knowledge/schema.2.1.5 Apply comprehension monitoring strategies before, during, and after reading:

predict and infer.2.1.6 Apply comprehension monitoring strategies before, during, and after reading:

monitor for meaning, create mental images.2.1.7 Apply comprehension monitoring strategies during and after reading:

summarize informational/expository text and literary/narrative text.

Traditional Approach to Item Development

1.2.2

1.3.2

2.1.3

2.1.5

2.1.7

Traditional Approach to Item Development

Item:

Beth says that 2 + 4 = 6. Explain why Beth is correct.

Content Standard 2.2.3: Perform addition accuratelyfor single and two digit numbers.

Applying Evidence-Centered Design to Item and Task Development

Beth says that 2 + 4 = 6.

Explain why Beth is correct.

Content Standard 2.2.3:Perform addition accurately for single and two digit numbers.

Weak Evidence

Applying Evidence-Centered Design to Item and Task Development

2 + 4 = ____

Content Standard 2.2.3:Perform addition accurately for single and two digit numbers.

Stronger Evidence

Applying Evidence-Centered Design toItem and Task Development

Beth says that 2 + 4 = 6.

Explain why Beth is correct.

2 + 4 = ____

Content Standard 2.2.4: Perform mathematical

operations and justify solutions.

Content Standard 2.2.3: Perform addition accurately for single and

two digit numbers.

6 Key Concepts of Evidence-Centered Design

1. Define the domain

2. Define claims to be made

Example of a Claim

English Language Arts Literacy Claim #1

Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts.

6 Key Concepts of Evidence-Centered Design

1. Define the domain

2. Define claims to be made

3. Define assessment targets

Example of an Assessment Target

Grade 11 – Assessment Target

Analyze the figurative (e.g., euphemism, oxymoron, hyperbole, paradox) or connotative meanings of words and phrases used in context and the impact of these word choices on meaning and tone.

6 Key Concepts of Evidence-Centered Design

1. Define the domain

2. Define claims to be made

3. Define assessment targets

4. Define evidence required

Example of an Evidence Description

Evidence for Assessment Target 1

When reading informational or argumentative texts, students analyze the figurative or implied meanings of words or phrases as they are used in a text and analyze how the choice of these particular words affects meaning and tone.

6 Key Concepts of Evidence-Centered Design

1. Define the domain

2. Define claims to be made

3. Define assessment targets

4. Define evidence required

5. Develop Task Models

Example of a Task Model

Task Model 1

A constructed response for which the student is prompted to identify an example of figurative language, explain the meaning, and describe how it affects meaning and tone. Stimulus text should be on grade level.

6 Key Concepts of Evidence-Centered Design

6. Develop Items or Performance Tasks

1. Define the domain

2. Define claims to be made

3. Define assessment targets

4. Define evidence required

5. Develop Task Models

Building a Logical Argument

Student Response

Evidence

Assessment Target

Claim

Common Core State Standards

6 Key Components of Evidence-Centered Design

6. Develop Items or Performance Tasks

1. Define the domain

2. Define claims to be made

3. Define assessment targets

4. Define evidence required

5. Develop Task Models

Examining an ItemThrough the Lens of Evidence-Centered Design

Use the line buttons to separate the rectangle into 6 equal sections. Then click on the sections to shade the area of the rectangle that represents 2/3 of the area of the whole rectangle.

Task Model:Partition tool is used to divide a given quadrilateral into up to 20 equal sections and to highlight fractions of the quadrilateral that represent halves, thirds, quarters, or fifths.

Task Model:Partition tool is used to divide a given quadrilateral into up to 20 equal sections and to highlight fractions of the quadrilateral that represent halves, thirds, quarters, or fifths.

Evidence:Student can create visual representations of fractions that range from one-half to four-fifths.

Using Evidence-Centered Design to Guide Item Design

1. What evidence is required given the assessment target I am measuring?

2. What are the key features that must be included in the item?

3. Will this item allow for the production of the evidenceI am seeking?

4. Is there anything about this item that may make it more difficult to collect evidence from some students?

Resources Smarter Balanced Website http://www.smarterbalanced.org/

Smarter Sample Items and Performance Tasks

http://www.smarterbalanced.org/smarterbalanced-assessments/#item

Smarter Item Writing and Review Materials http://www.smarterbalanced.org/smarter-balanced-assessments/

item-writing-and-review/

Common Core State Standards Toolkit http://www.ode.state.or.us/search/page/?id=3430

Common Core State Standards Assessment Resources http://www.ode.state.or.us/search/page/?id=3298

Next Smarter Balanced Webinar

Smarter Balanced 203:CCSS Across the Curriculum

March 5, 2013 (3:30-4:30 p.m.)*ELA Items and Performance Tasks that have Science

and Social Sciences context*Instructional gaps between CCSS and current Oregon

Math and ELA Standards

http://www.ode.state.or.us/news/events/.

Thank you! Please send additional input, questions, or

anecdotes to:Rachel Aazzerah (Math/Smarter Balanced Pilot Test)

Rachel.Aazzerah@state.or.usDerek Brown (Essential Skills)Derek.Brown@state.or.usHolly Carter (SBAC Accessibility/Accommodations)Holly.Carter@state.or.usMark Freed (Math)Mark.Freed@state.or.usKen Hermens (English/Language Arts)Ken.Hermens@state.or.us