Smarter Balanced Rubrics, Scoring, and Reporting

March 24, 2015

Office of Strategy, Innovation and PerformanceAssessment Section

(808) 733-4100

Rubrics

• Mathematics• Scoring Guide for Short-Text Mathematics Items

• ELA/Literacy• Rubrics for Full Writes (Practice Test)• Rubrics for Short-Text Items

• Rubric templates• for short text items are embedded in the item specifications for Claim 1 for targets 4 /11 and

targets 2/9• For research short-text items are in the performance task

• Rubrics for Brief Writes• For targets 1a, 3a, and 6a

Scoring Guidelines

Smarter Balanced Scoring Guide for Selected Short Text Mathematics ItemsGeneral Rubric for 2-Point Items (p. 3)Sample Grade 4• Sample 2-Point Short-Text Item for Grade 4 (Lemons) (p. 4)• Sample Item Specific Rubric for Grade 4 Lemon Item (p. 5)• Anchor Responses 1 – 9 for Lemon Item (pp. 6 – 14)Sample Grade 8• Sample 2-Point Short-Text Item for Grade 4 (Earth and Moon) (p. 15)• Sample Item Specific Rubric for Grade 8 Earth and Moon Item (p. 16)• Anchor Responses 1 – 10 for Earth and Moon Item (pp. 17 – 26)

Smarter Balanced Scoring Guide for Selected Short Text Mathematics Items (range up to 4)Sample High School• Sample 2-Point Short-Text Item for High School (Parabolas)(p. 27)• Sample Item Specific Rubric for High School Parabolas Item (p. 28)• Anchor Responses 1 – 8 for Parabolas Item (pp. 29 - 36)• General Rubric for 4-Point Items (p. 37)• General Rubric for 3-Point Items (p. 38)• General Rubric for 2-Point Items (p. 39)• General Rubric for 1-Point Item (p. 39)

Smarter Balanced Mathematics General Rubric for 2-Point Items (p. 39)

Item Specific Rubric for 2-pt Grade 4 Lemon Item

ELA/Literacy Full Write Rubrics

Rubrics for the Full Writes found in Practice Tests

Scoring Guides for Practice Test Items

Excerpt Grades 3-5 PT Writing Rubric

http://www.smarterbalanced.org/smarter-balanced-assessments/

Excerpt Grade 4 Item Specific Rubric (from Practice Test)

Rubric Templates for Short-Text ELA/Literacy Items

Rubric Templates for Short-Text Items • Rubric templates are embedded in the item specifications for Claim 1

short-text items (targets 4/11 and targets 2/9) and research short-text items in the performance tasks.• All short text rubrics are 0-2 point rubrics

Item Specifications

http://www.smarterbalanced.org/smarter-balanced-assessments/

Grade 4, Claim 1 Target 4 (1/6)

Grade 4, Claim 1 Target 4 (2/6)

Grade 4, Claim 1 Target 4 (3/6)

Grade 4, Claim 1 Target 4 (4/6)

Grade 4, Claim 1 Target 4 (5/6)

Grade 4, Claim 1 Target 4 (6/6)- Excerpt p.9

ELA/Literacy Brief Write Rubrics

ELA Brief Write Rubric Target 1a Narrative (1/3)

ELA Brief Write Rubric Target 1a Narrative (2/3)

ELA Brief Write Rubric Target 1a Narrative (3/3)

ELA Brief Write Rubric Target 3a Informational/Explanatory (1/3)

ELA Brief Write Rubric Target 3a Informational/Explanatory (2/3)

ELA Brief Write Rubric Target 3a Informational/Explanatory (3/3)

ELA/L Brief Write Rubric Target 6a Opinion/Argumentative (1/6)

ELA Brief Write Rubric Target 6a Opinion/Argumentative (2/6)

ELA Brief Write Rubric Target 6a Opinion/Argumentative (3/6)

ELA Brief Write Rubric Target 6a Opinion/Argumentative (4/6)

ELA Brief Write Rubric Target 6a Opinion/Argumentative (5/6)

ELA Brief Write Rubric Target 6a Opinion/Argumentative (6/6)

36

Degree of Alignment

Standards

Assess-ment

Assessment

Standards

Standards

StandardsAssessment

Assessment Items

From: TILSA Alignment Tool Dissemination Workshop, July 2005 in Boston Massachusetts Webb

Scores

• Composite • Scale• Claim Level• Threshold

Smarter Balanced Scale Scores

• Vertical scale• Increase across grade levels• Current level of achievement and growth over time• Aggregate scores for schools and complex areas• Can measure gaps for different groups of students

Composite ScoreSummative Assessment for ELA/Literacy

Computer Adaptive Test and Performance Task• Reading• Writing• Listening • Research

One Composite Score for ELA

Composite ScoreSummative Assessment for Mathematics

Computer Adaptive Test and Performance Task• Concepts and Procedures• Problem Solving & Modeling/Data Analysis • Communicating Reasoning

One Composite Score for Math

Claim Level Scores

English Language Arts/Literacy: (1) Reading(2) Writing(3) Listening(4) Research

Mathematics:(1) Concepts and Procedures(2) Problem Solving &

Modeling/Data Analysis(3) Communicating Reasoning

Claim Level AchievementAbove Standard Your test results clearly show that you understand

and are able to apply your knowledge to the standards in this content area of your grade.

At/Near Standard Your test results may be just above or just below the standard, but due to the error band, the result is too close too call.

Below Standard Your test results clearly show that you have not yet met the standard in this content area for your grade.

Raw Scores versus Scale Scores

Peter gets 5 out of 10 easy items correct Raw score of 5 or 50%

Bob gets 5 out of 10 difficult items correct Raw score of 5 or 50%

Peter gets 5 out of 10 easy items correct

Bob gets 5 out of 10 difficult items correct

Lower Scale Score than Bob

Higher Scale Score than Peter

Slide from Achievement Level Setting Workshop in Dallas, TX, October 2014

Achievement Levels

Hawaii State Assessments

Well Below ProficiencyApproaches ProficiencyMeets ProficiencyExceeds Proficiency

Smarter Balanced Assessments

Level 1Level 2Level 3Level 4

SB ELA/Literacy Reported Scale ScoresGrade Level 1 Level 2 Level 3 Level 4

3 2114 - 2366 2367 - 2431 2432 -2489 2490 - 2623

4 2131 - 2415 2416 -2472 2473 - 2532 2533 - 2663

5 2201 - 2441 2442 -2501 2502 - 2581 2582 - 2701

6 2210 - 2456 2457 - 2530 2531 - 2617 2618 - 2724

7 2258 - 2478 2479 - 2551 2552 - 2648 2649 - 2745

8 2288 - 2486 2487 - 2566 2567 - 2667 2668 - 2769

11 2299 - 2492 2493 - 2582 2583 - 2681 2682 - 2795

SB Mathematics Reported Scale ScoresGrade Level 1 Level 2 Level 3 Level 4

3 2189 - 2380 2381 - 2435 2436 - 2500 2501 - 2621

4 2204 - 2410 2411 - 2484 2485 - 2548 2549 -2658

5 2219 - 2454 2455 - 2527 2528 - 2578 2579 -2700

6 2235 - 2472 2473 - 2551 2552 - 2609 2610 - 2748

7 2250 - 2483 2484 - 2566 2567 - 2634 2635 - 2778

8 2265 - 2503 2504 - 2585 2586 - 2652 2653 - 2802

11 2280 - 2542 2543 - 2627 2628 - 2717 2718 - 2862

Mathematics and ELA/Literacy Achievement Level Descriptors – High School

Level 4 The student has exceeded the achievement standard and demonstrates the knowledge and skills in [content area] needed for likely success in entry-level credit-bearing college coursework after high school.

Level 3 The student has met the achievement standard and demonstrates progress toward mastery of the knowledge and skills in [content area] needed for likely success in entry-level credit-bearing college coursework after completing high school coursework.

Level 2 The student has nearly met the achievement standard and may require further development to demonstrate the knowledge and skills in [content area] needed for likely success in entry-level credit-bearing college coursework after high school.

Level 1 The student has not met the achievement standard and needs substantial improvement to demonstrate the knowledge and skills in [content area] needed for likely success in entry-level credit-bearing college coursework after high school.

Mathematics and ELA/Literacy Achievement Level Descriptors – Grades 6-8Level 4 The student has exceeded the achievement standard and demonstrates

advanced progress toward mastery of the knowledge and skills in [content area] needed for likely success in entry-level credit-bearing college coursework after high school.

Level 3 The student has met the achievement standard and demonstrates progress toward mastery of the knowledge and skills in [content area] needed for likely success in entry-level credit-bearing college coursework after high school.

Level 2 The student has nearly met the achievement standard and may require further development to demonstrate the knowledge and skills in mathematics needed for likely success in entry-level credit-bearing college coursework after high school.

Level 1 The student has not met the achievement standard and needs substantial improvement to demonstrate the knowledge and skills in mathematics needed for likely success in entry-level credit-bearing college coursework after high school.

Mathematics and ELA/Literacy Achievement Level Descriptors – Grades 3-5

Level 4 The student has exceeded the achievement standard and demonstrates advanced progress toward mastery of the knowledge and skills in [content area] needed for likely success in future coursework.

Level 3 The student has met the achievement standard and demonstrates progress toward mastery of the knowledge and skills in [content area] needed for likely success in future coursework.

Level 2 The student has nearly met the achievement standard and may require further development to demonstrate the knowledge and skills in mathematics needed for likely success in future coursework.

Level 1 The student has not met the achievement standard and needs substantial improvement to demonstrate the knowledge and skills in mathematics needed for likely success in future coursework.

Score Reports

Target Achievement LevelIcon Target Level Description

Better than performance on the test as a whole

This target is a relative strength. The group of students performed better on items from this target than they did on the rest of the test as a whole.

Similar to performance on the test as a whole

This target is neither a relative strength nor a relative weakness. The group of students performed about as well on items from this target as they did on the rest of the test as a whole.

Worse than performance on the test as a whole

This target is a relative weakness. The group of students did not perform as well on items from this target as they did on the rest of the test as a whole.

Insufficient information Not enough information is available to determine whether this target is a relative strength or weakness.

The Online Reporting System

an overview including the SB Interim Assessments and the Teacher Hand Scoring System

ICA mirrors the summative assessment and the components, recommended sequence and duration is the same. The ICA includes:

• a CAT component (fix form this year), • a classroom activity, and • a performance task for each subject.

The components need to be completed (along with the hand scoring of constructed response items) before scores will be available.

ICA OverviewContent Area Grades

Computer Adaptive Test (CAT) items

Performance Task (PT)

Total “Computer” Time

Classroom Activity (administered

prior to the PT)*Total Time

English Language

Arts/Literacy

3–5 1:30 2:00 3:30 :30 4:00

6–8 1:30 2:00 3:30 : 30 4:00

HS 2:00 2:00 4:00 : 30 4:30

Mathematics

3–5 1:30 1:00 2:30 : 30 3:00

6–8 2:00 1:00 3:00 : 30 3:30

HS 2:00 1:30 3:30 : 30 4:00

Total Timefor both ELA

and Math

3–5 3:00 3:00 6:00 1:00 7:00

6–8 3:30 3:00 6:30 1:00 7:30

HS 4:00 3:30 7:30 1:00 8:30

Blueprints for the ICAs are the same as the summative assessment and can be found at www.alohahsap.org >> Smarter Balanced >> Test Coordinator/Administrator >> General Information

• How much? • How?

The Interim Teacher Hand Scoring System under Smarter Balanced at www.alohahsap.org

Hand ScoringInterim Comprehensive Assessment Items to be Hand Scored

Total ItemsCAT PT PTSubject Grade Test Name Short Ans Short Ans EssayELA 3 ICA - Grade 3 3 2 1 47ELA 4 ICA - Grade 4 3 2 1 48ELA 5 ICA - Grade 5 3 2 1 47ELA 6 ICA - Grade 6 3 2 1 49ELA 7 ICA - Grade 7 3 2 1 49ELA 8 ICA - Grade 8 3 2 1 49ELA 11 ICA - Grade 11 3 2 1 46Mathematics 3 Math ICA - Grade 3 0 4 0 37Mathematics 4 Math ICA - Grade 4 0 3 0 36Mathematics 5 Math ICA - Grade 5 0 2 0 37Mathematics 6 Math ICA - Grade 6 0 2 0 36Mathematics 7 Math ICA - Grade 7 0 2 0 37Mathematics 8 Math ICA - Grade 8 0 2 0 37Mathematics 11 Math ICA - Grade 11 1 4 0 39

Blueprints for the IABs can be found at www.alohahsap.org >> Smarter Balanced >> Test Coordinator/Administrator >> General Information

Available Interim Assessment Blocks:

MathematicsGrade 3   Grade 4   Grade 5

Operations and Algebraic Thinking

  Operations and Algebraic Thinking  

Numbers and Operations in Base

10

Fractions   Numbers and Operations in Base 10

  Fractions

Measurement and Data

  Fractions   Measurement and Data

Mathematics PT   Mathematics PT   Mathematics PT

Grade 6   Grade 7   Grade 8

Ratio and Proportional Relationships

  Ratio and Proportional Relationships

 

Expressions & Equations, with

Proportionality, Statistics & Probability

Geometry   Number System   Geometry

Expressions and Equations

  Expressions and Equations   Functions

Mathematics PT   Mathematics PT   Mathematics PT

High School

Algebra and Functions - Linear Functions

Algebra and Functions - Quadratic Functions

Geometry - Right Triangle Ratios in Geometry

Mathematics PT

Available Interim Assessment Blocks:

ELASummary — Grades 3, 4 and 5

Read Literary Texts

Read Informational Texts

Edit/Revise

Brief Writes

Listen/Interpret

Research

Informational PT

Opinion PT (Grade 3)

Narrative PT (Grade 4, 5)

Summary — Grades 6, 7, and 8

Read Literary Texts

Read Informational Texts

Edit/Revise

Brief Writes

Listen/Interpret

Research

Argument PT

Summary — Grade 11Read Literary Texts Listen/Interpret

Read Informational Texts Research

Edit/Revise Explanatory PT

Brief Writes  

Blueprints for the IABs can be found at www.alohahsap.org >> Smarter Balanced >> Test Coordinator/Administrator >> General Information

• Accessing reports is exactly the same as it has been for HSA• The types of reports available

are similar to those available for HSA• SB Summative reports will

include “Target Level” summaries at the class level. • The data you will see depends

upon your “role” in TIDE

SB Reporting in the Online Reporting System (ORS)

• Assessment• School Year• Scores for Who• Complex Area• Test and Grade

• Click to edit Master text styles• Second level

• Third level• Fourth level

• Fifth level

• Click to edit Master text styles• Second level

• Third level• Fourth level

• Fifth level

Title and Content Layout with List

Questions ?