Evaluating Growth Recognizing and Supporting Educators

Through AchieveNJ

February 28th 2014

2

3

Broad Perspective

Nationwide Education Reform– NCLB– CCSS– PARCC– Race to the Top– ESEA waiver

4

ESEA Waiver

1. College- and Career-Ready Expectations for All Students• developing and administering annual, statewide, aligned,

high-quality assessments, and corresponding academic achievement standards, that measure student growth

2. State-Developed Differentiated Recognition, Accountability, and Support• systems to improve struggling schools… progress…

closing gaps3. Supporting Effective Instruction and Leadership

an evaluation system that will require• continual improvement of instruction; • use multiple valid measures in determining performance

levels, including data on student growth for all students

5

Educator evaluation in New JerseyThree Years in the Making

2010 2011 2012 2013 2014

Educator Effectiveness

Task Force formed

Task Force releases

recommendations

EPAC, Pilot 1 launched

$38 million Race to the

Top award for NJ

Pilot 2 selected

TEACHNJ Act passed

2nd round of

evaluation regulations proposed

All districts launch

improved evaluations

5

EPAC and external Rutgers

reports issued

1st round of evaluation regulations proposed

CCSS adopted

PARCC pilot

6

NOW NOWPerfunctory In-depth

One measure Multiple measuresOne observer Multiple observers

No connection to student growth Student growth counts

No evaluation training Comprehensive training required

Disconnected from professional development

Must be linked to professional development

Minimal educator involvement Educator-driven

New Jersey’s evaluation framework

BEFORE Perfunctory

Measured by observation onlyOne observer

No connection to student growth

Training not requiredDisconnected from

professional development Minimal educator involvement

6

7

NOW NOWPerfunctory In-depth

One measure Multiple measuresOne observer Multiple observers

No connection to student growth Student growth counts

No evaluation training Comprehensive training required

Disconnected from professional development

Must be linked to professional development

Minimal educator involvement Educator-driven

New Jersey’s evaluation framework

BEFORE Perfunctory

Measured by observation onlyOne observer

No connection to student growth

Training not requiredDisconnected from

professional development Minimal educator involvement

7

8

Evaluations use multiple measures

TeacherPractice

Student Growth

Percentile (SGP)

Student Growth

Objective(SGO)

Summative Rating

Practice Student Achievement

9

What do SGPs and SGOs have in common?

SGP

SGO

10

What do SGPs and SGOs have in common?

SGP

SGO

11

What do SGPs and SGOs have in common?

SGP

SGOGrowt

h

12

What do SGPs and SGOs have in common?

SGP

SGOGrowt

hStuden

ts

13

How do SGPs capture growth?

• Student Growth Percentiles (SGP) measure how much a student has learned from one year to the next compared to peers with a similar NJ ASK performance history from across the state.All students can show growth.

14

Proficiency

Partially Proficient

Gr. 3 Gr. 4 Gr. 5

100

200

250

150 160 165

NJ ASK Scale Score by Grade

NJ A

SK S

cale

Proficient

Advanced Proficient

230205

220

Albert Maria

15

SGP considers growth, not proficiency

Albert’s Prior

Scores

Academic Peers’ Prior

Scores

Albert has taken the 5th grade NJ ASK. How does his score compare to those of

his academic peers?

3rd Gr. 1504th Gr. 1605th Gr. 165

3rd Gr. ≈150

4th Gr. ≈160

5th Gr. ???

16

Determining an SGPNJ

ASK

Sco

re

Partially Proficient100

200

150160 165

Proficient

Advanced Proficient

250

Gr.3 Gr.5 Gr.4

Partially Proficient100

200

150160

200Proficient

Advanced Proficient

110

250

Gr.3 Gr.5 Gr.4

Albert’s 5th Grade NJ ASK ScoreAlbert’s Academic Peers’ NJ ASK Scores

29%

70%

17

Determining an SGP

1%

99%

70%

Albert’s Score

Academic Peers’ Scores

5th Gr. 165 5th

Gr. 110 - 200

SGP

18

How are student SGPs related to a teacher’s mSGP rating?

18

Albert’s teacher receives an

effectiveness rating by taking the median SGP score - in this

case, 51.

Albert’s

SGP along with the

SGPs of all his

teacher’s students

are arranged

from low to high.

Median SGP

Score

Student SGP ScoreHugh 12Eve 16Clarence 22Clayton 24Earnestine 25Helen 31Clinton 35Tim 39Jennifer 44Jaquelyn 46Lance 51Roxie 53Laura 57Julio 61Selena 65Ashlee 66Albert 70Mathew 72Marcus 85Charles 89Milton 97

19

SGP conversion from 1-99 to 1- 4mSGP Score

Evaluation Rating

1 – 20 121 1.122 1.223 1.324 1.425 1.526 1.627 1.728 1.829 1.930 231 2.132 2.233 2.334 2.4

mSGP Score

Evaluation Rating

65 3.566 3.567 3.568 3.669 3.670 3.671 3.772 3.773 3.774 3.875 3.876 3.877 3.978 3.979 3.9

80 - 99 4

mSGP Score

Evaluation Rating

35 2.536 2.537 2.638 2.639 2.740 2.741 2.842 2.843 2.944 2.945 346 347 348 349 3

mSGP Score

Evaluation Rating

50 351 352 353 354 355 356 3.157 3.158 3.259 3.260 3.361 3.362 3.463 3.464 3.4

Based on her mSGP score, Albert’s teacher receives an mSGP evaluation rating of 3.

This is combined with other evaluation components in a summative rating.

20

How do SGOs capture growth?A Student Growth Objective is a long-term academic goal that teachers set for groups of students and must be:

• Specific and measurable • Aligned to New Jersey’s curriculum standards • Based on available prior student learning data • A measure of student learning between two points in time

21

How do we effectively set a long-term academic goal for students?

1. Establish what students know and can do currently based on a range of information.

2. Use an assessment that fairly and thoroughly measures what students should be able to know and do after instruction.

3. Determine what an ambitious but achievable learning goal is as measured by the assessment based on information about prior learning.

22

1. Assessments

Traditional Assessments

Portfolio Assessments

Performance Assessment

• District, school and departmental tests e.g., modified final exams, benchmark exams

• State and national exams (except the NJ ASK), e.g. NOCTI, AP

• Writing and reflection sample

• Laboratory research notebook

• Portfolio of work• Project-based

assessment• Teaching Strategies

Gold®

• Lab Practicum• Sight reading in music• Dramatic performance • Skills demonstration• Persuasive speaking• DRA™2

23

2. Prior student learning

Performance Data to Determine Students’ Starting Points

Grades/performance in current year

Prior-year test results that assess knowledge and skills that are pre-requisites to the current subject/grade

Test results in other relevant subjects from prior years

Students’ grades in previous classes

Beginning-of-course diagnostic tests or performance tasks

24

3. Ambitious and achievable SGO targets

Growth• 75% of students will increase their performance by

an average of 15% on 4/5 measures of fitness over the course of 4 months.

• 85% of students will decrease the score between their starting points and 100 by at least 50% during the SGO period.

Achievement• 70% of students will score 80% on the social studies

final assessment• 90% of students will score 3/4 on at least 8

components of the art portfolio rubric.

25

Top Challenges

1. Collecting quality baseline data that can be used to effectively set realistic targets

2. Setting targets that are fair for a wide variety of students

3. Developing, administering, and scoring an assessment

4. Administrator – teacher collaboration

26

1. Baseline data taken from multiple high quality sources

Student ID

Prior Test Scores Current Year Test Scores Markers of Future Success

Preparedness GroupNJ ASK 8

MathUnit 1 Unit 2

Average Score

Participates in Class

Completes Retakes

Completes Homework

Total Points

1 230 100 97 98.5 Yes Yes No 2 High2 202 90 95 92.5 Yes Yes Yes 3 High3 211 95 95 95 Yes Yes Yes 3 High4 241 85 86 85.5 Yes No No 1 High5 263 90 92 91 Yes No Yes 2 High6 284 90 85 87.5 Yes No Yes 2 High7 199 91 88 89.5 Yes Yes Yes 3 High8 201 57 75 66 No Yes No 1 Low9 144 50 58 54 No No No 0 Low

10 182 58 58 58 No No No 0 Low11 143 62 83 72.5 Yes Yes No 2 Medium12 171 78 83 80.5 No Yes No 1 Medium

NJ ASK Math Score

Current Year Test Score Average

Number of Future Success Markers

Preparedness Group

Target Score on Summative

<200 <70 0 – 1 Low 70200 – 249 70 – 85 1 – 2 Medium 80200 – 300 85 – 100 2 – 3 High 90

27

2. Fair targets for a wide variety of students

• Use a growth measure, e.g. improvement on assessment such as DRA or MAP

80% students will show one year’s reading growth, or be on grade level, as measured by the DRA2

• Use tiered SGOsSet different achievement targets for students starting at different places

28

Tiered SGO using differentiated targets

Scoring Plan

Preparedness Group

Target Score on Final

Assessment

Objective Attainment Level Based on Percent and Number of Students Achieving Target Score

Exceptional Attainment

(4)

Full Attainment

(3)

Partial Attainment

(2)

Insufficient Attainment

(1)

Low 70>85%

students (31-36)

≥70% students (25-

30)

≥55% students (18-

24)

<55% students

(0-17)

Medium 80>85%

students (19-21)

≥70% students (15-

18)

≥55% students (11-

14)

<55% students

(0-10)

High 90 >85% students (8)

≥70 % students

(6-7)

≥55% students

(4-5)

<55% students

(0-3)

29

3. High quality assessments

• Common assessments developed or chosen by teachers

• Modify existing assessments as necessary• Develop a protocol for administering and scoring

all assessments to ensure consistency• If used at all, use pre-assessment to establish the

level of preparedness and pre-requisite knowledge

• Don’t test more, increase the quality of what is already there

30

4. Collaboration

• Teacher-created• Tailored• Collaborative• Process-based• Flexible

According to the law:• A teacher develops SGOs in consultation with his

or her supervisor.

31

32

In what ways do SGOs capture what effective teachers regularly do?

Teach a curriculum that is aligned to standards.

Determine the needs of students using several methods including a variety of assessments.

Differentiate instruction based on the needs of students.

Set goals for students appropriate to their grade, subject, and readiness level.

Use high quality assessments to measure student performance.

Formalize and document the process, and be recognized for doing these things well.

33

Getting better with time

• Learning from those who have gone before us• Learning from New Jersey’s educators• Adjusting the system as needed