NETWORK 536 PRINCIPAL’S INSTITUTE MOSL OVERVIEW

AUGUST 28, 2013

Agenda

1. MOSL/ADVANCE Deadlines and Overview2. Baseline assessments and Growth Measures3. MOSL Quiz4. NYC Performance Assessments -Overview 5. NYC Performance Assessments – Tools for

Norming with Staff.

CFN 536 – Gerard Beirne, Network LeaderCenter for Education Innovation – Public Education Association

Facilitators: Antonio Arocho and Niobe Hayes

August September October November

MOSL Menu Interactive Tool

released.

Advance Web Application 1.0

released.

Sample NYC Performance Assessments

available.

Continued MOSL training

webinars and resources.

Committees recommend

Local Measure. Principal

approves (or chooses

default). (9/9)

Principals select State Measure

(where the option exists).

Teachers administer and score baseline assessments.

Schools submit baseline

assessment data (optional–for

inclusion in DOE targets).(10/31) (deadline is 10/4 for inclusion in DOE targets)

DOE targets for individual

student goal-setting released

(goal-setting only).(11/15)

Goals finalized (goal-setting

only).

FALL TIMELINE

See: ADVANCE TOOLS and TIMELINES

SCORING ASSESSMENTSMOSL Scoring Requirements

In accordance with state teacher evaluation law, teachers may not score their own end-of-year assessments.

Schools may opt to either: Have another teacher in the school score both

baseline and end-of-year assessments for consistency.

Have the classroom teacher score the baseline and another teacher in the school score the end-of-year assessment.

3

WHO IS ELIGIBLE?

Note that teachers who teach less than 40% are

rated S or U—but still subject to Danielson rubric

Look at page 5 of your FAQ

BASELINE ASSESSMENTSWhere choices exist, schools may opt to use a DOE-recommended baseline assessment or a school-selected baseline.

• Remember: Baselines only need to be selected for a few state

assessments and AP exams.* All other 3rd party and Performance Assessments come with baseline assessments.

Where baselines must be selected, the PRINCIPAL chooses.

If the selected baseline is not on the DOE list, schools must use goal-setting. DOE targets provided to those schools will not include results from the baseline.

5

* Principals also have the option of selecting an alternative baseline assessment for Discovery Math in K only.

WHEN DOES A PRINCIPAL NEED TO CHOOSE A BASELINE?

Assessment Baseline

All NYC Performance Assessments

NYC Performance Assessment (fall administration)

Performance Series Performance Series (fall administration)

State Assessments: grades 4-8 ELA and Math

ELA/Math Assessment (prior year)

Discovery Math 1st and 2nd Grade

Discovery Math (fall administration)

The following assessments have required baselines:

Look at page 9 of your FAQ

WHEN DOES A PRINCIPAL NEED TO CHOOSE A BASELINE?

Assessment Baseline ChoicesDiscovery Math: Kindergarten (in K-2 schools only) 1. Discovery math (fall administration)

2. School-selected baseline

Advanced Placement (AP) Exams

1. PSAT(state assessment 8th grade if not available)2. School-selected baseline (should be selected by

schools if they want to use goal setting, since PSAT fall administration results will not be available until November 15 goal-setting deadline)

State Assessments: 3rd Grade ELA and Math 1. NYC 3rd Grade ELA/Math Performance Assessment (fall administration)

2. School-selected baseline

State Assessments: 4th Grade Science 1. NYC 4th Grade Science performance Assessment (fall administration)

2. State assessments: 3rd Grade Math (prior year)3. School-selected baseline

State Assessments: 8th Grade science1. NYC 8th Grade Science Performance Assessment

(fall administration)2. State Assessments: 7th grade Math (prior year)3. School-selected baseline

State Regent Exams 1. PSAT (State Assessment:8th Grade if PSAT not available. State Assessment: 7th Grade if Regents administered in 8th Grade.)

2. School-selected baseline

NYSESLAT 1. NYSESLAT (prior year) (Lab-R if prior year NYSESLAT not available)

2. School-selected baseline

NYSAA 1. NYSAA prior year and SANDI if prior year NYSAA not available.

2. School-selected baseline

REVIEW: GROWTH MEASUREMENT OPTIONS

8

Growth Models: DOE calculates student targets, results, and teacher scores. Results are shared after assessments have been administered so student growth can be compared to similar students’ performance on assessments.

Goal-Setting: DOE provides targets for how students will perform on assessments that principals and teachers can adjust based on their knowledge of students. Principals approve targets.

After choosing assessments and target populations, committees must select one of two growth measurements for each assessment.

CONTENT REVIEW: GROWTH MODELS

9

Growth models allow us to compare the progress that students make in a year to similar students.

• In the State Growth Model, similar students are defined by four student-level characteristics at the student and classroom levels:

Academic history Economic disadvantage Disability status Language learner status

• Citywide models account for similar characteristics to the State model.

Growth models control for the degree to which students are expected to make gains given their achievement history and demographic characteristics.

9

CONTENT REVIEW: GROWTH MODELS

• Does not introduce additional work in schools.

• Enables schools to compare their students’ performance to similar students.

• Gives teachers credit for the degree to which students exceed predicted growth.

• Better able to account for unexpected outcomes resulting from unfamiliar, new assessments.

• Growth model score results are not available until after assessments have been administered (i.e., the following spring/summer).

Ben

efi

tsC

halle

ng

es

10

GOAL-SETTING

1. Administer baseline assessment

2. DOE sends predicted student targets

3. Teachers review DOE- predicted targets

4. Principals approve or adjust targets

5. Administer end of year assessment

6. Teachers’ Ratings

Baseline assessment administered (not required for all assessments).

Report baseline assessment results.

DOE sends predictions for how individual students will perform.

Predictions are based on baseline performance, student achievement history, and student demographic characteristics.Teachers may choose to adjust these targets based on

additional information about their students.

Teachers submit student targets to principal.

Principal (or designee) reports finalized student targets.

End-of-year assessment administered to students.

Teachers’ HEDI ratings are calculated with a conversion chart based on students’ performance on outcome assessments relative to their targets.

11

GOAL-SETTING• Particularly valuable for

teachers/schools with unique student populations or high mobility.

• Allows teachers and principals to individually tailor student goals.

• Requires additional time/resources.

• Teacher’s rating is based on the percentage of students who meet their target, but does not consider the degree to which students fall below or exceed their target.

• Targets must be set early in the school year, possibly before much diagnostic info is gathered

• Setting goals may be challenging if:

• Teachers are not familiar with the comparability between assessments

• Assessments are new or changing

Ben

efi

tsC

halle

ng

es

GOAL-SETTING CONSIDERATIONS

• This is not the same as the goal-setting you may typically see in schools.

• Goals are scored against a state conversion chart which makes the target-setting process difficult and non-intuitive.

• Before considering goal-setting, make sure you understand the work and additional training this entails.

13

GOAL-SETTING: FALL IMPLEMENTATION STEPS

By 9/9

• School Local Measure Committees recommend whether or not to use goal-setting for Local Measure.

• Principals decide the same for State Measure (where the option exists).

September

• Schools who choose goal-setting norm on expectations across classrooms.• Schools administer baseline assessments.• 9/24 – DOE target reference tables and goal-setting worksheets released on

Intranet.

Octobe

r

• By 10/4 – Schools submit results of baseline assessments if they want data included in DOE-suggested targets (otherwise submit by 10/31).

• By 10/15 – Principals submit grade/school level goals to superintendent for review.

• Teachers start process of goal-setting using baseline data and reference tables.

November

• 11/1 – Schools can access goal-setting worksheets which include baseline results and DOE-suggested targets.

• 11/15 – Principals finalize teacher-set goals. Superintendents finalize school/grade goals.

14

GOAL-SETTING – WHO SETS GOALS?

15

Target Population Teachers Principals

Super.

Individual Level• By student; or• By subgroups of students;

or• Whole class goal

Set Approve*

Grade Level• By subgroups of students;

or• Whole grade goal

Set Approve

School Level • By subgroups of students;

or• Whole school goal

Set Approve

Teachers set goals for students in their classroom (target population: individual), but when goals go beyond an individual

teacher’s students (target population: grade or school), the principal sets the goal and the superintendent approves.

* If a teacher fails to submit goals, the principal will set the goals for that teacher. The principal should not simply “approve,” but rather work with the teacher on adjusting targets as necessary.

MOSL QUIZ

Please use your MOSL FAQ Document to complete the MOSL quiz (orange paper)

(First principal finished will receive a prize!!)

PERFORMANCE ASSESSMENTS – OVERVIEW

17

• NYC Performance Assessments are available in multiple K-12 subjects, some as optional, others as required. These include baseline and end-of-year assessments (no additional selections for baseline needed).

• If the principal and/or committee chooses the default option, the mandated NYC Performance Assessments in grades 4-8 ELA for the Local Measure do not need to be administered as all teachers in the school will use the default measure.

• There are Performance Assessments required in some grades/subjects for the State Measure. Those assessments are not affected by the selection of the default option for the Local Measure.

• In most grades/subjects with Performance Assessments, there is one type of assessment. However, in K-8 ELA, committees have two options within the Performance Assessment category:

• Running Records NYC Performance Assessment (using one of three reading programs) (TC, DRA, F&P)

• Writing-based NYC Performance Assessment

PERFORMANCE ASSESSMENTS: NYC PERFORMANCE ASSESSMENTS

Considerations: Review of where they exist on assessment list

ELA (writing-based): K-12 Math: Grade 3, Integrated Algebra Social Studies: 6-8, Global History, U.S. History Science: 4, 6-8, Living Environment

In K-8 ELA, also have choice of Running Records. All NYC Performance Assessments (except ELA)

translated into 9 languages.

Require norming on rubrics at the school level. Samples available for download (final versions by 9/9).

18

PERFORMANCE ASSESSMENTS: IMPLEMENTATION STEPS

By 9/9

• Final performance assessments released online• School Local Measure Committees recommend whether or not to administer Performance Assessments where optional. Principal approves or chooses default.

• For ELA K-8, principals and committees decide between Running Records and a writing-based performance assessment.

• Principals plan necessary teacher support around administration and scoring.

September

• Schools develop scoring protocols.• Schools norm teachers on Performance Assessment rubrics to ensure a shared understanding of demands of the rubric and scoring across student work.

• Teachers administer Performance Assessments.

October

• Schools score and scan Performance Assessments. Must submit scores by 10/4 for inclusion in DOE targets (otherwise by 10/31).

19

DEEP DIVE: NYC PERFORMANCE ASSESSMENTSACTIVITY #1 – TASK DESIGN INVESTIGATION

20

Objective

Participants will be able to interpret a sample NYC Performance

Assessment by analyzing the requirements for students and

predicting students’ strengths and struggles.

Activity

Preview the sample NYC Performance Assessment and consider

the four key questions on the handout. Use the chart to take

notes while you’re reviewing the task and texts. Share your

reflections with your table.

DEEP DIVE: NYC PERFORMANCE ASSESSMENTS

ACTIVITY #1 – TASK DESIGN

21

DEEP DIVE: NYC PERFORMANCE ASSESSMENTSACTIVITY #2 – NORMING & SCORING

22

Activity

Individually read and score Sample 1 according to the rubric.

For this experience, you are only asked to grade the rubric

dimension of “Focus: Position”, and then discuss your

findings with your table. Refer to evidence in the sample of

student work and the rubric to justify your choices and seek

consensus. Repeat the process for Sample 2 (time

permitting).

Objective

Participants will be able to apply the rubric as an evaluative

tool for scoring student writing and establish a shared

understanding of the expectations of the NYC Performance

Assessments.

DEEP DIVE: NYC PERFORMANCE ASSESSMENTS

ACTIVITY #2 – NORMING & SCORING

23

  12th Grade ELA Sample A 12th Grade ELA Sample B

Rubric Dimensions

My Score

Normed Score

Evidence from the TextMy Score

Normed Score

Evidence from the Text

Focus: Position

           

Development: Elaboration

           

Development: Textual Analysis

           

Development: Counterclaims

           

Reading            

Organization            

Conventions            

DEEP DIVE: NYC PERFORMANCE ASSESSMENTSACTIVITY #3 – PLANNING FOR INSTRUCTION

24

Activity

Choose one dimension/trait (in this case, we will choose

“Focus: Position”) from the rubric to analyze in-depth. List

the rubric dimension and key characteristics of the rubric, then

use this as a lens to reread and analyze the sample. With this

new lens, consider the student’s strengths and struggles as well

as strategies that might support the student’s growth.

Objective

Participants will be able to analyze a sample of student work

within a specific element of the rubric to consider the implications

of assessment practices on instruction and the development of

student skills.

DEEP DIVE: NYC PERFORMANCE ASSESSMENTSACTIVITY #3 – PLANNING FOR INSTRUCTION

25

DEEP DIVE: NYC PERFORMANCE ASSESSMENTS

ACTIVITY #4 – IMPLICATIONS

26

Activity

As you review additional sample tasks, consider the implications

of the NYC Performance Assessments on various stakeholders. As

a result of today’s activities, what are the next steps? What

questions, comments, or considerations are you contemplating?

Objective

Participants will reflect on the cycle of assessment and

identify the implications of this assessment option for a

variety of stakeholders. They will then plan next steps.

CONSIDER THE IMPLICATIONS OF THE NYC PERFORMANCE ASSESSMENTS ON VARIOUS STAKE-HOLDERS. AS A RESULT OF

TODAY’S ACTIVITIES, WHAT ARE THE NEXT STEPS? WHAT QUESTIONS, COMMENTS OR CONSIDERATIONS ARE YOU

CONTEMPLATING?

Reflection Notes

••

Administration

••

Data Specialists

••

Teachers

PLEASE INCLUDE YOUR NAME ON ANY INDEX CARD QUESTIONS SO THAT WE MAY RESPOND

VIA EMAIL IF WE DO NOT RESPOND TODAY