9 20-11 va-teacher-level_reporting_dval 2

Value-Added Analysis: Teacher-Level Value-Added Reporting

Copyright ©2011. Battelle for Kids.

Teacher-Level Value-Added ReportingKeys to ImprovementImplications and Cautions About Teacher-Level Value-Added Reports

Presentation Overview


Teacher-Level Value-Added Reporting


Teachers have the greatest potential to improve student growth

Comparison of Effect Sizes



Rollout Schedule 30% of LEAs Link in Year 1 RttT (reports received

fall 2011) 60% of all LEAs in Year 2 (all RttT LEAs included) 100% of all LEAs in Ohio in Years 3 and 4

Accuracy of Reporting Must conduct linkage Minimum number of students and time enrolled


Access to Reporting Online via EVAAS® accounts Password protected Teachers will automatically receive email from SAS with

username and password Grades/Subjects Available

ODE: grades 4-8, math & reading and grades 5 & 8 science

BFK: grade 3, math and reading; grades 3, 4, 6 & 7, science and grades 3-8 social studies; high school—algebra I and II, geometry, pre-calculus, biology, chemistry, English 9, 10 and 11



What is Linkage?

Process of accurately linking students to teachers for the purpose of ensuring accuracy of value-added reports

Captures the instruction that each teacher provides over the course of a school year

Designed to ensure that teachers are accurately “linked” to the students they taught and the instruction they provided

Teachers verify the students and subjects that they taught

Teachers indicate the timeframe that they taught each student and whether any other teacher also provided instruction for a particular content area


Why is Linkage Necessary?

It provides reliable data that reflects what is actually taking place in the classroom

Without accurate linkage data, value-added reports would not accurately reflect the instruction, and school leaders and teachers may make inaccurate assumptions about their instructional practices and students’ performance

For more details on linkage: Webinars/videos FAQs Linkage training


Teacher-Level Value-Added Report:Aggregate Level Reporting

Aggregate-level effectiveness of this teacher over time

The table displays how that teacher performed compared to the state’s 3-year average and the district’s 3-year average in that same grade level and subject area (most recent year first).

In 2011, you will only see one year of data.


Teacher-Level Value-Added Report:State Distribution of Progress Levels

The distribution of teacher effectiveness across the state in this teacher’s grade level and subject area


Teacher Value-Added Diagnostic Report: Disaggregate-Level Report

This teacher’s relative effectiveness with her three prior achievement subgroups

Teacher Report Help Screen

MRM Teacher Report Language.

Versus, a URM Teacher Report (science and other extended reporting) will say:

“The Teacher Progress Table The table at the top of the teacher report contains information on the effectiveness of the teacher compared to the State Average. Multi-Yr-Averages will be provided when sufficient data exist for this calculation. The teacher's Progress Estimate, expressed in scale score units, and Standard Error are provided for each year reported. The teacher's Index and Effectiveness Level complete the table.”


School Improvement Conversations

1. How do your aggregate level results compare to the aggregate level results of your team (compare your value-added estimate and standard error to the value-added estimate and standard error in the school value-added report)?

2. Is this subject an area of strength or an area of challenge for you relative to your team?

3. If this subject is an area of strength for you, how can you provide support for other members of your team?

4. If this subject is a challenge for you, who on your team can you call on for support?


Cautions about Teacher-Level Value-Added Reporting All value-added reports are estimates

All available student performance information is used to overcome the errors of measurement at the individual student level

Statistical tools are used to protect teachers from being “misclassified,” especially those who don’t have many students

Estimates are improved with more data A 2-year average is a better measure of teacher effectiveness

than a 1-year estimate A 3-year average is better than a 2-year average Multiple measures are better than a single measure.

The real danger is in jumping to conclusions Given all of these issues, value-added estimates are still the best

measures of student progress that are currently available


Summary Reports


District Summary Report – All Schools


District Summary Report – All Schools


District Summary Report – One School


Keys to Improvement


Factors That Influence Instructional Effectiveness Instructional Practices

Was student progress measured‘in time’? Are the learning, the feedback, and the assessments focused

on the right goals? Instructional Arrangements

How were the data used to determine student assignments to courses and to create flexible groupings?

Focused Professional Development Data-driven? Embedded: collaborative time centered around student

learning? Focused on‘vital’behaviors? Sustainable?


A Few Strategies for Helping Teachers to Improve

Partner teachers with other teachers who complement their strengths.

Determine if an issue is a teacher problem or a curricular problem.

Identify students who are not making sufficient progress and design intervention plans.

Customize professional development based on student growth patterns.

Stimulate discussions during the school year about ongoing measures of student growth.

Pair teachers with students with whom they are most successful.


Improvement is All About Changing the Distribution of Educator Effectiveness


Cultural Impact ofTeacher-Level Value-Added Reporting


Cultural Implications

How do you deal with the cultural pressures that surround the use of teacher-level data?

Be aware of and responsive to the new demands and responsibilities that teacher-level value-added information brings. Consider the nature of parent demands.

Find appropriate ways to share and discuss teachers’ report(s) with them discreetly.


Cultural Implications

Be sensitive to the impact teacher reports have on a teacher’s sense of self-worth.

Build an environment where teacher-level data can be routinely shared openly among colleagues in a manner that is safe and respectful.

Help teachers respond to the information they receive by guiding them toward reflective thinking and taking productive action.


Consider and Discuss

As a district superintendent:

Concern is growing among teachers as they hear more about HB 153 and how teacher-level value-added reports will impact evaluation and/or compensation. As the leader of your district, how will you prepare your staff for the changes that are to come with this current political environment?



As a building principal:

After a teacher reviews her teacher-level value-added reports, she comes to you and tries to explain away the data, claiming that she has the students with the lowest ability and parents who do not support the school. How are you going to respond?



As a classroom teacher:

You are concerned that teacher-level value-added

reporting may have a negative impact on the

current collaborative culture that exists in your

team. What can you do to address this?



As a union representative:

A teacher calls you and says that after she and

her principal went over her teacher-level value-

added reports, the principal was ‘harassing’ her by

sharing the data and expecting an action plan.

What would you do to respond to her problem?


Scenario Activity I: Superintendent

1. Take the Scenarios for VAL Training sheet from your materials

2. Ask one person from the table to read aloud the Superintendent Perspective Primary Scenario

3. Have a five minute discussion about this scenario. (Keep in mind that you are thinking like a superintendent!) After about five minutes, then…

4. Pair off with someone from your table and discuss one of the additional examples for superintendent perspective.

5. Be ready for a brief whole-room discussion!


Scenario Activity II: Principal

1. Take the Scenarios for VAL Training sheet from your materials.

2. Ask one person from the table to read aloud the Principal Perspective Primary Scenario.

3. Have a five minute discussion about this scenario. (Keep in mind that you are thinking like a principal!) After about five minutes, then…

4. Pair off with someone from your table and discuss one of the additional examples for principal perspective.



Scenario Activity III: Teacher


2. Ask one person from the table to read aloud the Teacher Perspective Primary Scenario.

3. Have a five minute discussion about this scenario. (Keep in mind that you are thinking like a teacher!) After about five minutes, then…

4. Pair off with someone from your table and discuss one of the additional examples for teacher perspective.



Scenario Activity IV: Union and Community


2. Ask one person from the table to read aloud the Union Leader OR Community Perspective Primary Scenario.

3. Have a five minute discussion about this scenario. (Keep in mind that you are thinking like a union leader or community member!) After about five minutes, then…

4. Pair off with someone from your table and discuss one of the additional examples.



Questions?Thank you!Resource information found on the Ohio Student Progress Portal:

www.BattelleforKids.org/Ohio

NEW! Getting Ready for Teacher-Level Reports resources for administrators on Portal, including a chaptered, recorded

webinar. (access from homepage or under the RttT section)

BFK Support Desk: [email protected] or (866) 543-

7555

www.BattelleforKids.orghttp://twitter.com/BattelleforKids

Education

9 20-11 va-teacher-level_reporting_dval 2