Measuring Teacher Impact on Student Learning

Measuring Teacher Impact on Student Learning

PEAC Discussion Document| August 20, 2010

2 © The New Teacher Project, 2010

To ensure the most accurate and complete understanding of each teacher’s performance and development needs, an evaluation system should draw from a wide range of evidence. Potential components of an evaluation system can include:

Component Description

Student Learning Outcomes

Student learning measured in multiple ways:1. Student growth as indicated by an applied growth model, where available; and2. Student mastery of rigorous academic goals and standards, based upon a variety of summative assessments.

Instructional Practice

The extent to which a teacher executes a set of core competencies, through observations of teacher and student actions and document reviews.

Professional Responsibilitie

s

The extent to which a teacher exhibits non-skill and knowledge based actions and attitudes that reflect a clearly defined set of professional responsibilities.

Contribution to School

Community

The extent to which a teacher’s actions build a positive school culture for students and families.


Given the current availability of data in the state, Illinois must rely on an evaluator’s assessment of student learning until new standardized measures are built. There are two main alternatives to consider.

Option A – Goal Attainment Processo Teachers measure student growth by setting student academic

goals, aligned to meaningful standards.o Evaluators confer with teachers to establish each goal’s degree of

ambition and select the appropriate assessments for measuring progress against the goals.

o Teacher evaluation is based on students’ progress on the established goals, as determined by an end-of-the-year evaluator review of the pre-determined assessments and their results.

Option B – Review of Student Progresso Student learning is measured through the evaluator’s assessment of

the extent to which a teacher’s students have mastered or made growth toward specific standards, as indicated by performance on multiple assessments.

o Teachers track student performance over time, and evaluators review ongoing student work and make judgments using a rubric.


In Option A, teachers work with evaluators to set ambitious goals surrounding selected student assessments.

5th Grade

Sample Goal 1: Increase class average of % mastery across all prioritized math standards from X% to 85% Sample Goal 2: 80% of students in Subgroup A will show growth on prioritized science and writing standards through achievement of at or above “meets expectations” on summative “Animal Encyclopedia” project (>3 out of 4 on rubric)

9th Grade Art

Sample Goal 1: 90% of final portfolio presentations attain the level of “shows significant improvement” or “shows outstanding improvement” on rubric at year-end review. Sample Goal 2: 80% of “Artist Spotlight” booklet submissions show growth AND achieve a rating of at least “Good” (>40/ 50) on rubric and are published in our end-of-year famous artist profiles magazine.

11th Grade English

Sample Goal 1: 90% of students achieve improved score on 21st Century Skills Rubric AND score at or above “above expectations” for final research paper (>85/ 100) Sample Goal 2: Median score of at least 80% on end of year summative examination

At the beginning of the year, teachers, in collaboration with their evaluator, decide the appropriate number of goals to set for their particular classes, as well as to ensure that the goals set are “acceptable” (aligned to standards, challenging but attainable, and measureable). Some sample goals can include:

Some sample goals can include:


At the end of the year, teachers are evaluated based on goal ambition and achievement using the rubric below to assign a student learning rating for each goal.

Teacher has at least one high-quality source of evidence (i.e., one that is rigorous and

reliable) demonstrating that student progress:

- Exceeded a very ambitious goal.



- Met a very ambitious goal; OR

- Exceeded an adequately ambitious goal.



- Nearly met a very ambitious goal; OR

- Met an adequately ambitious goal.


reliable) demonstrating that students:

- Nearly met an adequately ambitious goal; OR

- Met less ambitious goal; OR

- Exceeded a less ambitious goal.

1

Evidence of student progress toward goal is missing, incomplete, and/ or unreliable; AND/OR Students did not demonstrate significant academic progress (regardless of

ambitiousness of goal); AND/OR Student progress nearly met a less ambitious goal.

2

Table 3 - Goal Attainment Rubric

5

4

3


o Evaluator assesses the quality of the assessment instrument(s) used as well as the student learning outcomes to assign a rating.

o Teacher tracks ongoing progress

o Evaluator monitors student progress during classroom observations and through ongoing reviews of student work.

o Evaluator flags any tracking and assessment issues, and helps the teacher overcome these issues

In Option B, evaluators review student progress on an ongoing basis using multiple sources of information and assess progress relative to standards.

o Teacher and evaluator choose a set of priority standards against which student outcomes will be measured

o Teacher articulates a plan including assessments for tracking student progress toward mastery of the chosen standards

o Evaluator reviews the assessment instrument(s) using a rubric and offers feedback until the assessment(s) meets expectations

Beginning of Year During Year End of Year


Criteria

Strongly disagree

=1Disagree

=2Agree

=3

Strongly Agree

=4

1) Were planned outcomes ambitious and feasible?

The amount and type of content and skills the teacher prioritized and taught during the year were:•Aligned to approved grade/course content and end-of-year assessments.•Differentiated by students’ starting points so that students starting near grade level could grow at least one year and students starting below grade level could grow more than one year (e.g., included remedial objectives)•Allowed for a scope and sequence was appropriately paced and included time for re-teaching•Applied to appropriate and increasing levels of text difficulty (if applicable)•Discrete, clear, and testable•Aligned to or more ambitious than IEP goals and ELL goals

2) Were the assessments good?

The assessments the teacher chose, designed, and used to summatively and conclusively determine mastery were:•Aligned to standards, the right level of rigor, and any external end-of-year assessments•Assessed mastery of aligned content and skills accurately and reliably

3) Is the data credible?

• The teacher proctored and scored summative assessments such that results accurately reflect students’ knowledge and skills

• Each student’s trend over the course of the year made sense and was realistic• There is consistency between the teacher’s data and separately verifiable evidence of

student mastery (e.g., benchmark assessments)

The evaluator must agree with statements 1 through 3 in order to agree or strongly agree with criteria 4.4) Have the students learned an ambitious amount of material?

The vast majority of the teacher’s students who started near grade level learned at least a year’s worth of material*

The vast majority of the teacher’s students who started far behind, including mild/moderate SPED students, learned more than a year’s worth of material*

The teacher’s moderate/severe students learned an ambitious amount of material including their functional, life, and vocational skills*

* Note: Missing data is counted as no mastery.

At the end of the year, evaluators use a rubric to evaluate teachers and assign a rating.


Professional learning community meetings and feedback sessions

Targeted development activities (i.e., coaching, co-teaching, etc.)

Note: Additional evaluation and development activities for non-tenured teachers, developing teachers, and/or teachers in need of improvement (such as mentoring from coaches, progress check-ins with instructional managers, additional conferences, etc) are proposed in greater frequency but do not appear in the timeline above.

Self-assessment and develop professional focus areas

Beginning of year

conference to develop evaluation

plan

End of year summative evaluation conference

Self-assessment and discuss next year’s

professional focus areas

Mid-year check-in

conference, informed by all available

data

Self-assessment and revisit

professional focus areas

Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun

Instructional Rounds (drop-ins) and/or full-period classroom observations

Situational feedback conversations; student data reviews and data team meetings

Regardless of the option chosen, the process must align with the overall evaluation system and should promote discussions between the teacher and evaluator on student performance and teacher professional growth.


PEAC can consider the following discussion questions when selecting a process for measuring student learning outcomes.

Discussion Questions:

•What are the strengths of each of the options? What concerns or challenges does each of these options raise?

•What are the implications for school-level capacity between the two options? For the kinds of support that districts will need to implement the process with fidelity?

•How should this align to the overall process and outcomes of the evaluation system?

•How might these two options differ in practice for an elementary vs. secondary level?

•Is there another way to go?


Appendix


Categories of Student Learning Measures

Measure of Student Learning Rating Tool Used Current Status in Illinois

Category 1: Student growth on state standardized tests that are developed and/or scored by ISBE (e.g., ISAT)

Value-Add or Growth Measures

• Illinois currently assigns teacher identification numbers and unique student identifiers, but does not have a data system capable of matching individual teacher records to individual student records needed for growth measures (NCTQ 2009 Report Card)

Category 2: Student performance (as measured by growth) on standardized district-wide tests that are developed and/or scored by either the district or by an external party but not by ISBE (e.g. NWEA, AP, PLAN, EXPLORE, ACT, DIBELS, ACCESS, etc.)

Evaluator’s Assessment of Student Learning (EASL)

• Evaluators can take student performance on district assessments into account and can begin developing processes around this.

Category 3: Other, more subjective measures of student performance that would likely be developed and/or scored at the district- or school-level (e.g., student performance on school- or teacher-selected assessments)

Evaluator’s Assessment of Student Learning (EASL)

• Evaluators can take student performance on school and classroom assessments into account and can begin developing processes around this.

Documents

Measuring Teacher Impact on Student Learning