Special Educator Evaluation Matt Holloway Educator Effectiveness Specialist

Special Educator EvaluationMatt HollowayEducator Effectiveness Specialist

Massachusetts Department of Elementary and Secondary Education

2

Guiding Questions How can the Educator Evaluation

Framework support improved services and instruction for students with special education eligibilities?

How can the Educator Evaluation Framework support the professional growth of special education personnel?

How can the Educator Evaluation Framework support inclusive educational models for schools and districts?


3

Common Concerns Are all evaluators knowledgeable about

research-based best practices for students with disabilities?

Are the unique job responsibilities of special educators accurately captured by the ESE Model Rubrics?

Will students with special education eligibilities have lower growth scores that disproportionately affect Student Impact Ratings?


4

Structure for the Day

Summative Performance Rating

Resources Student Impact Rating Resources


5

Framework Overview

All Massachusetts educators receive two independent but linked ratings that focus on the critical intersection of practice and impact, while creating a more complete picture of educator performance.

Performance Rating

Exemplary 1-yr

Self-Directed Growth

Plan

2-yr Self-Directed

Growth PlanProficient

Needs Improvement

Directed Growth Plan

Unsatisfactory

Improvement Plan

Low Moderate High

Impact Rating


6



7

The Summative Performance Rating


Ratings for:Standard IStandard IIStandard IIIStandard IV

Rubrics

Assessment of:Educator Goal Attainment

Student Learning & Professional Practice

Goals


8

Rubrics

The appropriate rubric for Special Education Teachers is the Teacher Rubric.

However, districts have expressed the need for customization options.

Ratings for:STANDARD ISTANDARD IISTANDARD IIISTANDARD IV

Rubrics


9

Customizing the Rubric Prioritizing indicators/elements Developing role-specific resources

Addition of a “role-specific” indicator “Look-for” documents

Creating a “hybrid” rubric


10

Prioritizing Indicators/Elements

Consider the extent to which the use of existing rubrics can be customized through a differentiated emphasis on and prioritization of Indicators and Elements.

This is commonly done district-wide, but can also be a tool for educators in differentiated roles.

For example, districts have shared that the following are areas of priority for special educators. II-A-3: Meeting Diverse Needs II-B-2: Collaborative Learning Environment II-C-1: Respects Differences II-D: Expectations Indicator


11

Role-Specific Resources

In order to address the need for more specificity, some professional organizations have developed role-specific resources that could be added to the Model rubrics.

Special Education Directors have reported preparing similar resources for other district evaluators.

http://www.doe.mass.edu/edeval/model/PartIII-AppxE.pdff


12

Hybrid Rubrics Some educators may be in a role that

combines the responsibilities of both a classroom teacher and an administrator, such as a department head. Others may be in roles that combine the responsibilities of classroom teachers and SISP educators.

In this scenario, the parties could create a hybrid rubric including Standards, Indicators and Elements or descriptors from multiple rubrics appropriate to the responsibilities of the educator.


13

Rubrics & Observations

The rubrics are written to support educators and evaluators in making judgments about patterns of evidence, gathered across multiple points in time.

The rubric has not been designed to be a classroom observation tool and should not be used for that purpose.


14

Educator Goals

Assessment of:EDUCATOR

GOAL ATTAINMENT

Student Learning & Professional

Practice


15

Educator Goals Student learning goal - Focus on the needs

of the students – what’s the greatest area of need for support? How can you meet the unique needs of your students?

ESE does not suggest aligning these goals directly with IEP goals.

Professional practice goal – Focus on your professional growth opportunity – what do you, as an educator, want to get better at or learn more about?


16

Educator Goals (cont.) Some districts have described the use of

shared goals as a way to encourage collaboration between general education and special education teachers in inclusion settings.

This approach should be balanced with the provision of specially designed instruction to meet the unique needs of students with disabilities.


17

Three Categories of Evidence

Products of Practice

Multiple Measures of Student Learning

Other Evidence (e.g. feedback)

THREE CATEGORIES

OF EVIDENCE


18

Three Categories of Evidence

All three categories of evidence should yield information about the educator’s practice related to the four Standards and/or the educator’s goals.


19

Evidence Collection FAQ

There is no minimum or maximum requirement associated with the provision of evidence. There needs to be enough evidence associated with each Standard such that a rating on that Standard can be supported. The body of evidence should be aligned to the individual educator's goals, the focus of the evaluation, and school/district priorities.


20

Products of Practice

(1) artifacts related to educator practice, and

(2) observations of practice.


21

Artifacts

Educators and evaluators should think strategically about evidence collection, keeping in mind that one piece of evidence often reflects practice associated with multiple Standards and Indicators.

While the majority of special education teachers in MA are evaluated by school level administrators, the artifacts of their practice will often be very specific to the special education process.

This is a chance for educators and evaluators to connect and share in professional learning. In some districts, school administrators collaborate with special education administrators to calibrate their evaluation practices.


22

Artifacts (cont.)

Educators and evaluators may want to agree on artifacts of practice in advance.

Special educators who have significant case management responsibilities may want to use sample student folders as artifacts of practice.

Special education artifacts can be an opportunity to start conversations about specially designed instruction that meets the unique needs of students with disabilities.


23

Observations Evaluators need multiple opportunities

and settings to observe and assess educator practice

Multiple observations paired with timely feedback are a key part of a strong evaluation system

Special educators perform vital work in a variety of settings.


24

Multiple Measures ofStudent Learning

In contrast with the measures that inform the Student Impact Rating, which must be comparable across grade or subject level district-wide, these measures may include classroom specific assessments that provide evidence of practice related to the Standards or goal attainment.

Data pertaining to student-specific interventions would be appropriate.


25

Student/Staff Feedback

The third category of evidence must include feedback from students or staff. Other sources of information, such as evidence of fulfillment of professional responsibilities and evidence of family engagement, may also be considered.

Collecting feedback from students with special needs is a valuable part of the evaluation process. Districts should make every effort to include all students, or a representative sample of all students, in their feedback collection.


26

Student/Staff Feedback (cont.)

Districts have indicated a need for supports around collecting feedback from students with significant cognitive disabilities.

ESE has gathered stakeholder input on the process and is developing resources.

Guidance will be published in May to support implementation in 2015-2016.


27



28

Professional Judgment Assessment of goal attainment &

analysis of evidence against rubrics yields a Summative Performance Rating

Key Factors: Transparent process between evaluators and

educators (“no surprises) Collaboration with special education

administrators where appropriate


29

Resources www.doe.mass.edu/edeval/specialed Quick Reference Guides:

Summative Performance Rating Educator Evaluation Framework

Rubrics & Role-Specific Resources Evidence Collection Toolkit Student and Staff Feedback

http://www.doe.mass.edu/edeval/specialed

http://www.doe.mass.edu/edeval/resources/QRG-SummativeRating.pdf

http://www.doe.mass.edu/edeval/resources/QRG-Framework.pdf

http://www.doe.mass.edu/edeval/resources/rubrics/

http://www.doe.mass.edu/edeval/resources/implementation/CollectionToolkit.pdf

http://www.doe.mass.edu/edeval/feedback/


30

The Student Impact Rating


31

Regulations


32

Multiple Measures Multiple Years

Patterns refer to results from at least two different measures of student learning, growth and achievement. Either Student Growth Percentiles (SGP) or

District-Determined Measures (DDMs) Trends refer to results from at least two

years.


33

Student Growth Percentile

While SWDs often demonstrate high growth compared to other groups, SWDs have a lower average SGP. As a result, educators working primarily with SWDs may be more likely to meet the definition of low growth set by ESE than to attain moderate or high growth. This is one reason why professional judgment is such a critical part of the process of determining an educator’s Student Impact Rating.


34

Professional Judgment

Evaluators are encouraged to bear in mind an educator’s student population, specific instructional context, and considerations related to a specific measure or combination of measures when determining Student Impact Ratings.

Considerations for Special EducationDDM Implementation Brief

35


36

District Determined Measures

Measures of student learning, growth, and achievement

Aligned to content Yield meaningful information


37

Accessibility & Universal Design

As a part of the Continuous Improvement process, districts will want to review for and identify barriers in assessments that interfere with students’ ability to demonstrate their knowledge.

Does the assessment item require skills beyond the content being assessed?

Eg. a Math item that requires reading comprehension skills beyond the intent of the assessment or a Science item that requires culturally specific background knowledge

http://www.doe.mass.edu/edeval/ddm/Improvement.pdf


38

Key Messages for Special Educators

Use or modify existing measures District-wide message about professional

judgment Differentiate between growth and

achievement All students should have an equal

chance to demonstrate growth Make clear connections between

baseline scores and parameters for growth (“bands”)

http://www.doe.mass.edu/edeval/ddm/UsingAssessments.pdf


39

Banding

Q: Do all students have to meet the same score on an assessment to demonstrate moderate growth?

A: No – student growth levels should be aligned to baseline data.


40

Banding (cont.) Some districts have set parameters for

low/moderate/high growth differently according to baseline scores.

Three “bands” could be set, corresponding to baseline data, with different parameters for growth attached to each.

Bands do a lot to address the problem of measuring differential growth.


41

Banding (cont.)

Initial Fitness Portfolio Rubric Score

(start-of-course)Low Growth Moderate

Growth High Growth

19-32 points 19-32 points 33-70 points 71-100 points




42

MCAS-Alt as a DDM MCAS-Alt does not generate SGP scores

and is not a required aspect of the Educator Evaluation regulations

Significant percentage of districts have indicated using or planning to use MCAS-Alt as a DDM

ESE has gathered stakeholder input on the process and is developing resources

Guidance will be published in May to support implementation in 2015-2016


43

MCAS-Alt as a DDM (cont.)

Key Considerations:

Use “measurable outcomes” as growth measures

Calibrate measurable outcomes with teams of educators

Set a range for moderate growth


44

IEP Goals & DDMs

Districts may find that IEP goals provide a meaningful starting point for DDMs, but it is important to make a distinction between the assessment purpose of each.


45

Challenges to using IEP Goals in DDMs

Variations in IEP timelines IEP goals are designed to be dichotomous, that is either

met or not met. DDMs require three levels (high, moderate, and low).

IEP goals are developed with the expectation that they will be met. In contrast, determinations of high, moderate, and low growth are designed to describe a range of growth without a predetermined expectation.

Educators may not be responsible for a certain IEP goal. (e.g., a reading support teacher is not responsible for a goal supported by the occupational therapist).

IEP goals are not designed to be comparable across students.

IEP goals must meet specific legal requirements, such as parental involvement, that may not be consistent with the process of determining goals of a DDM.


46

DDMs for Inclusion – reported District A:

All teachers using general education DDMs to encourage co-teaching and collaborative approach to inclusion.

District B: Special Educators using DDMs tied to access

and independence measures based on specially designed instruction for individual students.

Indirect Measures & SISP EducatorsDDM Implementation Brief

47


48

Direct vs. Indirect Measures

SISP Educators Example DDMs for SISP Educators

Counselors School Psychologists School Nurses SLPs OTs/PTs


50

DDMs for Special Education Administrators

Direct Progress of specific student populations on common

assessments Closing achievement gap for specific student populations

Indirect Removal/suspension/expulsion rates Parent satisfaction surveys Measures to address disproportionate identification and

placement Special education timelines Coordinated Program Reviews Measures of placement in the least restrictive environment


51

Resources www.doe.mass.edu/edeval/specialed Quick Reference Guides:

Student Impact Rating Summative Performance Rating

DDM Implementation Briefs: Considerations for Special Education Administrators Indirect Measures and SISP Educators Scoring and Parameter Setting Continuous Improvement

Using Current Assessments in DDMs

http://www.doe.mass.edu/edeval/specialed

http://www.doe.mass.edu/edeval/resources/QRG-ImpactRating.pdf

http://www.doe.mass.edu/edeval/resources/QRG-SummativeRating.pdf

http://www.doe.mass.edu/edeval/ddm/SpecialEduEducators.pdf

http://www.doe.mass.edu/edeval/ddm/Admin.pdf

http://www.doe.mass.edu/edeval/ddm/IMSISP.pdf

http://www.doe.mass.edu/edeval/ddm/Scoring-ParameterSet.pdf

http://www.doe.mass.edu/edeval/ddm/Improvement.pdf

http://www.doe.mass.edu/edeval/ddm/UsingAssessments.pdf


52

Guiding Questions How can the Educator Evaluation

Framework support improved services for students with special education eligibilities?

How can the Educator Evaluation Framework support the professional growth of special education personnel?

How can the Educator Evaluation Framework support inclusive educational models for schools and districts?

Educator Evaluation Contacts

Matt Holloway, Education [email protected] (781) 338-6622

Craig Waterman, Assessment [email protected](781) 338-3244

Claire Abbott, Educator Effectiveness [email protected](781) 338-3253

53

mailto:[email protected]



Documents

Special Educator Evaluation Matt Holloway Educator Effectiveness Specialist