Orientation Module-13NOV13

The Student Learning Objective (SLO) Process is comprised of three (3) Components: Design, Build, and Review. Student Learning Objectives provide a valid assessment of teacher effectiveness through student performance outcomes based on standards. This training series is comprised of four (4) training Modules to include: an Orientation Module, a Design Module, a Build Module, and a Review Module.

Welcome to the Orientation Module: ________________________________________________________________________

Technical Notes“Structure”

1. Concept – “What is this slide telling the audience?”

2. Key Points – “What/Where are the details ‘needed for teaching’?”

3. Learning Activity – “How can the participant’s learning be enhanced?” (This item will not be populated for every slide.)

SLO Orientation Module 1

Goal & Objectives

Concept

Within the Orientation Module, the trainer will provide participants with an overview of the SLO process and how it is an integral part of the Educator Effectiveness System. The three phases include: Design, Build, Review.

Key Points for Trainers

1. Explain in macro-terms the “big picture” of how SLOs are but one component of a much larger teacher effectiveness system.

a. As required by Act 82, an Educator Effectiveness Rating Tool must contain measures based on teacher observation and practice and multiple measures of student performance. Student Performance data will include Elective Data at the 20% and 35% attribution levels. Please refer to the Educator Effectiveness Administrative Manual (http://www.portal.state.pa.us/portal/server.pt/community/educator_effectiveness_project/20903).

b. This Educator Effectiveness Rating Tool functions as a framework for the evaluation and summative process for classroom teachers.

2. Ensure the 5 W’s are addressed (What are SLOs; Where are SLOs in the larger system; Who uses SLOs; When are SLOs designed, built, reviewed; Why SLOs are helpful tools).

a. What are SLOs? SLOs are locally selected and developed Student Learning Objectives.b. Where are SLOs in the larger system? SLO is the process through which Elective data is applied to a

teacher’s effectiveness rating.c. Who uses SLOs? Classroom teachers will use the SLO process.d. When are SLOs designed, built, and reviewed? SLO’s are developed prior to the beginning of the

school year.e. Why are SLOs helpful tools? SLOs are helpful tools as they are selected, designed, and reviewed by

educators themselves and provide one layer of data associated with the multiple measures of student performance data, as well as guiding instruction.

Learning Activity


http://www.portal.state.pa.us/portal/server.pt/community/educator_effectiveness_project/20903

Tool Organization

Concept

As you work through the various training modules, several resources are available. All SLO training materials can be found at both the PDE’s SAS portal and Research in Action’s Homeroom learning platform. Go to http://www.pdesas.org. Once there you can login to the Homeroom learning portal.


1. Differentiate how tools are organized into four groups: videos, guides, templates, and “other stuff”. a. Videos are PowerPoint, mp3, etc. that provide information in a visual format. b. Guides have materials such as handouts, rules of thumb, model SLOs, etc. that reinforce

content presented in the videos. They contain the details within each element of the process.

c. Templates are used to complete each phase of the process. They create a structure for creating learning objectives and performance measures. [Note: Performance measure templates are located within the Assessment Literacy Series found within the Homeroom learning portal].

i. During the Design Phase, teachers will complete Template #1 Goal Statement, Template #2 Targeted Content Standards, and Template #3 SLO Blueprint

ii. During the Build Phase, teachers will complete Template #4 SLO Process Template and Template #5 Performance Task Framework

iii. During the Review Phase, teachers will complete Template #6 SLO Coherency Rubric

d. “Other Stuff” is where additional materials, references, and information reside that may be helpful to the process.

Learning Activity

1. Have participants go to www.pdesas.org to review resources that are available on the SAS portal and the Homeroom learning portal.


SLO Design Coherency

Concept

This figure is designed to help visualize how the “parts” fit into the “whole.”


1. It is essential for the audience to fully understand how only a coherent SLO approach should be used within the greater Educator Effectiveness System.

a. Goal Statements are the “big idea” upon which the SLO is based. These goals must be based upon PA standards and provide a rationale for choosing a particular standard which addresses the important student learning.

b. Performance Indicators are the expected levels of achievement for students in the SLO population based on the performance measures.

c. Performance Measures are the various tools/assessments which will be used to measure student achievement toward a specific goal.

d. Ratings are determined by the overall results given the expectation ranges established at the beginning of the year.

Learning Activity

1. Have participants go to the SAS portal and review the Curriculum Framework. “The Curriculum Framework specifies what is to be taught for each subject in the curriculum. In Pennsylvania, Curriculum Frameworks include Big Ideas, Concepts, Competencies, and Essential Questions aligned to Standards and Assessment Anchors and, where appropriate, Eligible Content.” (SAS Portal)


Training Resources

Concept

As noted previously, a significant number of resources are available on the above mentioned sites. Go to the SAS portal at http://www.pdesas.org for PA content standards and PDE developed material. Also on the SAS portal is a link to RIA’s Homeroom portal. Both sites provide trainings and participants with any information needed for the SLO process.

Note: the icons provided on the slide are hyper-linked.


1. Built to support teachers and principals.2. Build template on-line, save, edit, print.3. View training at a time that “fits” in to user’s schedule (resources are available online).

Learning Activity

1. Pause training and have audience go to the SAS Portal and locate (a) a content-specific curriculum framework, and (b) the SLO “Homeroom” webpage.

(New participants should register on the RIA website when the Login page appears.)

2. As preparation for the next slide, participants should click on the SLO box and the ALS box to gain familiarity with the visual presentation.

Note: These web resources will assist in completing templates in the Design module.


Assessment Literacy Series vs. Student Learning Objectives

Concept

Assessment Literacy deals with various types of performance measures, ranging from the traditional pre/post assessments to long-term, culminating events such as a dance recital.

Student Learning Objectives is a process comprised of three (3) components: Design, Build, and Review. PDE’s definition of an SLO is: A process to document a measure of educator effectiveness based on student achievement of content standards.

Note: ALS is also a process with the same three components (i.e., Design, Build, Review); but the outcome is a teacher-made assessment.

Both the Assessment Literacy Series and the Student Learning Objectives (SLO) provide educators with the tools necessary to create fair, reliable measures of student learning to be used as one (1) part of educator effectiveness.


1. All educators will need to complete the SLO Process Template and the Performance Measure Task Framework Template. Some educators will use vendor-developed assessments, thus will only need to check the quality and appropriateness of the performance measure.

2. The Assessment Literacy Series provides tools in both developing and evaluating performance measures. Creating high quality performance measures for use beyond the single classroom is a time intensive task.

Learning Activity

1. Reference the Homeroom site with applicable URL within the SAS Portal and RIA website, as explored in the activity for the previous slide.


Outline of the Orientation Module

Concept

Now that the background, reasons, and resources available for educators have been provided, we will move into the various components that make up the SLO process.


1. This “learning map” provides an outline of both the SLO content (components) and process, including some ALS (Assessment Literacy Series) techniques (steps) to begin the work, including the presentation, refinement, and review of created SLOs.

2. SLO Process Componentsa. DESIGN: This component is the “thinking” step in the process used to conceptualize

the learning objective in terms of content, students, and performance measures. Within the Design Module, the trainer will begin to guide participants through the structure that is necessary to build a Student Learning Objective.

b. BUILD: This component is the “action” step in the process that focuses on completing the SLO Process Template [Template #4] and creating and/or selecting performance measures and performance indicators.

c. REVIEW: This component is the “reflection” step used to examine the three “Cs” (i.e., Completeness, Comprehensiveness, and Coherency) of quality.

i. Completeness: Refers to the correct data entered into the SLO Process Template.

ii. Comprehensiveness: Refers to the performance measures meeting the technical quality necessary to measure student achievement.

iii. Coherency: Refers to the alignment of the performance measures to the SLO goal and objectives (targets).

Learning Activity


Student Learning Objective

Concept

It is important to keep in mind that the Student Learning Objective (SLO) is a process that allows educators to have a great deal of control and responsibility over their instructional practice and evaluations. In this process, teachers take responsibility for selecting the goals, setting performance indicators/targets, and selecting performance measures which will be used to determine their effectiveness with the students they are instructing in a specific grade level, course, or subject area. These SLOs are one part of the multiple measures that will be used to evaluate teachers and comprises 20% (for teachers with eligible PVAAS data) or 35% (for teachers without eligible PVAAS data) of a teacher’s final rating. Further clarification can be found on the PDE website, Educator Effectiveness Administrative Manual (http://www.portal.state.pa.us/portal/server.pt/community/educator_effectiveness_project/20903).


1. Articulate that the student learning objectives are being used in emerging teacher effectiveness systems throughout the nation. They have become increasingly important as teacher effectiveness systems have begun linking student outcomes directly to specific educators, rather than in broad terms associated with school accountability systems.

2. Clarify that SLOs based on non-cognitive measures and used for school effectiveness systems (graduation rates, matriculation rates, AP course enrollment, suspensions, etc.) will be excluded from this SLO process and its usage within educator effectiveness systems.

• Select non-cognitive measures are part of the Building Level Data.

Learning Activity

1. Take three minutes to think:



• Write down two ways SLOs could positively influence your teaching.2. Take two minutes to share:

• Share “Round Robin” in your groups.


Terminology

Concept

Having a clear understanding of the vocabulary (i.e., educational jargon) within the SLO process is helpful throughout the training modules.


1. Assessment Literacy addresses numerous types of performance measures ranging from traditional, on-demand assessments to long-term, culminating events (e.g., dance recital). Teachers should be encouraged to reflect on the measures they currently use and then identify those most appropriate for the SLO process.

2. Goal statements are based upon what areas within the curricula are essential for students to master, while having leverage in other subject areas.

3. Goals must be based on PA Content Standards, professional standards, and/or big ideas associated with PA standards. PA standards and/or professional organization standards can be located at http://pdesas.org/standard/views.

Learning Activity

1. Have participants review the Curriculum Framework on the SAS portal to help in selecting the “big idea.”


http://pdesas.org/standard/views

Additional Details (continued)

Concept

After teachers identify a specific standard and develop a goal statement they must provide a “rationale statement” which explains why it is important to measure a specific standard and how this impacts student learning.

Once the goal statement and rationale are complete, teachers can begin completing a “blueprint.” A blueprint shows the various aspects of the SLO. It includes the goal statement, the targeted standards, performance indicators (expected levels of achievement based on the performance measures), and performance measures (the tool used to measure student learning).


1. A blueprint is a tool that teachers use to aide in the development of a SLO. It lays out how the various components of the SLO are related and aligned. Blueprints are the foundation of creating the SLO. (See Handout #3) for an example of a SLO Blueprint.

Learning Activity

1. Have participants find and review Template #1 Goal Statement, Template #2 Targeted Content Standards, and Template #3 SLO Blueprint. These materials can be found in SLO/Design/Template.


Additional Details (continued)

Concept

As part of the SLO process and the Educator Effectiveness Rating Tool, teachers will be required to set four (4) levels that describe an expected number of students who will successfully meet the Performance Indicator targets as stated in the SLO. These rating categories include: Distinguished, Proficient, Needs Improvement, and Failing. Educators determine the percentages within each rating category prior to the beginning of instruction. These performance ratings will be examined at the end of the evaluation period and be used to determine an Elective Rating, which will be applied to the overall final teacher evaluation rating.


1. For information regarding the application of the “Elective Data” rating into the complete teacher evaluation rating, please refer to the Educator Effectiveness Administrative Manual (http://www.portal.state.pa.us/portal/server.pt/community/educator_effectiveness_project/20903).

Learning Activity

1. At this point, take questions from the audience and get any underlying issues addressed or noted in the “Parking Lot”. After which, take a short break before going into the next six (6) slides.



SLO Process Components

Concept

Time to take a few minutes to provide information about the SLO Process Components.


1. This approach will allow the audience to cognitively process the material.2. During the introduction, lay out what will be covered and how the SLO materials are

organized (i.e., DESIGN, BUILD, and REVIEW).a. DESIGN: This component is the “thinking” step in the process used to

conceptualize the learning objective in terms of content, students, and performance measures.

b. BUILD: This component is the “action” step in the process that focuses on completing the SLO Process template and creating and/or selecting performance measures.

c. REVIEW: This component is the “reflection” step used to examine the three “Cs” (i.e., Completeness, Comprehensiveness, and Coherency) of quality.

Learning Activity


Guiding Principles

Concept

Keep in mind that there are several guiding principles associated with the SLO process. These principles are key to the SLO process and ensure the development of high-quality, valid, and fair measures of teacher effectiveness through student performance outcomes based on standards.


1. The guiding principles are essential in developing a mental picture of the characteristics of the student learning objective.

a. Principle #1-Performance must be aligned to the standards with a specific course/content. Other types of performance, such as attendance, classroom behavior, time-on-task, cooperativeness, engagement, etc. are not part of the SLO process.

b. Principle #2-Goal statements articulate a controlling “big idea”; however, more granular content standards are necessary in order to develop/identify performance measures.

c. Principle #3-Performance measure quality is critical in producing results associated with the selected standards. Often, assessments are used for purposes the author did not design the tool to perform, thus creating a validity threat to those inferences about student learning. Furthermore, ensure that equitable opportunity for the student to demonstrate learning is provided within the administration and scoring of the performance measure.

d. Principle #4-Metrics have strengths and weaknesses. i. Status [i.e., Mastery] metrics have absolute standards and are easily understood;

however, they do not reflect changes (improvement) in student performanceii. Growth metrics are sensitive to changes in learning; however, they are more

unstable and limited for high performing studentse. Principle #5-Performance indicators provide a clear expectation of performance on the

identified assessment. Multiple indicators can often provide a more defensible evidence of student learning. Linking and weighting indicators can be helpful in some instances but add additional layers of complexity to the SLO process.

Learning Activity


SLO Process Components

Concept

The Student Learning Objective (SLO) Process is comprised of three (3) Components: Design, Build, Review. Within the Design(ing) Phase, the trainer will begin to guide the participant in the development of a “Goal Statement”, identifying underlying content standards, and creating a blueprint. The Build(ing)Phase focuses on the completion of the Template #4-SLO Process Template which includes identifying and/or creating performance measures. The Review(ing) Phase allows educators to go back and refine, edit, and finalize the SLO.


1. Explain that all components are done before the school year (initial conversation with principal) in preparing the SLO; however, the REVIEW component may also continue until the final results are available to determine whether or not the performance expectations have been reached.

2. Clarify that specific timelines for the SLO process will be determined by local education agencies (LEAs) and not by the state; however, a generic timeline for the SLO process outlining before, during, and after school year activities should be presented. In general:

a. Teacher develops SLO, along with applicable performance measures before school starts.b. Principal reviews and discusses with teacher; adjustments may be required.c. Teacher reviews SLO progress at a midpoint in the year.d. Principal receives mid-year update from teacher for review.e. Teacher summarizes performance measure data and evaluates each performance indicator.f. Teacher presents final SLO results to the principal.g. Principal assigns final rating in Section 5 of the SLO Process Template.

Learning Activity


SLO Process Components: DESIGN

Concept

The SLO Design Process is the “planning” process where teachers work collaboratively to identify targeted standards and types of performance measures that reflect the “big ideas” within standards which would effectively measure student learning and teacher effectiveness.

Teachers will work through activities that allow time for thinking, brainstorming, organizing, discussing, and researching content standards, possible performance measures, and performance indicators to be used in the creation of SLOs.

The first step in the Design Process is the development of a “goal statement” which reflects the “big ideas” related to the specific standards, along with a rationale statement that explains the alignment to standards and addresses the important learning for the class/course/content area. Training Module 1: Designing will provide further details related to this phase of development. [Note: Teachers will be using Templates #1, #2, and #3 during the DesignPhase.]


1. Designing is planning for the SLO, and examining what is needed as well as how performance measures are used to collect information about student achievement.

2. Activities during this stage establish the foundation for developing a student learning objective, including:a. Identifying target content standardsb. Discussing the “Big Idea” in the standardsc. Thinking about the goald. Collaborating with other teacherse. Brainstorming the type of performance measures to be used

Learning Activity

1. Using the SAS portal, have participants review the Curriculum Framework which outlines Pennsylvania’s Big Idea, Concepts, Competencies, and Essential Questions. Have each participant select a specific content area and grade level and share a few of the “big ideas” identified.

2. Using the Homeroom learning portal, have participants review the following materials: (a) Handout #1 – Goal Statement, (b) Handout #2 – Targeted Content Standards, and (c) Handout #3 – Blueprinting.


SLO Process Components: BUILD

Concept

Within the Build Phase, teachers will be completing the SLO Process Template which identifies the Classroom Context, SLO Goal, Performance Indicators (PI), and Performance Measures (PM).

When selecting or developing Performance Measures, the Assessment Literacy Series (ALS) will guide educators to ensure that the measures used meet the principles of well-developed measures.

Well-developed measures must:a. Be built to achieve the designed purpose;b. Produce results that are used for the intended purpose;c. Align to targeted content standards;d. Contain a balance between depth and breadth of targeted content;e. Be standardized, rigorous, and fair;f. Be sensitive to testing time and objectivity; and, g. Have score validity and reliability evidence.

Training Module 2: Building will provide further details related to this phase of development. [Note: Teachers will be using Templates #4 and #5 during the Build Phase.]


1. Ensure the participants understand that Building is an iterative process between the original design and the creation of the SLO. Often, the original design must be changed after it is decided how standards will be measured and performance indicatortargets have been developed.

2. Activities during this stage [complete the SLO Process Template] include:a. Selecting (or creating) the performance measures that are aligned to the targeted content standards;b. Developing mastery and/or growth metrics associated with the performance measures;c. Establishing performance indicator targets;d. Identifying students included in the SLO data; and,e. Creating performance expectations.

Learning Activity

1. Using the SAS portal and the Homeroom learning portal, have participants review the following: (a) Template #4 SLO Process Template, (b) Template #5 Performance Task Framework, and (c) Model #1 (Art) and Model #2 (Physical Education).

2. Have participants identify and explain each section of Template #4 SLO Process Template. (A list of “Help Desk” statements designed to support this activity can be found in SLO/Build/Other Stuff.)


SLO Components: REVIEW

Concept

The SLO Review Process provides an opportunity for educators to complete a 3-tier Quality Assurance Review to ensure the SLO, along with its applicable performance measures, are complete, comprehensive, and coherent. Teachers will be using checklists and rubrics to determine the Completeness of the SLO Template, the comprehensiveness of the performance measures, and the coherency (alignment) of the SLO. Training Module 3: Reviewing will provide further details related to this phase of the process. [Note: Teachers will use Handouts #4 and #5, and Template #6 during the Review Phase.]


1. Ensure the participants understand that Reviewing requires an extensive evaluation of the SLO’s quality in terms of the 3C’s.

a. Completeness, Comprehensiveness, and Coherencyb. Quality assurance checklist and rubric

2. Process activities during this stage occur before and after the presentation to the principal, and include:a. Finalizing and submitting the proposed SLO;b. Refining the SLO based upon feedback from the principal;c. Collecting performance data on student achievement;d. Monitoring the SLO during the school year;e. Updating the SLO with data;f. Evaluating each performance indicator; and,g. Determining the Elective Rating.

Learning Activity

1. Allow time for participants to access and review the following documents from the Homeroom learning portal:a. Handout #4 – Quality Assurance Checklist-SLOb. Handout #5 – SLO Procedural “Cheat Sheet”c. Template #6 – SLO Coherency Rubric d. Refinement Control Checkliste. Performance Measure Rubric for Teachers


Summary

Concept

As noted previously, the SLO process was created to meet the need of Act 82, which requires PA to have an Educator Effectiveness Rating Tool that contains measures based on teacher observation and practice and multiple measures of student performance. Pennsylvania Department of Education defines Student Learning Objectives as a process to document a measure of educator effectiveness based on student achievement of content standards.

PDE contracted with Research in Action (RIA) who developed the SLO process. This process has three (3) components which include: Design, Build, Review. Module 1: Design, Module 2: Build, and Module 3: Review. Each training module gives educators sufficient details, tools, and models necessary to develop an individual, grade-level, or school-based student learning objective.

All training materials and resources are available at both PDE’s SAS portal and Research in Action’s Homeroom learning platform. Go to http://www.pdesas.org for access to content standards and the Homeroom link.


1. The Summary slide must be used to revisit the major areas associated with the SLO Process.

Learning Activity

1. Address “Parking Lot” questions or identify sources of the information (e.g., PDE, RIA, etc.).2. Randomly probe audience to solicit “tell me one thing you learned, remembered, stuck in

your mind, etc.” as a way to gauge the audience’s understanding.
