WyCAS/NCEO 1

WyCAS-AltImplementation Guide

A Workshop for Pilot Implementation Sites

Jim Ysseldyke, Martha Thurlow, & Sandy Thompson

National Center on Educational Outcomes

University of Minnesota

WyCAS/NCEO 2

These slides are available for your use on NCEO’s website

http://www.coled.umn.edu/nceo

WyCAS/NCEO 3

Part 1

The Big Picture

WyCAS/NCEO 4

The Push for Accountability

• Goals 2000

• Improving America’s Schools Act - Title I

• 1997 Amendments to Individuals with Disabilities Education Act

• Wyoming’s Enrolled Act 2

WyCAS/NCEO 5

All of the Laws Require:

• Participation of ALL students in state and district assessments

• Information about the performance of students with disabilities relative to other students

WyCAS/NCEO 6

New IDEA Provisions

• Participation in State Assessments

• Availability of Alternate Assessments

• Reporting of Results on the performance of students with disabilities

WyCAS/NCEO 7

Sec 612(a)(17). Participation in State Assessments

• Children with disabilities will be included in general state and district wide assessments with appropriate accommodations.– SEA or LEA shall develop guidelines for

participation; and develop, and, no later than July 1, 2000, conduct alternate assessments

WyCAS/NCEO 8

Reporting

• SEA must report, with the same frequency it does for nondisabled students– # of children participating in regular

assessments and in alternate assessments– performance on regular assessments, beginning

not later than July 1, 1998 (NOW) – performance on alternate assessments

beginning not later than July 1, 2000

WyCAS/NCEO 9

Fundamental Assumptions Underlying Accountability Systems

• An accountability system is needed for all students (regardless of the type of assessment used), and should apply to all students regardless of their characteristics.

• When schools are required to account for students with disabilities and report on the results of their performance, schools are more likely to focus on improving results for students with disabilities.

• When students with disabilities participate in a state accountability system, it enhances the possibility that educators will assume responsibility for them, and it raises expectations for their performance.

WyCAS/NCEO 10

Accountability v assessment v testing

• Accountability is a systematic method to assure those inside and outside the educational system that schools are moving in desired directions

• Assessment is a process of collecting data for the purpose of making decisions about students

• Testing is a process of administering a test to an individual or group for the purpose of obtaining a score. Testing is one way to gather assessment information.

WyCAS/NCEO 11

Why an Inclusive Assessment System is Necessary

• For an accurate picture of education

• For all students to benefit from reforms

• To make accurate comparisons

• To avoid unintended consequences of exclusion

• To meet legal requirements

• To promote high expectations and access to high standards

WyCAS/NCEO 12

What We Know About Assessment

• 48 states have statewide assessments

• 2 states do not have statewide assessments (Nebraska and Iowa)

WyCAS/NCEO 13

What We Know About Standards

• 49 states have state standards. Iowa has local standards.

• About 50% of states specify that their standards are for “all students”

WyCAS/NCEO 14

Participation Issues Across the U.S.

• Large numbers of students with disabilities are excluded from assessments

• Students in unique settings (separate schools, institutions, jails, homebound) are excluded

• Exemption rules are applied differently by different schools or for different assessments

WyCAS/NCEO 15

The Wyoming Comprehensive Assessment System (WyCAS)

• Developed as part of school reform legislation in 1998

• Standards based on the work of local communities, reflecting what they want students to learn

• Designed to complement classroom and district assessments

WyCAS/NCEO 16

The Wyoming Comprehensive Assessment System (WyCAS)

• Administered in grades 4, 8, and 11

• Measure of state standards in language arts and mathematics

• Benchmarks are used in making judgments about the performance and progress of students

WyCAS/NCEO 17

Six principles supported by WyCAS

• WyCAS must account for the performance and progress of ALL Wyoming students.

• The performance and progress of ALL students must be considered when education policy decisions are made.

• ALL students must be challenged to higher levels of achievement.• ALL students must have the opportunity to learn the content

necessary to meet the Wyoming state standards.• Wyoming standards are expanded for a small group of students with

significant disabilities. • Standards, assessments, curriculum and instruction must be aligned

for ALL students

WyCAS/NCEO 18

WyCAS is designed to include each Wyoming student through:

• Participation in the general assessment without accommodations

• Participation in the general assessment with accommodations

• Participation in the alternate assessment

WyCAS/NCEO 19

Participation decisions

• Made by each student’s (IEP) team

• Based on a student’s current level of functioning and learning characteristics

• NOT based on category of disability

WyCAS/NCEO 20

Alternate Assessment Defined

A substitute way of gathering information on the performance and

progress of students who cannot participate in typical state or district

assessments.

WyCAS/NCEO 21

Alternate Assessments Offer

• Individual administration

• Progress toward standards measured through “real world” performance indicators

• Focus on authentic skills and on assessing experiences in community and other real life environments

• Both quantitative and qualitative information

WyCAS/NCEO 22

NCEO Alternate Assessment Cyber Survey

• NCEO’s website: http://www.coled.umn.edu/nceo• What you can do at our Website

• Review your own state’s responses to the alternate assessment survey

• Review other states’ approaches to alternate assessment• E-mail the respondent of a particular state through the use of

hyperlinks• Hyperlink to other NCEO reports

WyCAS/NCEO 23

WyCAS-Alt

• Alternate assessment within Wyoming’s Comprehensive Assessment System

• Measures progress toward expanded standards through “Real World Performance Indicators”

• Includes less than 2% of Wyoming’s students

• Consists of a body of evidence rather than a single test

WyCAS/NCEO 24

Criteria to Guide WyCAS-Alt Participation Decisions

• Start with the premise that ALL students will participate in WyCAS or WyCAS-Alt

• Participation decisions are made by each student’s IEP team

• Consider a student’s IEP goals • Don’t let category of disability influence participation

decisions• Inform parents of participation options and include

them in decisions• Record decision, with rationale, on IEP

WyCAS/NCEO 25

Questions to Guide WyCAS-Alt Participation Decisions

• Is the student working toward Wyoming’s regular standards and benchmarks?

• If YES, the student should participate in WyCAS, with accommodations as needed.

• If NO, is the student working toward Wyoming’s expanded standards through Real World Performance Indicators?

• If YES, the student should participate in WyCAS-Alt.

WyCAS/NCEO 26

Myths and Truths about Alternate Assessment Systems

• IDEA ‘97 says that states must have alternate assessments in place by July 1, 2000.

• Alternate assessment is clearly defined in IDEA ‘97 statutes and regulations.

• Alternate assessments are intended for students who aren’t very good at taking tests.

WyCAS/NCEO 27

Myths and Truths about Alternate Assessment Systems

• Offering alternate assessments is one way to ensure that all students count in state accountability systems.

• Most states already have alternate assessments for students with significant disabilities.

• There is one alternate assessment that will work with all students with significant disabilities.

WyCAS/NCEO 28

Myths and Truths about Alternate Assessment Systems

• WyCAS is an acronym for “Wyoming Cares about All Students”

• Wyoming’s Alternate Assessment System is aligned with Wyoming’s Expanded Standards

• Real World Performance Indicators show progress toward Expanded Standards

WyCAS/NCEO 29

Part 2

Wyoming’s Educational Standards and

Expanded Standards

WyCAS/NCEO 30

Wyoming’s Educational Standards

• Content Standards - define what students are expected to know and be able to do

• Benchmark Standards - specify what students are expected to know and be able to do at the end of grades 4, 8, & 11

• Performance Standard Descriptors - describe how well students must perform the benchmark standards

WyCAS/NCEO 31

Wyoming’s Expanded Standards

• Expanded Content Standards - essential skills in Language Arts and Mathematics for students not able to meet regular standards

• Real World Performance Indicators - Measurable skills that show progress toward expanded standards

WyCAS/NCEO 32

Wyoming’s Expanded Standards

• Expanded Performance Standard Descriptors - Judgment of student performance in relation to expanded standards. Levels include:– Skilled– Partially skilled– Beginner

WyCAS/NCEO 33

Differences Between Regular and Expanded Standards

• Regular Reading Standard: Students read a variety of grade level materials, applying strategies appropriate to various situations.

• Expanded Reading Standard: Students read a variety of materials, applying strategies appropriate to various situations.

WyCAS/NCEO 34

Differences Between Regular and Expanded Standards

• 4th Grade Benchmark: Students use decoding skills to read fluently.

• Performance Standards:– Advanced

– Proficient

– Partially Proficient

• Real World Perf. Ind: Students recognize functional words in a variety of contexts.

• Expanded Perf. Standards:– Skilled

– Partially Skilled

– Beginner

WyCAS/NCEO 35

Activity: Use Handout,Work in Small Groups

Match Real World Performance Indicators to Expanded Standards

WyCAS/NCEO 36

WyCAS-Alt Requirements

• Expanded Standards selected by IEP team based on student goals and needs

• Select 4 Expanded Standards to work toward– Language Arts

• 1 expressive standard (speaking or writing)

• 1 receptive standard (listening or reading)

– Mathematics• 2 expanded standards

WyCAS/NCEO 37

WyCAS-Alt Requirements

• Assess at least 1 Real World Performance Indicator for each Expanded Standard

• Use at least 2 assessment strategies– Observation– Recollection– Record Review– Test (performance event)

• Collect evidence over time

WyCAS/NCEO 38

Real World Performance Indicators and IEP Goals

• A student’s current goals should guide an IEP team’s selection of Real World Performance Indicators to assess as evidence of progress toward meeting expanded standards.

WyCAS/NCEO 39

Real World Performance Indicators and IEP Goals

• Choose a student who might participate in an alternate assessment

• Find an example of a Real World Performance Indicator that the student is currently working on (may be an IEP goal)

• Do these indicators include all students?

WyCAS/NCEO 40

Part 3

Assessment Strategies

WyCAS/NCEO 41

Assessment Strategies

• Observation

• Recollection (interview, survey, rating scale)

• Record review (IEP, other records)

• Testing (performance event, portfolio)

WyCAS/NCEO 42

Observation Defined

• Watch and record student behavior• Systematic (staged, set up) or nonsystematic

(naturally occurring behavior)• Behavior could be video or audiotaped• Conducted at home, school, work, or in

community settings• Observed by teacher, parent, employer, or

others

WyCAS/NCEO 43

Effective Observation

• Clearly define:– behavior to be observed– procedures for gathering information

• Observe student:– in natural settings– in a variety settings – several times – using more than one observer for reliability

WyCAS/NCEO 44

Recollection Defined

• Recall behavior and events and document via interview or rating scale

• Interviews may be conducted face-to-face, by phone, or in small group

• Rating scales gather data in structured and sequenced way and facilitate data aggregation

WyCAS/NCEO 45

Effective Recollection

• Select people to provide recollections who know a student well

• Interviews should include: introduction, focus, information gathering, and summary

• Construct rating scale based on specific information needed

• Generally use 5 points on rating scale

• Verify information with additional sources

WyCAS/NCEO 46

Record Review Defined

• Document existing information through review of records

• Records can include: – student records (i.e., IEP)– databases– student products– anecdotal records– non-school records

WyCAS/NCEO 47

Effective Record Review

• Develop standardized record extraction forms and procedures

• Maintain confidentiality

• Make sure information is current

• Use clear, objective, and measurable information

• Use multiple records when possible for reliability

WyCAS/NCEO 48

Tests Defined

• Process of measuring competency by presenting a challenge or problem and having student generate a response.

• Scores are assigned to indicate student standing

• Portfolio system - collection of information about student skills

WyCAS/NCEO 49

Effective Tests

• Select specific areas to test and design test to fit behavior tested

• Develop accessible formats for presentation and response

• Design systematic scoring procedures and criteria to interpret student performance

• Test using same materials a student uses in natural environments

WyCAS/NCEO 50

Practice Using Assessment Strategies

• Review examples individually

• As a small group, select an Expanded Standard and Real World Performance Indicator

• Demonstrate use of one assessment strategy for large group

WyCAS/NCEO 51

Alternative Views on Alternate Assessment

Take a Stand and Defend Your Position!

WyCAS/NCEO 52

Expanded Standards:

• 1. Help include all students in Wyoming’s Accountability system

• 2. Aren’t worth the bother

WyCAS/NCEO 53

Real World Performance Indicators:

• 1. Connect the functional skills students are working on to standards

• 2. Are too much of a stretch to relate to standards

WyCAS/NCEO 54

WyCAS-Alt:

• 1. Is part of WyCAS

• 2. Is really a different assessment system from WyCAS

WyCAS/NCEO 55

WyCAS-Alt:

• 1. Includes all students who can’t take WyCAS

• 2. Still leaves some students out

WyCAS/NCEO 56

WyCAS-Alt:

• 1. Could raise expectations for students with significant disabilities

• 2. Won’t change anything

WyCAS/NCEO 57

Every student with severe cognitive impairments:

• 1. Should be included in WyCAS-Alt

• 2. Should have a team that makes individual decisions about what assessment they participate in

• 3. Should be exempt from any testing

WyCAS/NCEO 58

IEPs:

• 1. Should be referenced to standards (some goals align with standards)

• 2. Should be based on standards (all goals align with standards)

WyCAS/NCEO 59

I have:

• 1. Enough information to start talking about recording and reporting alternate assessment results

• 2. Specific questions that need to be answered before we can move on

• 3. A headache

WyCAS/NCEO 60

Part 4

Reporting Performance on WyCAS-Alt

WyCAS/NCEO 61

Why Report Results of WyCAS-Alt

• Reporting all results ensures inclusion of all students in accountability system

• Failure to report scores of students with disabilities sends message that they are not important and do not count

• “What is measured is treasured”

WyCAS/NCEO 62

All Wyoming Students Count

• All students will be reported in Wyoming’s Accountability System

• Participation = 1 group – WyCAS + WyCAS-Alt = participation

– Reports include rates of non-participation and reasons

• Performance = 2 groups– WyCAS and WyCAS-Alt

WyCAS/NCEO 63

WyCAS-Alt Requirements

• IEP team review body of evidence and determine performance level on each expanded standard

• District submit record of WyCAS-Alt results to Wyoming Dept. of Ed. by date of WyCAS administration

WyCAS/NCEO 64

Performance Levels for Math Expanded Standards

• Skilled = consistent command of knowledge and skills in the mathematics expanded standard

• Partially Skilled = partial knowledge and skills in the mathematics expanded standard

• Beginner = emerging knowledge and skills in the mathematics expanded standard

WyCAS/NCEO 65

Part 5: Tough Issues and Commonly Asked Questions

• Why isn’t there a single alternate “test” for everyone who can’t take WyCAS? – Alternate assessment needs to be flexible

enough to include EVERY STUDENT

• Why isn’t the IEP considered to be a student’s alternate assessment? – The IEP is not designed to show

performance across a group of students

WyCAS/NCEO 66

Part 5: Tough Issues and Commonly Asked Questions

• How can we avoid making WyCAS-Alt a “dumping ground” for students who aren’t expected to do well on WyCAS?– Allow only students working on expanded

standards to participate in WyCAS-Alt

• Can students who take WyCAS-Alt still graduate from high school?– Passing WyCAS/WyCAS-Alt is not required

for a high school diploma

WyCAS/NCEO 67

Part 5: Tough Issues and Commonly Asked Questions

• Do students have opportunities to learn what is tested?– WyCAS-Alt is designed to assess progress

toward expanded standards

• How can confidentiality be maintained?– To protect the privacy of individual students,

there will not be reports issued on schools with less than 10 participants in WyCAS-Alt.

WyCAS/NCEO 68

Part 5: Tough Issues and Commonly Asked Questions

• How technically adequate is WyCAS-Alt? – We don’t know yet.

• What other tough issues and questions need to be addressed?– 1– 2– 3

WyCAS/NCEO 69

Standard for piloting WyCAS-Alt

• In small groups, write a standard you expect to achieve in the pilot administration of WyCAS-Alt.

WyCAS/NCEO 70

Indicators showing that the WyCAS-Alt pilot standard has been met

• In small groups, list measurable indicators that the pilot of WyCAS-Alt has successfully met your standard.

WyCAS/NCEO 71

Logistics of WyCAS-Alt Pilot

• List Questions that need to be answered in order to effectively implement the pilot

• For example:– When do we start?– How do we keep track of how it’s going?– What do we do with the info. collected?

• Develop responses to questions

• Responses will be sent to all participants