Students in the Gap: Understanding Who They Are & How to Validly Assess Them

Students in the Gap: Understanding Who They Are &

How to Validly Assess Them

Findings from Two Projects

Montana Modified Achievement Standards Pilot

1 Year Technical Assistance on State Data Collection Grant

New England Compact Study of Students in the Gap

NH, ME, RI, VT

2 Year Enhancing State Assessment Grant

Montana Modified Achievement Standards Overview

Developed and administered a pilot alternate assessment

Based assessment on modified achievement standards

Evaluated feasibility of incorporating it into the statewide assessment system

New England Compact (NEC): Students in the Gap

Looking at Gaps in Large-Scale Assessment Systems:

Two Percent… or Two Gaps?

NEC Enhanced Assessment Grant

Funded by US Department of Education

Four states: NH, ME, RI, VT

2005-2007

Challenge: describe students in gap, and design an assessment that will meet needs of students in gap

8th grade mathematics– 8th grade to look at complexity– Mathematics to avoid reading

comprehension issues

Project history

Original goals− Identify students in the gaps− Develop varied assessment modules− Pilot/validate assessment modules

Issues we faced– Not easy to identify the gap, or the

students– Impossible to develop an

assessment without knowing target students’ needs

Revised goals

• Identify students in the gaps through multiple methods, triangulating evidence

• Define common criteria for identifying students in the gap

• Plan and develop task module assessment strategies (assessment prototypes)

• Recommend core components of an assessment structure that would lessen the gaps

• Disseminate products to others considering assessments for students in the gap

Accountability context

• Project began February 2005• Modified achievement standards

announced April 2005• Proposed “2%” regulations released

December 2005• Studies were designed before 2%, not

in response to 2%• Findings speak to needs of all students

not effectively assessed in current system, not necessarily dovetailing with 2% definitions

Big questions• Who are the students in the gaps?

• Of all the students who are not proficient, how can states identify those who are in the assessment gaps?

• What are the attributes of students in the gaps, and how do these students perform?

• What issues in the assessments themselves contribute to the gaps?

• Are there specific aspects of multiple-choice items used in state assessments that contribute to the assessment gaps?

Gap identification processConduct exploratory

interviews with teachers to identify the assessment

gaps

Review student assessment data

Review teacher judgment data

Operationalize gap criteria

Conduct focused teacher interviews to

confirm gap criteria

Parker and Saxon: Teacher

views of students and assessments

Bechard and Godin: Finding the real assessment

gaps

The process for investigating gap profiles

Bechard and Godin: Who

are students in gaps?

Conduct focused teacher interviews to confirm gap criteria

Investigate characteristics of students in gap 1

Investigate characteristics of students in gap 2

Investigate achievement patterns of students in gap 1

Investigate achievement patterns of students in gap 2

Develop profiles of students in gap 1

Develop profiles of students in gap 2

Alternative test itemsHypothesize alternate test

items

Decompose items into requisite skills/

knowledge

Provide alternative formats:Item format Item content Visuals Multimedia

Review with mathematics

experts

Pilot and evaluate items

Russell and Famularo: Utility of a prototype assessment

Dolan et. al.: Providing

students with choice

Session Goals

Consider:

How we define “students in the gap”

If and how we alter achievement standards for “students in the gap”

How we assess “students in the gap”

Participants• Judy Snow, MT Office of Public Instruction

– Lessons from MT Modified Achievement Standards Project

• Carrie Parker, EDC– Teachers’ Views of Gaps in Large-Scale

Assessment

• Sue Bechard, Measured Progress– Using State Assessment Data to Identify Students in

the Gap

• Bob Dolan, CAST– What Literature Tells Us about Math Instruction and

Assessment

• Mary Ann Snider, RI Department of Education– State Director of Assessment

• Gaye Fedorchak, NH Department of Education– Moderator, State Supervisor of Alternate Assessments

?

New England CompactStudents in the Gap:

Final Questions

• How can we discriminate between the instruction gap and the assessment gap?

• Should students be taught the skills that help to close the context gap between classroom and assessment?

• Are different assessment approaches needed for either gap group?

Final Questions (cont.)

• Can a test developed for the 2% option stretch low enough to reach students in gap 2?

• How should expectations be adjusted? The items? The accommodations or choices provided? The achievement standards?

• If you were in charge of a state assessment program, what else would you need to know before deciding how to proceed?

• What would the students say?

2% Flexibility• Students in Gap 2 not tested on what they

are taught

• Students in Gap 2 are performing far below grade level

• Regulations won’t allow off-grade level tests for Gap 2

• A modified assessment to Gap 1 or non-Gap 1 would mean lower standards for students who could be at grade level

• Conclusion: The 2% flexibility option will not increase the validity or relevance of the test for students in either gap

www.necompact.org