New Mexico Alternate Performance Assessment

1

New Mexico Alternate Performance Assessment

(NMAPA)

Test Administrator & RaterTraining

New Mexico Public Education DepartmentAmerican Institutes for Research

February 2008

2

NMPEDAmerican Institutes for ResearchNMAPA Team

Mona Martin, Assessment Administrator, NMPED

Lynnett Wright, Project Director, AIR

Dan Hauser, Operations, AIR

Werner Wothke, Technical Advisor, AIR

3

Administration Window

March 10 to April 25

Testing materials will arrive in the districts by February 29.

4

Old Alt – New Alt

(OLD) Original NM Alternate Assessment

(NEW) New Mexico Alternate Performance Assessment

See handout for key comparisons

5

Alternate Assessments

Alternate assessments are designed for the small number of students who are unable to participate in regular grade-level state assessments even with appropriate accommodations. (IDEA 1997 & NMAC)

6

Alternate Assessments

Must be aligned to the state’s content standards

Must yield results in both English language arts and mathematics and must be designed and implemented in a manner that supports use of results as an indicator for AYP

Can measure progress based on alternate achievement standards (NCLB)

7

Alternate Achievement Standards

Set an expectation of performance that differs in complexity from a grade-level achievement standard

Must be linked to state’s academic content standards

Must promote access to the general education curriculum

Must reflect professional judgment of the highest achievement standards possible (NCLB 34 C.F.R.§200.1(d))

8

NM Participation CriteriaNMAC 6.31.2.11

(a) The student’s past and present levels of performance in multiple settings (i.e., home, school, community) indicate that a significant cognitive disability is present;

(b) The student needs intensive, pervasive, or extensive levels of support in school, home, and community settings; and

(c) The student’s current cognitive and adaptive skills and performance levels require direct instruction to accomplish the acquisition, maintenance, and generalization of skills in multiple settings (home, school, community).

9

NMAPA: Purpose

To promote maximum access to the general education curriculum

To ensure that all students are included in the statewide assessment program

To drive instruction and appropriate pedagogical practices with high expectations

10

Test Administrator Requirements

A certified employee of the district

An employee of the district who is a special education teacher who has an interim certificate

Test Administrators may not administer the assessment to close relatives.

11

Training Requirements

Every Test Administrator (generally the student’s teacher) who will administer the NMAPA must attend one of the statewide training sessions.

12

Administration and Scoring Fidelity

A random sample of the test administrations will be double-scored.

Random sample Based on the students Determined by PED

13

Review and Refinement Process

Rigorous review and refinement by NMPED and AIR, as with any content area achievement test (e.g., fairness, psychometric)

Bias and content review panels of expert New Mexico teachers

14

Test Administrator

A random sample of NMAPA administrations must be double-scored: The Test Administrator’s record is scored for AYP; the Rater’s record is used for our inter-rater reliability study.

Test Administrator: Teacher, related service provider, diagnostician

Rater: Any of the above and paraprofessionals (EAs)

15

New Mexico Expanded Grade Band Expectations (EGBEs)

Assessment Frameworks

16

EGBE Development

Began task force work in early spring 2006, finalized on December 15, 2006: focus on academic expectations

Involved a variety of diverse stakeholders from throughout the state, including our Alternate Assessment Advisory Council

17

EGBE Development

Special, ELL, and general education teachers and administrators wrote and reviewed the EGBEs.

They are available on the Assessment and Evaluation Bureau’s homepage, along with an approval memo from Dr. Garcia and a user’s guide.

18

EGBE Assessment Frameworks

Linked to grade-level content standards

Basis for IEP goals and objectives

Basis for classroom instruction for students with significant cognitive disabilities

19

Importance of Font in the EGBEs

Bold = This EGBE is assessed on this year’s NMAPA.

Italic = This is an instructional objective that may be on future tests but is currently to be used for classroom assessment only.

20

Levels of Complexity

Complexity in terms of demands on students Content (e.g., from simple to complex standards) Cognitive (e.g., from concrete to more abstract

understanding) Communication levels

Extended Symbolic (Clusters 7 and 8) Symbolic (Clusters 5 and 6) Pre-symbolic (Clusters 3 and 4) Engagement (Clusters 1 and 2)

21

Link to All of the EGBEs

http://www.ped.state.nm.us/div/acc.assess/assess/Expanded_Grade_Band_Expectations/egbe.html

22

Lessons Learned

Teachers may use the EGBEs as a basis for IEP goals.

Teachers should teach from the EGBEs.

The NMAPA is directly drawn from the EGBEs.

23

Determining Starting and Concluding Tasks

24

Pre-assessment to determine the most appropriate starting point

Student Placement Questionnaire (SPQ) 12–15 “can do” statements addressing

student skills and knowledge based on teacher’s prior knowledge

Allows maximum opportunity for student to demonstrate his or her skills without prolonging the assessment

25

Student Placement Questionnaire

SPQ is designed to identify the most appropriate starting task for each student.

Teachers complete a series of questions regarding student knowledge of the content.

Based on responses, teachers compute a score to determine most appropriate starting task in the test form.

26

Student Placement Questionnaire (cont.)

Each content area SPQ is located in the student score form.

The SPQ contains directions for Identifying the starting task using the SPQ; Administering and concluding the NMAPA.

All students will complete at least these tasks: 1–5 or 3–9 or 6–12

27

Student Placement Questionnaire (cont.)

Student “responds successfully” to the beginning task

When the student gets at least three points on all of the items in the beginning task.

If the student is not successful, you should drop back to the beginning of the previous group of tasks.

28

Student Placement Questionnaire (cont.)

Student who “responds successfully” to the concluding task: Continue to the next task, then the next,

and so forth until the student no longer responds successfully.

Responding successfully When the student gets at least three

points on the items in the concluding task

29

Student Placement Questionnaire (cont.)

For example: Starting at Task 1: Tasks 1–5, 6, 7, …

Starting at Task 3: Tasks 3–9, 10, 11, …

Starting at Task 6: Tasks 6–12

30

Student Placement Questionnaire (cont.)

Possibilities

Starting at Task 11 2 3 4 5 6 7 8 9 10 11 12

31

Student Placement Questionnaire (cont.)

Possibilities

Starting at Task 33 4 5 6 7 8 9 10 11 12

32

Student Placement Questionnaire (cont.)

Possibilities

Starting at Task 66 7 8 9 10 11 12

33

Student Placement Questionnaire (cont.)

Possibilities Starting at Task 1

Starting at Task 3

Starting at Task 6

1 2 3 4 5 6 7 8 9 10 11 12

3 4 5 6 7 8 9 10 11 12

6 7 8 9 10 11 12

34

Move Forward or Stop Example

Task 5Successful

Task 6

2 points - STOP

3 points – GO on to Task 7

2 points - STOP

35

Lessons Learned

It is important to follow the directions for completing the SPQ accurately.

The student’s teacher or Test Administrator must complete the SPQ.

If you start too high, you must drop back.

36

Video Clip

Let’s watch as Edward is administered an the task -

Being Healthy and Safe

37

NMAPA Task and Item Information

38

NMAPA Tasks

A task is a collection of items and materials organized around a theme (e.g., a story, a writing activity, a math activity). Tasks contain 4 to 6 items.

Each item Scripted Scaffolded to reduce complexity Students can respond verbally or nonverbally Includes directions for scoring student

responses

39

NMAPA Tasks

Materials include printed and physical manipulatives, storybooks.

Almost all materials are provided in a manipulatives kit.

40

Task Information

Cover page includes Materials needed to administer each

item Some are provided by the teacher.

Special adaptive instructions Access limitations and instructions Introductory and closure statements

41

Item Information

For each item:

Materials Directions for setup

Placement of manipulatives Response cards Displaying a text

42

Item Scripting

Scaffolded scripting

Opening statement in a say/do format

Tell me or show me

43

Item Scripting

Opening statement or question “Here is a _______________.”

“We are going to talk about __________”

Followed by: The student showing or telling which

one is the correct response option or which one is correct

44

Graphic Setup

Tells where each of the materials is to be placed.

Script is always on the left.

Scoring scaffolding directions are always on the right.

45

Scaffolding

If the student does not answer correctly or fails to respond, there are specific instructions for the Test Administrator.

These instructions are in boxes within each test item.

46

Scaffolding (cont.)

For example:• If a student indicates _________,

record a 3. If not, continue with the scaffolded script below.

47

Item Format

Example of an Item

48

Video Clips

Let’s watch as Devin is administered the task -

Money Identification

And

Tim is administered

Ride of My Life

49

Adapting and Accommodating to Meet Student’s Needs

50

Allowable Accommodations

Mayer-Johnson pic-syms have been used throughout the tasks and items.

If your student uses a different symbol for the same word, you may substitute that symbol for the one provided. For example:

51

Adapting and Accommodating Teacher can say the

response option aloud. Teacher can point to all

of the response options or concrete objects.

Have student touch or hold the object.

Place the objects in a specific location or orientation when the student has a limited visual field.

Change the background.

52

Wording

You may substitute a more familiar word or term for scripted language as long as doing so does not change what is being measured.

“Tennis shoes” for “sneakers” is fine.

53

Assistive Technology

54

Assistive Technology

Low-Tech Not electrical or

mechanical Specifically designed

devices for a specific purpose:

Special spoons Corner chairs Large pencil grips Photographs Raised lines

55

Assistive Technology (cont.)

Medium-Tech Less sophisticated high-

tech devices May have electronic or

mechanical components

Single-message communication devices

Computer programs

56

Assistive Technology (cont.)

High-Tech• Computer• Speech to text • Text to speech• IntelliKeys with custom

overlays

57

Access Limitations

Clearly marked

Do not administer.

A bubble is provided for all items marked as Access Limited (AL).

58

Getting Ready

59

Where to Begin

Verify that you have the correct form for your students.

Organize your assessments by forms if you have two or more forms.

60

TA/Raters Responsibilities

Read the Test Administrator Manual (TAM)

Must attend an NMPED training Assist with logistics if needed TA/Rater: must score student

responses independently and must not confer as they score

61

Schedule Location

Will you need to administer the assessment in a room that is not your classroom?

Will you need to arrange to have any special equipment (seating, table) moved to the testing location?

Consider the student’s optimal time of day.

62

Teacher-provided Materials

Objects familiar to the student: Book Pencil Ruler

Other objects: Ball Paper Clip Leaves

63

Teacher-provided Materials (cont.)

Materials typically used in instruction

Not significantly different

Objects familiar to the student

Age- and grade- appropriate

64

Advance Preparations

Read through each task. Determine which teacher-provided

materials you will need. Arrange them in an organized manner

to ease assessment administration. Consider administering one content

area to all of your students and then moving to the next content area.

65

Other Preparations

Determine if any of the students’ assistive technology devices need special programming. If so, complete these tasks prior to test administration.

Practice administering the assessment with a colleague.

Practice again. It will make the administration much smoother.

66

Important Things to Remember

Use pencil only (no ink pen please).

Use caution and no double bubbling. Be sure to use NR for items that you did not

administer during the assessment. Score as you go. Clip your printed materials to each task. Place your manipulatives in small plastic

baskets. Place Velcro on the printed manipulatives

and place them on Velcro strips.

67

More Important Things to Remember

Take breaks as needed. You may reread the item, story, or poem. If you take a break, you may review the last

item completed prior to taking a break, but do not rescore.

Do not re-administer or rescore an item. Do not lead the student by inflections in your

voice.

68

Lessons Learned: Check Your Materials Carefully

Verify that you have the correct forms for your students. Be sure to use the same grade band form

for each student for all content areas. Organize your assessments by forms if

you have two or more forms. Notify the DTC immediately if you are

missing any materials.

69

Advanced Preparation

Teacher Comments

Video Clips

70

Administering NMAPA

71

Item Information

For each item: Script Directions for setup

Placement of manipulatives Response cards Scoring directions

72

Task Format

Materials and Setup

Adaptive Instructions

Access Limitations

73

Item Scripting

Script:

Say: We’re going to talk about _________

Do: Give the student the __________

OR

Say: Show (tell) me which one means _______. This one (indicate the __card), this one (indicate the __card), or this one (indicate the __card)?

74

Practice Administration

Read a complete task.

In groups of two, administer the task to each other with one being the administrator and the other the student:

75

Practice Administration (cont.)

Think about how you would administer this task to your student(s).

Think about how you would use assistive technology with an item.

76

Appropriate Encouragement

What do you usually do to praise or encourage the student? High five “Great!” “Way to go ______!” “Awesome!”

At the completion of an item, it is OK to confirm the student’s response or show/tell the correct statement. Caution: Do not lead the student to the correct answer for the next item.

77

Scoring the NMAPA

78

Scoring

One of the key features of the NMAPA is that the Test Administrator scores student responses to each item.

79

Scoring Procedures

Teacher (Test Administrator) scores the assessment as it is administered.

Scores are recorded on the student score form—not in the test book.

An optional scoring worksheet is provided to assist with this process.

80

Item Scoring: Overview

Key features: Score points vary from item to item. At each level, the teacher follows

directions to guide a score decision. All scoring directions are on the right

side of the script. Scoring is scaffolded downward to the

directions for assigning a score of zero or NR (i.e., no response).

81

Scaffolding

Try 1If the student answers correctly record ___

and move to the next item.If the student answers incorrectly remove

____ and move toTry 2.If the student does not respond remove

____move to Try 2.

82

Scaffolding (cont.)

Try 2If the student answers correctly record ___

and move to the next item.If the student answers incorrectly remove

____ and move to Try 3 .If the student does not respond remove

____move to Try 3.

83

Scoring Rubric

Engagement Provide evidence

That the student is engaged in the task Of extended focus and persistence

Teacher makes a judgment using a scoring rubric.

84

Engagement Rubric

Score 4 - Sustained involvement

Score 3 - Generally maintained involvement

Score 2 - Intermittent/irregular involvement

Score 1 - Fleeting awareness with little or no involvement

No response - Does not demonstrate engagement in the task

85

Scoring Practice

Let’s watch as Zack is administered an the task that begins with an engagement item.

I Like Apples

86

Scoring Practice

Let’s practice scoring Tim:

Ride of My Life

87

Scoring Practice

Let’s practice scoring Tim:

Land Forms

88

Scoring Practice

Let’s practice scoring Edward:

Land Forms

89

Scoring Practice

Let’s practice scoring Joshua:

My Favorites

90

Scoring Practice

Let’s practice scoring Maddy:

Ride of My Life

91

Remember Make sure that you have all the materials that you

need before you get started. Verify that you have the correct test booklets for your

students. Rehearse the administration and scoring procedures. Get test materials organized and ready. Make sure you understand what the student is being

asked to do Practice, practice, practice!

92

What You Will Receive

93

District Materials

District Security Checklist Instructions on returning materials UPS return service labels DTC Manual Overage materials

94

District Materials

Overage Materials 10% overage test booklets and printed

manipulatives Student score forms Test Administrator Manual (TAM) Principal’s certificate

95

Teacher Materials Two student score forms per

student Biogrids are complete Each school’s Principal’s

Certification of Proper Test Administration complete and signed

All secure materials returned (test booklets, student score forms, LA storybooks, unused materials)

Security checklist Pre-ID labels (2 per student, 1 for

the TA and one for the Rater)

96

Teacher Materials

Additional BoxesTest Booklets/Manipulatives

Physical Manipulatives Kit Each kit will contain physical

manipulatives for each content area.

97

Pre-ID Labels

Pre-ID Labels 2 per student Affix to

Student score form by the TA Student score form by the Rater

98

No Pre-ID Label

Use for students who do not receive Pre-ID labels.

Complete demographic information on the TA’s student score form.

Rater’s student score form, student’s name, school, and district

99

Other Materials

Scoring Worksheet Used for recording students’ scores Must be transferred onto actual student score form

Student Placement Questionnaire (in student score form)

Used to determine most appropriate starting task in the test form for a student

100

After the Assessment

101

Complete the Student Score Form

Affix the student pre-code label Transfer the scores from the scoring

worksheet to the student score form Use pencil only (no ink pen) Return all printed cards, posters,

storybooks, student-generated artwork to your STC or DTC

102

Remember

Do not return physical manipulatives.

103

Final Important Dates

Last day of testing is April 25, 2008.

Materials must be returned to the contractor on or before April 30.

Your District Test Coordinator will tell you when to return the materials to him or her.

104

Enhanced Score Reporting

105

Enhanced Score Reporting

106

What Do People Want?

Educators want information that will help them enhance student learning and ensure compliance with NCLB and state standards.

Teachers seek information to target instruction and raise student achievement.

Parents want reports that: Are easy to interpret; Explain expectations and performance; Highlight their child’s strengths and weaknesses.

107

District Report

Personalized for each district

Explains the purpose of the assessment as well as what the results mean and how they are used

Directs educators to the PED website for additional information

108

District Report (cont.)

Describes levels of performance

Provides information for each subject tested and each grade band

Provides a comparison of each district’s score with the state score for each subject

Based on standards, not relative position

109

District Report (cont.)

Provides alphabetized rosters of all the students in the district along with their associated scale scores and performance levels

110

District Report (cont.)

Provides a link to the task-release website. This link directs educators to training materials that can be used to enhance student learning.

111

School Report (cont.)

Personalized for each school

Explains the purpose of the assessment as well as what the results mean and how they are used

Directs principals to the PED website for more information

112

School Report (cont.)

Like the district report, the school report provides principals with an alphabetized school roster of student scale scores and performance levels.

113

Family Report Contains an individualized

letter providing information to the family about the structure and function of the NMAPA

Explains the purpose of the assessment and how the results are used

Directs parents to the PED website for additional information

114

Family Report (cont.)

Describes levels of performance

Provides information for each subject tested and each grade band

Based on standards, not relative position

115

Family Report (cont.)

Provides a list of tasks that students performing in their child’s score range are typically able to do

Allows families to associate their child’s scores with real-world tasks

116

Family Report (cont.)

Uses family-friendly language and examples to explain which standards are being assessed

117

Family Report (cont.)

Provides families with individualized, family-friendly activities that are appropriate to their child’s abilities These activities can be done at home or in the community

118

Contact Information

For questions about ordering materials,

receiving materials, returning materials,

and administering the test:

Call AIR at 1-800-254-6130 E-mail AIR at NMHelpDesk@air.org

119

Contact Information

For questions about participationcriteria, state and federal regulations,and Expanded Grade BandExpectations (EGBEs):

Contact Mona Martin, NMPED, at 505-827-6577 or mona.martin@state.nm.us

120

The NMAPA Path to Success

Read the TAM. Plan for administration. Happy practicing. Contact us if you have questions. Have a safe trip home!

121

Thank you, Teachers!!

Please look at the last page of the TAM for a complete list of those who helped with the NMAPA development process. We would also like to extend special thanks to:

Deborah Chavez Charmarie Salazar Clare Stott