Upload
ora
View
38
Download
0
Tags:
Embed Size (px)
DESCRIPTION
New Mexico Alternate Performance Assessment. (NMAPA) Test Administrator & Rater Training New Mexico Public Education Department American Institutes for Research February 2008. NMPED American Institutes for Research NMAPA Team. Mona Martin, Assessment Administrator, NMPED - PowerPoint PPT Presentation
Citation preview
1
New Mexico Alternate Performance Assessment
(NMAPA)
Test Administrator & RaterTraining
New Mexico Public Education DepartmentAmerican Institutes for Research
February 2008
2
NMPEDAmerican Institutes for ResearchNMAPA Team
Mona Martin, Assessment Administrator, NMPED
Lynnett Wright, Project Director, AIR
Dan Hauser, Operations, AIR
Werner Wothke, Technical Advisor, AIR
3
Administration Window
March 10 to April 25
Testing materials will arrive in the districts by February 29.
4
Old Alt – New Alt
(OLD) Original NM Alternate Assessment
(NEW) New Mexico Alternate Performance Assessment
See handout for key comparisons
5
Alternate Assessments
Alternate assessments are designed for the small number of students who are unable to participate in regular grade-level state assessments even with appropriate accommodations. (IDEA 1997 & NMAC)
6
Alternate Assessments
Must be aligned to the state’s content standards
Must yield results in both English language arts and mathematics and must be designed and implemented in a manner that supports use of results as an indicator for AYP
Can measure progress based on alternate achievement standards (NCLB)
7
Alternate Achievement Standards
Set an expectation of performance that differs in complexity from a grade-level achievement standard
Must be linked to state’s academic content standards
Must promote access to the general education curriculum
Must reflect professional judgment of the highest achievement standards possible (NCLB 34 C.F.R.§200.1(d))
8
NM Participation CriteriaNMAC 6.31.2.11
(a) The student’s past and present levels of performance in multiple settings (i.e., home, school, community) indicate that a significant cognitive disability is present;
(b) The student needs intensive, pervasive, or extensive levels of support in school, home, and community settings; and
(c) The student’s current cognitive and adaptive skills and performance levels require direct instruction to accomplish the acquisition, maintenance, and generalization of skills in multiple settings (home, school, community).
9
NMAPA: Purpose
To promote maximum access to the general education curriculum
To ensure that all students are included in the statewide assessment program
To drive instruction and appropriate pedagogical practices with high expectations
10
Test Administrator Requirements
A certified employee of the district
An employee of the district who is a special education teacher who has an interim certificate
Test Administrators may not administer the assessment to close relatives.
11
Training Requirements
Every Test Administrator (generally the student’s teacher) who will administer the NMAPA must attend one of the statewide training sessions.
12
Administration and Scoring Fidelity
A random sample of the test administrations will be double-scored.
Random sample Based on the students Determined by PED
13
Review and Refinement Process
Rigorous review and refinement by NMPED and AIR, as with any content area achievement test (e.g., fairness, psychometric)
Bias and content review panels of expert New Mexico teachers
14
Test Administrator
A random sample of NMAPA administrations must be double-scored: The Test Administrator’s record is scored for AYP; the Rater’s record is used for our inter-rater reliability study.
Test Administrator: Teacher, related service provider, diagnostician
Rater: Any of the above and paraprofessionals (EAs)
15
New Mexico Expanded Grade Band Expectations (EGBEs)
Assessment Frameworks
16
EGBE Development
Began task force work in early spring 2006, finalized on December 15, 2006: focus on academic expectations
Involved a variety of diverse stakeholders from throughout the state, including our Alternate Assessment Advisory Council
17
EGBE Development
Special, ELL, and general education teachers and administrators wrote and reviewed the EGBEs.
They are available on the Assessment and Evaluation Bureau’s homepage, along with an approval memo from Dr. Garcia and a user’s guide.
18
EGBE Assessment Frameworks
Linked to grade-level content standards
Basis for IEP goals and objectives
Basis for classroom instruction for students with significant cognitive disabilities
19
Importance of Font in the EGBEs
Bold = This EGBE is assessed on this year’s NMAPA.
Italic = This is an instructional objective that may be on future tests but is currently to be used for classroom assessment only.
20
Levels of Complexity
Complexity in terms of demands on students Content (e.g., from simple to complex standards) Cognitive (e.g., from concrete to more abstract
understanding) Communication levels
Extended Symbolic (Clusters 7 and 8) Symbolic (Clusters 5 and 6) Pre-symbolic (Clusters 3 and 4) Engagement (Clusters 1 and 2)
21
Link to All of the EGBEs
http://www.ped.state.nm.us/div/acc.assess/assess/Expanded_Grade_Band_Expectations/egbe.html
22
Lessons Learned
Teachers may use the EGBEs as a basis for IEP goals.
Teachers should teach from the EGBEs.
The NMAPA is directly drawn from the EGBEs.
23
Determining Starting and Concluding Tasks
24
Pre-assessment to determine the most appropriate starting point
Student Placement Questionnaire (SPQ) 12–15 “can do” statements addressing
student skills and knowledge based on teacher’s prior knowledge
Allows maximum opportunity for student to demonstrate his or her skills without prolonging the assessment
25
Student Placement Questionnaire
SPQ is designed to identify the most appropriate starting task for each student.
Teachers complete a series of questions regarding student knowledge of the content.
Based on responses, teachers compute a score to determine most appropriate starting task in the test form.
26
Student Placement Questionnaire (cont.)
Each content area SPQ is located in the student score form.
The SPQ contains directions for Identifying the starting task using the SPQ; Administering and concluding the NMAPA.
All students will complete at least these tasks: 1–5 or 3–9 or 6–12
27
Student Placement Questionnaire (cont.)
Student “responds successfully” to the beginning task
When the student gets at least three points on all of the items in the beginning task.
If the student is not successful, you should drop back to the beginning of the previous group of tasks.
28
Student Placement Questionnaire (cont.)
Student who “responds successfully” to the concluding task: Continue to the next task, then the next,
and so forth until the student no longer responds successfully.
Responding successfully When the student gets at least three
points on the items in the concluding task
29
Student Placement Questionnaire (cont.)
For example: Starting at Task 1: Tasks 1–5, 6, 7, …
Starting at Task 3: Tasks 3–9, 10, 11, …
Starting at Task 6: Tasks 6–12
30
Student Placement Questionnaire (cont.)
Possibilities
Starting at Task 11 2 3 4 5 6 7 8 9 10 11 12
31
Student Placement Questionnaire (cont.)
Possibilities
Starting at Task 33 4 5 6 7 8 9 10 11 12
32
Student Placement Questionnaire (cont.)
Possibilities
Starting at Task 66 7 8 9 10 11 12
33
Student Placement Questionnaire (cont.)
Possibilities Starting at Task 1
Starting at Task 3
Starting at Task 6
1 2 3 4 5 6 7 8 9 10 11 12
3 4 5 6 7 8 9 10 11 12
6 7 8 9 10 11 12
34
Move Forward or Stop Example
Task 5Successful
Task 6
2 points - STOP
3 points – GO on to Task 7
2 points - STOP
35
Lessons Learned
It is important to follow the directions for completing the SPQ accurately.
The student’s teacher or Test Administrator must complete the SPQ.
If you start too high, you must drop back.
36
Video Clip
Let’s watch as Edward is administered an the task -
Being Healthy and Safe
37
NMAPA Task and Item Information
38
NMAPA Tasks
A task is a collection of items and materials organized around a theme (e.g., a story, a writing activity, a math activity). Tasks contain 4 to 6 items.
Each item Scripted Scaffolded to reduce complexity Students can respond verbally or nonverbally Includes directions for scoring student
responses
39
NMAPA Tasks
Materials include printed and physical manipulatives, storybooks.
Almost all materials are provided in a manipulatives kit.
40
Task Information
Cover page includes Materials needed to administer each
item Some are provided by the teacher.
Special adaptive instructions Access limitations and instructions Introductory and closure statements
41
Item Information
For each item:
Materials Directions for setup
Placement of manipulatives Response cards Displaying a text
42
Item Scripting
Scaffolded scripting
Opening statement in a say/do format
Tell me or show me
43
Item Scripting
Opening statement or question “Here is a _______________.”
“We are going to talk about __________”
Followed by: The student showing or telling which
one is the correct response option or which one is correct
44
Graphic Setup
Tells where each of the materials is to be placed.
Script is always on the left.
Scoring scaffolding directions are always on the right.
45
Scaffolding
If the student does not answer correctly or fails to respond, there are specific instructions for the Test Administrator.
These instructions are in boxes within each test item.
46
Scaffolding (cont.)
For example:• If a student indicates _________,
record a 3. If not, continue with the scaffolded script below.
47
Item Format
Example of an Item
48
Video Clips
Let’s watch as Devin is administered the task -
Money Identification
And
Tim is administered
Ride of My Life
49
Adapting and Accommodating to Meet Student’s Needs
50
Allowable Accommodations
Mayer-Johnson pic-syms have been used throughout the tasks and items.
If your student uses a different symbol for the same word, you may substitute that symbol for the one provided. For example:
51
Adapting and Accommodating Teacher can say the
response option aloud. Teacher can point to all
of the response options or concrete objects.
Have student touch or hold the object.
Place the objects in a specific location or orientation when the student has a limited visual field.
Change the background.
52
Wording
You may substitute a more familiar word or term for scripted language as long as doing so does not change what is being measured.
“Tennis shoes” for “sneakers” is fine.
53
Assistive Technology
54
Assistive Technology
Low-Tech Not electrical or
mechanical Specifically designed
devices for a specific purpose:
Special spoons Corner chairs Large pencil grips Photographs Raised lines
55
Assistive Technology (cont.)
Medium-Tech Less sophisticated high-
tech devices May have electronic or
mechanical components
Single-message communication devices
Computer programs
56
Assistive Technology (cont.)
High-Tech• Computer• Speech to text • Text to speech• IntelliKeys with custom
overlays
57
Access Limitations
Clearly marked
Do not administer.
A bubble is provided for all items marked as Access Limited (AL).
58
Getting Ready
59
Where to Begin
Verify that you have the correct form for your students.
Organize your assessments by forms if you have two or more forms.
60
TA/Raters Responsibilities
Read the Test Administrator Manual (TAM)
Must attend an NMPED training Assist with logistics if needed TA/Rater: must score student
responses independently and must not confer as they score
61
Schedule Location
Will you need to administer the assessment in a room that is not your classroom?
Will you need to arrange to have any special equipment (seating, table) moved to the testing location?
Consider the student’s optimal time of day.
62
Teacher-provided Materials
Objects familiar to the student: Book Pencil Ruler
Other objects: Ball Paper Clip Leaves
63
Teacher-provided Materials (cont.)
Materials typically used in instruction
Not significantly different
Objects familiar to the student
Age- and grade- appropriate
64
Advance Preparations
Read through each task. Determine which teacher-provided
materials you will need. Arrange them in an organized manner
to ease assessment administration. Consider administering one content
area to all of your students and then moving to the next content area.
65
Other Preparations
Determine if any of the students’ assistive technology devices need special programming. If so, complete these tasks prior to test administration.
Practice administering the assessment with a colleague.
Practice again. It will make the administration much smoother.
66
Important Things to Remember
Use pencil only (no ink pen please).
Use caution and no double bubbling. Be sure to use NR for items that you did not
administer during the assessment. Score as you go. Clip your printed materials to each task. Place your manipulatives in small plastic
baskets. Place Velcro on the printed manipulatives
and place them on Velcro strips.
67
More Important Things to Remember
Take breaks as needed. You may reread the item, story, or poem. If you take a break, you may review the last
item completed prior to taking a break, but do not rescore.
Do not re-administer or rescore an item. Do not lead the student by inflections in your
voice.
68
Lessons Learned: Check Your Materials Carefully
Verify that you have the correct forms for your students. Be sure to use the same grade band form
for each student for all content areas. Organize your assessments by forms if
you have two or more forms. Notify the DTC immediately if you are
missing any materials.
69
Advanced Preparation
Teacher Comments
Video Clips
70
Administering NMAPA
71
Item Information
For each item: Script Directions for setup
Placement of manipulatives Response cards Scoring directions
72
Task Format
Materials and Setup
Adaptive Instructions
Access Limitations
73
Item Scripting
Script:
Say: We’re going to talk about _________
Do: Give the student the __________
OR
Say: Show (tell) me which one means _______. This one (indicate the __card), this one (indicate the __card), or this one (indicate the __card)?
74
Practice Administration
Read a complete task.
In groups of two, administer the task to each other with one being the administrator and the other the student:
75
Practice Administration (cont.)
Think about how you would administer this task to your student(s).
Think about how you would use assistive technology with an item.
76
Appropriate Encouragement
What do you usually do to praise or encourage the student? High five “Great!” “Way to go ______!” “Awesome!”
At the completion of an item, it is OK to confirm the student’s response or show/tell the correct statement. Caution: Do not lead the student to the correct answer for the next item.
77
Scoring the NMAPA
78
Scoring
One of the key features of the NMAPA is that the Test Administrator scores student responses to each item.
79
Scoring Procedures
Teacher (Test Administrator) scores the assessment as it is administered.
Scores are recorded on the student score form—not in the test book.
An optional scoring worksheet is provided to assist with this process.
80
Item Scoring: Overview
Key features: Score points vary from item to item. At each level, the teacher follows
directions to guide a score decision. All scoring directions are on the right
side of the script. Scoring is scaffolded downward to the
directions for assigning a score of zero or NR (i.e., no response).
81
Scaffolding
Try 1If the student answers correctly record ___
and move to the next item.If the student answers incorrectly remove
____ and move toTry 2.If the student does not respond remove
____move to Try 2.
82
Scaffolding (cont.)
Try 2If the student answers correctly record ___
and move to the next item.If the student answers incorrectly remove
____ and move to Try 3 .If the student does not respond remove
____move to Try 3.
83
Scoring Rubric
Engagement Provide evidence
That the student is engaged in the task Of extended focus and persistence
Teacher makes a judgment using a scoring rubric.
84
Engagement Rubric
Score 4 - Sustained involvement
Score 3 - Generally maintained involvement
Score 2 - Intermittent/irregular involvement
Score 1 - Fleeting awareness with little or no involvement
No response - Does not demonstrate engagement in the task
85
Scoring Practice
Let’s watch as Zack is administered an the task that begins with an engagement item.
I Like Apples
86
Scoring Practice
Let’s practice scoring Tim:
Ride of My Life
87
Scoring Practice
Let’s practice scoring Tim:
Land Forms
88
Scoring Practice
Let’s practice scoring Edward:
Land Forms
89
Scoring Practice
Let’s practice scoring Joshua:
My Favorites
90
Scoring Practice
Let’s practice scoring Maddy:
Ride of My Life
91
Remember Make sure that you have all the materials that you
need before you get started. Verify that you have the correct test booklets for your
students. Rehearse the administration and scoring procedures. Get test materials organized and ready. Make sure you understand what the student is being
asked to do Practice, practice, practice!
92
What You Will Receive
93
District Materials
District Security Checklist Instructions on returning materials UPS return service labels DTC Manual Overage materials
94
District Materials
Overage Materials 10% overage test booklets and printed
manipulatives Student score forms Test Administrator Manual (TAM) Principal’s certificate
95
Teacher Materials Two student score forms per
student Biogrids are complete Each school’s Principal’s
Certification of Proper Test Administration complete and signed
All secure materials returned (test booklets, student score forms, LA storybooks, unused materials)
Security checklist Pre-ID labels (2 per student, 1 for
the TA and one for the Rater)
96
Teacher Materials
Additional BoxesTest Booklets/Manipulatives
Physical Manipulatives Kit Each kit will contain physical
manipulatives for each content area.
97
Pre-ID Labels
Pre-ID Labels 2 per student Affix to
Student score form by the TA Student score form by the Rater
98
No Pre-ID Label
Use for students who do not receive Pre-ID labels.
Complete demographic information on the TA’s student score form.
Rater’s student score form, student’s name, school, and district
99
Other Materials
Scoring Worksheet Used for recording students’ scores Must be transferred onto actual student score form
Student Placement Questionnaire (in student score form)
Used to determine most appropriate starting task in the test form for a student
100
After the Assessment
101
Complete the Student Score Form
Affix the student pre-code label Transfer the scores from the scoring
worksheet to the student score form Use pencil only (no ink pen) Return all printed cards, posters,
storybooks, student-generated artwork to your STC or DTC
102
Remember
Do not return physical manipulatives.
103
Final Important Dates
Last day of testing is April 25, 2008.
Materials must be returned to the contractor on or before April 30.
Your District Test Coordinator will tell you when to return the materials to him or her.
104
Enhanced Score Reporting
105
Enhanced Score Reporting
106
What Do People Want?
Educators want information that will help them enhance student learning and ensure compliance with NCLB and state standards.
Teachers seek information to target instruction and raise student achievement.
Parents want reports that: Are easy to interpret; Explain expectations and performance; Highlight their child’s strengths and weaknesses.
107
District Report
Personalized for each district
Explains the purpose of the assessment as well as what the results mean and how they are used
Directs educators to the PED website for additional information
108
District Report (cont.)
Describes levels of performance
Provides information for each subject tested and each grade band
Provides a comparison of each district’s score with the state score for each subject
Based on standards, not relative position
109
District Report (cont.)
Provides alphabetized rosters of all the students in the district along with their associated scale scores and performance levels
110
District Report (cont.)
Provides a link to the task-release website. This link directs educators to training materials that can be used to enhance student learning.
111
School Report (cont.)
Personalized for each school
Explains the purpose of the assessment as well as what the results mean and how they are used
Directs principals to the PED website for more information
112
School Report (cont.)
Like the district report, the school report provides principals with an alphabetized school roster of student scale scores and performance levels.
113
Family Report Contains an individualized
letter providing information to the family about the structure and function of the NMAPA
Explains the purpose of the assessment and how the results are used
Directs parents to the PED website for additional information
114
Family Report (cont.)
Describes levels of performance
Provides information for each subject tested and each grade band
Based on standards, not relative position
115
Family Report (cont.)
Provides a list of tasks that students performing in their child’s score range are typically able to do
Allows families to associate their child’s scores with real-world tasks
116
Family Report (cont.)
Uses family-friendly language and examples to explain which standards are being assessed
117
Family Report (cont.)
Provides families with individualized, family-friendly activities that are appropriate to their child’s abilities These activities can be done at home or in the community
118
Contact Information
For questions about ordering materials,
receiving materials, returning materials,
and administering the test:
Call AIR at 1-800-254-6130 E-mail AIR at [email protected]
119
Contact Information
For questions about participationcriteria, state and federal regulations,and Expanded Grade BandExpectations (EGBEs):
Contact Mona Martin, NMPED, at 505-827-6577 or [email protected]
120
The NMAPA Path to Success
Read the TAM. Plan for administration. Happy practicing. Contact us if you have questions. Have a safe trip home!
121
Thank you, Teachers!!
Please look at the last page of the TAM for a complete list of those who helped with the NMAPA development process. We would also like to extend special thanks to:
Deborah Chavez Charmarie Salazar Clare Stott