Embed Size (px)
Citation preview
Georgia Alternate AssessmentTeacher Training Workshop
Fall 2009
2
A New Year Begins
This slide presentation is designed to assist educators as they work to improve and fine-tune student portfolio submissions for the 2009-2010 GAA.
A companion presentation is available online for new teachers to guide them through the initial process of compiling portfolios.
These presentations serve as introductory components for training. Reading and understanding the GAA Examiner’s Manual, 2009-2010, is necessary to implement the portfolio process.
2009-2010
3
A New Year Begins
A Look Back at the 2008–2009 Administration
It’s Raining Elements—Don’t Forget the Umbrella Alignment Being True to the Standard Aligning to the Specific Aspects of the Element
4
A New Year Begins
It’s All in the Details Filling out the Entry Sheet Choosing the Appropriate Type of Evidence Annotating Evidence Documentation of Level of Independence/Prompting
Who Said What to Who? Documentation of Interaction for Generalization Documentation of Interaction to Demonstrate Independence Interactive Documentation Activity
5
A New Year BeginsAll Eyes On It
How to Structure a Peer Review of the Portfolio Checklist for Teachers and Portfolio Reviewers
And the Results Are In Training Procedures Scoring Dimensions–What Do We Look for During
Scoring It’s Your Turn
A Sneak Peek at the 2010-2011 Administration
6
That Was So Last Year!A Look Back at 2008-2009
Congratulations on a very successful administration of the 2008-2009 GAA!
Each administration has shown marked improvements in the quality of the portfolio entries. Instructional activities were well thought out and
purposeful for the students.In general, the portfolios were complete and well
executed. Entries were well organized.
2008-2009
7
That Was So Last Year! GAA Portfolios Submitted
2006-2007 2007-2008 2008-2009
Number of Portfolios submitted 10,647 10,822 10,993
Number of Portfolio Entries 59,020 59,944 60,778
The number of portfolios submitted for the GAA has gone up with each subsequent administration.
The number of portfolio entries submitted is a factor of how many portfolios are submitted by grade multiplied by how many entries are required for that grade.
Grades K–2: 4 entries
Grades 3–8 and 11: 6 entries
8
That Was So Last Year!
Hooray! The overall percentage of nonscorable entries went down.
There continue to be decreases in both number and percent of nonscorables seen in most categories.
Analysis of nonscorables provides invaluable information for the focus of future professional development (especially in those areas that increased or remained the same).
These decreases can be attributed to both the experience of the “Year 3” administration and the focus on training and peer review of portfolios.
Continued emphasis on Peer Review is critical.
9
Frequency of Nonscorablesby Number and Percent
2008-2009NS Code Number Percent Comparison to 07-08
ME 252 0.41% ▼
ES 193 0.32% ▲
NA 3152 5.19% ▲
IE 1872 3.08% ▼
IT 143 0.24% ▼
OG 37 0.06% ▼
IS 3 ≈ 0.0% ▼
Total 5652 100%
5652 of 60,778 9.30% ▼
10
That Was So Last Year! The portfolios provided valuable information as to areas of
focus for ongoing professional development. Alignment Annotating Evidence Documentation of Level of Independence Documentation of Interactions
Training and support will continue to be provided in the form of: GAA Fall Workshops, Elluminate sessions focusing on GAA administration as well as
access to the curriculum, the GAA Resource Board, and portfolio review by peers and building administrators.
1111
Instruction should be adjusted during the school year based on student performance on the GAA and other formative assessments.
Don’t wait until reports arrive in June to reevaluate the most appropriate mode of instruction and/or assessment for the individual student.
How does the GAA connect to daily practice?
12
It’s Raining Elements: Don’t Forget the Umbrella!
Instruction and assessment should promote individual student growth through alignment to the academic content via alternate achievement standards.
Alternate achievement standards are decreased in depth, breadth, and complexity, but still demonstrate a clear connection to the academic content standards.
Stand
ards
13
It’s Raining Elements: Don’t Forget the Umbrella!
Consider alignment first and foremost when designing instructional tasks.
The instructional task must be true to the standard.
The task must address the distinct characteristics of the element.
The task must be appropriately challenging for the individual student.
14
What do we mean by Alignment?Alignment is the match between the written,
taught, and tested curriculum.¹
In order for an instructional task to be considered aligned, it must demonstrate a clear connection to the Academic Content Standard and element being tested.
1. Diane Browder, 2006
Curriculum Standard
Instruction Assessed Task
15
It’s Raining Elements: Don’t Forget the Umbrella!
Be True to the Standard The curriculum standards are the goals for
instruction, learning and assessment. Achievement of the concepts and skills inherent in
the elements leads to the achievement of the overall standard.
Although tasks for assessment must align to the distinct aspects of the element, they must do so under the umbrella of the standard.
16
It’s Raining Elements: Don’t Forget the Umbrella!
Address the distinct characteristics of the element.
What are the specific components that make-up the element ?• focus on the language/terminology as written
What are some prerequisite skills to give the student access to the element?
Look to the GA Frameworks* for guidance to understanding the enduring concepts and essential components targeted by the standards and elements.
*https://www.georgiastandards.org/Frameworks.
17
It’s Raining Elements: Don’t Forget the Umbrella!
Example: Standard: ELA11LSV1– The student participates in student-to-
teacher, student-to student, and group verbal interactions.Element: c– responds to question with appropriate information The essence of this standard is reciprocal interaction between
the student and another person. The essence of the element is the response to questions. The skill assessed must demonstrate the student’s ability to
respond to questions via reciprocal interaction between the student and teacher/ another student/ group.
All 4 pieces of evidence must align to the standard and element.
Consider the following examples:
18
The student is responding to questions via his voice output device.
Does this task align to the standard and element?
YES
ELA11 LSV1
19
Teacher annotatio
n recognizes
the require
ment that th
e
essence of the element is
the
reciprocal interactio
n.
Does this task align to the standard and element?
NO. The task was completed independently with no reciprocal interaction.
Although th
e student is
responding
to te
st questio
ns. the annotatio
n clearly
states that t
here w
as no intera
ction.
NOT ALIG
NED–DO NOT U
SE
20
It’s Raining Elements: Don’t Forget the Umbrella!
As teaching academic curriculum through the academic content standards and elements becomes more a part of daily instruction, lesson plans are being designed that provide access to the curriculum while still embedding the student’s IEP goals.
But Alignment MUST come first!
21
It’s Raining Elements: Don’t Forget the Umbrella!
Creating instructional units and activities that can be used for multiple students is encouraged.
However, the activities must be tailored to the needs and the abilities of the individual student.
As such, be certain that the task is still the best choice for the individual student and that the evidence submitted clearly aligns to the academic content standard and element.
22
It’s All in the Details–Completing the Entry Sheet
The Entry Sheet serves as the Table of Contents which organizes the entry.
The Entry Sheet must be filled out completely and accurately in order for the entry to be scorable.
An electronic version of the Entry Sheet with drop-down boxes will be available online. Instructions for completing the electronic Entry Sheet
will be provided online along with the Entry Sheet.
23
2009-2010 GAA
Entry Sheet
pg. 1
24pg. 2
25
It’s All in the DetailsCompleting the Entry Sheet
It is of utmost importance that the Entry Sheet be filled out completely with all required information. Dates recorded for the tasks on the Entry Sheet must match those
found on the evidence. Task descriptions written on the Entry Sheet must be the same as those
submitted as evidence. A Characteristic of Science must be recorded on the Science Entry Sheet
and be clearly documented in the evidence. Should any of the necessary fields not be completed correctly,
the entry could be nonscorable.The Entry Sheet is NOT the place to include annotations about
student performance, prompting, settings, or interactions.
26
It’s All in the DetailsChoosing the Appropriate
Type of Evidence The type of evidence submitted should be the best means
through which to demonstrate the student’s knowledge and skills. Primary Evidence must SHOW the student’s responses during and
at the completion of the instructional activity. It is therefore vital that the type of evidence used is the
appropriate choice to clearly demonstrate the student’s response. It is important that the criteria for the type of evidence has been met and
that all necessary information has been documented. It is not recommended that worksheets or captioned photos be used to
document “verbal” responses–this makes them more like an observation than a primary type of evidence and puts excessive burden on the teacher in their documentation.
27
It’s All in the DetailsAnnotating Evidence
Complete and thorough documentation of evidence is critical! Incomplete or ineffective documentation can result in
lower scores or in the entry being nonscorable. The student’s response must be clearly and specifically
evaluated or graded. If the correctness of the student response cannot be
verified, the entry will receive the Nonscorable Code of IE (Insufficient Evidence).
Information regarding the nature of the task, the setting in which it was completed, any interactions that occurred during the task, and the type and frequency of prompting must be included.
28
It’s All in the DetailsAnnotating Evidence
Each piece of evidence must be clearly labeled with the following information:
the student's name (Who) and date (When) description of task (What) the setting in which the task was completed (Where) specific evaluation of student response (How Well) interactions that occurred during the task (With Whom) Independence–type and frequency of prompting (Prompts)
29
It’s All in the Details–Annotating Evidence
Who: the student’s name must be on each piece of evidencealways refer to the student by name, not “the student” or
“students”
What: specific description of the task what was the student asked to do?do not be vague or overly general in the description
Where: setting in which the task was completedmust be purposeful for the particular task
30
It’s All in the Details–Annotating Evidence
When: date each piece of evidence record the date on which the task was completed, even if
the task took multiple days to complete for Data Sheets, a minimum of 3 distinct dates are required
for scoring
How Well: evaluate the student’s performancedocument the questions or actions asked of the student
and his/her actual responsesgrade, score, evaluate, or provide an answer key so that
the student’s performance can be clearly determined
31
It’s All in the Details–Annotating Evidence
With Whom: describe the interactionwith whom did the student interact during the task
and what was the nature of the interactionreciprocal communication should be specifically
documentedPrompts: Level of Independence
prompting should be documented only if it guides the student to the correct answer (e.g., directions and encouragement are not considered prompting)
annotate both the type (e.g., physical, gestural, model, verbal) and frequency (e.g., continuous, frequent, limited, independent) of prompting provided
32
It’s All in the Details–Annotating Evidence
Level of Independence Increased independence, whether during academic or
functional tasks, is a primary goal for our students and an effective way through which to demonstrate Achievement/Progress.
Tasks should be designed to demonstrate the highest level of independent response of which the student is capable.
If the student can demonstrate a differentiated response via eye gaze, gesture, vocalization, or assistive technology, the student’s performance should be a reflection of that independent response before full physical prompting is employed.
33
It’s All in the Details–Annotating Evidence
Prompting– The amount of support the student requires and is given to accurately complete a task For the purpose of the assessment, prompting refers only
to that which leads the student to the correct answer. Do not include task instructions, encouragement, or
behavioral interventions when documenting prompting. Document the Type of Prompt provided (verbal, gestural,
model, physical, etc.) Document the frequency of prompting (continuous,
frequent, limited, none/independent, etc.) Documentation of Prompting should include a description
of reciprocal interaction when appropriate.
34
Who Said What to Who?
Interaction: The reciprocal exchange/communication between
the student being assessed and others which occurs during the instructional activity. This can include:
• peers (both with and without disabilities)• instructional personnel (including the special education
teacher, para-pro, general education teachers, OT, PT, Speech Therapist, or anyone else who provides regular support and instruction to the student)
• school staff (principal, nurse, cafeteria worker, etc.)• community members (job supervisor, neighbor, bus driver,
wait staff, cashier, etc.)
35
Who Said What to Who?
Interaction In order to demonstrate Generalization, annotation of
interaction must describe the nature of the interaction.• the interaction must occur during the instructional task• who said what to whom in reference to the task• do not provide a list of all the people with whom the student
has interacted without describing how that interaction occurred during the instructional task
• differentiate between prompting and interaction whenever possible
36
Who Said What to Who?
The interaction must occur during the instructional task.Scenario 1: M4G1, element c, Examine and classify quadrilaterals.Billy is in the school hallway looking for and recording the quadrilaterals he observes in the school environment (door: rectangle; library book: square) when Scott, a non-disabled general education peer, passes by and says hello. Billy returns the greeting and they speak for a few moments about the Brave’s win against the Cubs before Billy resumes his task.
Does this reciprocal communication qualify as interaction that occurred during the instructional task? NO.
37
Who Said What to Who?The interaction must occur during the
instructional task.Scenario 2: M4G1, element c, Examine and classify quadrilaterals.Billy is in the school hallway looking for and recording the quadrilaterals he observes in the school environment (door: rectangle; library book: square). Scott, a non-disabled general education peer, is working with Billy on this task and asks him questions (Is that book a quadrilateral? What shape is the book?) as they walk through the school.
Does this reciprocal communication qualify as interaction that occurred during the instructional task?
YES.
38
Who Said What to Who?The interaction must occur during the
instructional task.Scenario 3:ELA4LSV1, element c, Responds to questions with appropriate information.For his second ELA entry, Billy will be assessed on his ability to respond appropriately to questions during a conversation. He is in the school hallway looking for and recording the quadrilaterals he observes in the school environment when Scott, a non-disabled general education peer, passes by and says hello. He asks Billy if he watched the Braves game last night, and Billy responds that he did. Scott then asks if Billy caught the final score, and Billy responds, “yes, it was 4-3, Braves.” Billy then goes on to complete his geometry task.
Does this reciprocal communication qualify as interaction that occurred during the instructional task? YES.
39
Who Said What to Who?
What is the difference between interaction and prompting? Although there is certainly interaction between student and
teacher via prompting, beginning in grade 3, L/S/V standards require that the interaction documents “student-to-teacher, student-to-student, and group verbal interaction.”
“Verbal” interaction refers to whatever mode of reciprocal communication the student is capable of.
Interaction must be specific to the element assessed.• If the student is to “respond to questions,” it should be during
“student-to-teacher, student-to-student, and group verbal interaction.”
I asked him...
And he replied...
40
Interactive Documentation ActivityRead the Observation Form provided in your packet and consider the following questions.
ELA8R1 b. Applies, analyzes, and evaluates common organizational structures (e.g., graphic organizers, logical order, cause and effect relationships, comparison and contrast)
• What is the task? Does it align to the standard and element?
•Has all required documentation been included?
• Has the student’s performance on the task been clearly and specifically evaluated?
•Is the documentation of interactions clearly described and specific to those that occurred during the instructional task?
41
Interactive Documentation Activity
Example:
By focusing on one specific task that aligns to the standard and element, this Observation Form includes all information required for scoring and will allow the student to receive the most appropriate score.
42
All Eyes On ItPortfolio Review
The importance of peer review during and after completion of a portfolio cannot be overstated!
Missing forms/pieces of evidence Incomplete Entry Sheet Lacking documentation Missing dates Inappropriate materials Alignment issues
It is critical that the peer review process go beyond counting pieces of evidence and consider all of the portfolio requirements.
43
All Eyes On ItPortfolio Review
Portfolio review by the teacher should be an ongoing process. Have I selected the best standard and element for this student?
• Does he seem engaged?• Is he making progress?
Are the tasks and materials appropriate?• Is he able to demonstrate what he knows?• Can he communicate with a reliable response?
44
All Eyes On ItStructuring a Peer Review
Who should conduct the portfolio review? Trained GAA Administrator Core Access Teacher School Test Coordinator SPED Coordinator Building Administrator
When should the review be conducted? 1st Review: mid-administration, after Collection Period
1 evidence has been compiled Final Review: before submission, after all evidence
has been compiled and organized in the portfolio binder
45
All Eyes On ItStructuring a Peer Review
Steps to complete a peer review of the portfolio:
1. Carefully review the Entry Sheet• Student name• Teacher name• Required standard• Eligible standard• Strand, standard, element match up• Characteristic of Science• Task Descriptions
46
All Eyes On ItStructuring a Peer Review
2. Consider all aspects of the evidence requirements
• All four pieces of evidence align to standard and element • Primary and Secondary for each collection period• 21 calendar days from Primary to Primary• Type of evidence is the best choice to clearly
demonstrate the student response• Grade-appropriate materials
47
All Eyes On ItStructuring a Peer Review
3. Review documentation to ensure that all necessary annotation has been provided
• Collection period labels • Name (Who)• Dates (When)• Task (What)• Setting (Where)• Student performance (How well)• Interactions (With Whom and Describe)• Independence (Prompts)
4. Date and sign Checklist for Teachers and Portfolio Reviewers 5. Validation Form must be signed by Building Administrator
and Person Responsible for submitting the Portfolio.
48
A Checklist for Teachers and Portfolio Reviewers will be provided in the portfolio binder to ensure that all procedures and requirements have been satisfied before the portfolio is submitted.
•The checklist can be part of the portfolio validation process as the reviewer signs and dates the form after each content area entry is checked.
•It is recommended that the portfolio be reviewed twice– once after the first collection period has been completed, and again before the portfolio is submitted.
•The GAA Evidence Checklist, specific to each type of evidence submitted, can be found in the GAA Examiner’s Manual.
49
The GAA Validation Form will be provided in the portfolio binder to verify that all requirements and procedures have been followed and that the contents are the work of the student being assessed.
•This is an important step- the signatures validate the contents of the portfolio
•Be certain that signatures have been obtained from both the person submitting the portfolio and the building administrator.
•For transfer students, Validation Forms must be submitted by both the sending and the receiving school/system.
Required signature
Required signature
Administrator checks
one box
50
And the Results Are In What Do We Look for When Scoring the GAA Portfolios?
Evidence has been compiled, Entry Sheets have been completed, and the entries have been organized.
Portfolios have undergone peer review and have been determined to be ready for submission.
Binders have been packed in boxes and sent from the School to the System Test Coordinator and on to Questar.
It’s time for scoring.
2 3 3
51
And the Results Are InTraining ProceduresReaders are trained to score portfolios using entries
that have been scored during rangefinding sessions in Georgia with Georgia educators. Rangefinding is a process wherein teachers score actual
student entries to set the score point ranges in each dimension (e.g., determining what it takes to get a “3” in Achievement/Progress).
Entries with consensus scores are used to create training and qualifying sets for readers.
Representatives from the GaDOE are involved throughout rangefinding and are on-site and/or in constant contact throughout training and scoring.
52
And the Results Are In
Training ProceduresReaders undergo 4–5 days of extensive training and
must pass a series of qualifying tests to demonstrate that they know how to apply the scoring rubric before they can begin scoring.
Readers are monitored throughout the scoring process to ensure they are scoring accurately and consistently.
Team leaders, who serve as nonscorable experts, have previous experience in scoring the GAA and go through additional extensive training before being charged with assigning scores and nonscorable codes.
53
And the Results Are In
GAA Portfolios are scored for 4 discrete dimensionsFidelity to StandardContextAchievement/ ProgressGeneralization
Scoring is holistic – all pieces of evidence are considered and the totality of the information we have about the student’s achievement is used to make scoring decisions.
54
And the Results Are In
Fidelity to Standard assesses the degree to which the student’s work addresses the grade-level standard to which it is aligned.Does the instructional activity demonstrate a clear
connection to the standard and element? Is the student work focused on academic content at a very
introductory level considering the student’s grade level? Is the student work focused on academic content at or
approaching the student’s grade level?Does the student work address all aspects of the element?
55
And the Results Are InContext assesses the degree to which the student work exhibits the use of grade-appropriate materials in a purposeful and natural/real-world application.Are all the materials grade appropriate? Is the instructional activity a purposeful means through which the
student can learn and demonstrate what they know and can do? Is the student working in a simulated (practice) situation? (Almost
all classroom instruction is considered “simulated.”) Is the student working in a real-world (following a list to purchase
groceries) or natural situation (working in the general education classroom on the same activity as general education peers)?
56
And the Results Are In
Achievement/Progress assesses the increase in the student’s proficiency of skill across the two collection periods.Are the skills assessed across the collection periods similar
enough to reliably assess progress? Is there an increase in accuracy from one collection period
to another? Is there an increase in independence from one collection
period to another? Is there an increase in the complexity of the tasks from
one collection period to another?
57
And the Results Are In
Generalization assesses the student’s opportunity to apply the learned skill in other settings and/or with various individuals in addition to the teacher or paraprofessional.In what meaningful settings is the student
performing the activities? (The setting should be purposeful for the instructional task.)
With whom and it what way is the student interacting during the standards-based instructional activity?
58
59
And the Results Are In
Now, it’s your turn...
• Review the following student entry and assign scores for Fidelity to Standard, Context, and Achievement/Progress
• Discuss as a group.
60
61
62
63
64
65
66
A Sneak Peek
New in 2009-2010The state’s new graduation rule went into effect for
entering 9th graders in 2008-2009. Students with significant cognitive disabilities are eligible
for a high school diploma two criteria are met:1. Enrollment in and successful completion of a series of access
courses.2. A score of established or extending on all sections of the high
school GAA.The grade 11 GAA serves an alternate for the GHSGT.
67
A Sneak PeekNew in 2009-2010
A GAA must be developed for these coursesSeveral access courses correspond to End of Course Tests (EOCT):
Ninth Grade Literature and CompositionAmerican Literature and CompositionMathematics I Mathematics IIPhysical ScienceBiologyU.S. HistoryEconomics
.
68
A Sneak Peek
New in 2009-2010
Development of new EOCT for GAA will occur in2009-2010, with operational administration inthe 2010-2011 school year.
69
A Sneak Peek
GHSGT math will transition from QCC to GPS GAA math for grade 11 will transition to GPS
EOCTs are mandated for all students who are enrolled in an EOCT course.
Retest opportunities will be available for the high school GAA.
New W2 standards/elements assessed for grades K-3.The 2010-2011 Entry Sheets will be web-based in
order to collect standards and elements.• There will be no paper Entry Sheets in binders you receive–
save some trees! You can still download from the website.
2010-2011
Have on Your Radar for 2010-2011
70
Contact Information
Questions About Test Administration
Call: GaDOE Assessment Administration Division Toll free (800) 634-4106
Call: Deborah Houston, Assessment Specialist (404) 657-0251
Email: [email protected]
71
Contact Information
For information about access to the GPS for students with significant cognitive disabilities
Contact: Toni Bowen, Ph.D.Educational Program Specialist
Call: 404-463-0616 E-Mail: [email protected]
72
Contact Information
Questions About Materials, Distribution, or Collection
Call: Questar’s GAA Customer Service Toll free (866) 997-0698
Email:Questar’s GAA Customer Service [email protected]
