Training Outline Introductions General Scoring Guidelines
Mathematics Learning Progressions Levels of Scoring Clarifications
Scoring Forms, Template, Evidence Video Logistics Question &
Answer Period
Slide 3
Introductions Department of Education Gaye Fedorchak NHDOE
Director of Alternate Assessments & Access Services Allyson
Vignola Reading and Writing Scoring Leader and Content Expert Marie
Cote Mathematics and Science Scoring Leader and Content Expert
Measured Progress Staff Tina Haley - Program Manager Sarah Greene
Program Assistant Sharman Lyons Program Support Susan Izard
Measured Progress SPED Director
Slide 4
General Scoring Guidelines Score only what you see in the
portfolio Avoid biases - (See: Scoring Manual, Gen. Guidelines
Detail) Score each task entry independently, separately Respect
student confidentiality Test breach considerations Keep portfolio
in order it was submitted Do not write or leave notes in
portfolios
Slide 5
1. What are we assessing? The NH Curriculum Frameworks Its the
same for ALL. The Learning Progressions Standards as Written
Access-based definitions of the standards as written NH ALPs: 3 Big
Concepts 2. How are we observing what our students know and are
able to do? An Effective Communication System Access to Academic
Learning & Performance Student as author of own work! Active
agent
Slide 6
3. What is the Evidence of performance? Constructivism:
Authentic Tasks 3 Big Concepts, continued... Making sense of
mathematics ideas... to reason & solve problems. Show the
student actively engaging with ideas about quantity, numbers,
equality, patterns, change, measurement, & data, Show us your
student
Slide 7
Learning Progressions describe how we expect students knowledge
and skills in one content area will develop over time. Each
learning progression represents a cluster of knowledge and skills
(GLEs) that develop together based on years of professional
experience and research. Learning progressions contain many
sequential steps that are needed for certain ideas and concepts to
develop and build upon one another before full, more comprehensive
understanding of the topic is gained. Sometimes students grow
directly upward in learning new, more challenging content, but
sometimes they grow sideways learning new content at the same level
of challenge before they are ready to move upward toward higher
skill clusters. What is a Learning Progression?
Slide 8
What we expect can be wrong. Students can (and often DO)
surprise us. NH Alternate Learning Progressions are built in a way
that allows us to observe our students as they grow We can map
current levels of performance based on NH Curriculum Standards AND
we can map year-to-year growth in academic skills, last years base
is this years starting point. Learning progressions offer a clear,
concise, visual way of presenting information to families and to
educational teams when making instructional planning decisions.
Learning Progressions continued
Slide 9
4 Progressions 1. Understanding Rational Numbers 2. Solving
Problems with Operations 3. Equality 4. Patterns and Change Math
Learning Progressions
Slide 10
How to Read the Progressions The Mathematics progressions are
meant to be read as one complete sentence. When you read the
Mathematics progressions in this way, you find that every
Mathematics sub-level becomes one defined assessment task.
Slide 11
How to Read the Progressions continued... In this case there
are 5 scorable units (A1, A2, A3, A4, B1). Scorable units within
the Mathematics progressions are always indicated by a number (1,
2, 3, etc..) or alpha numeric code (A1, B2, C1, etc.).
Slide 12
Scorable units A1, A2, A3, A4 need to be demonstrated using
whole numbers 0 20 as indicated by letter A in the first column.
Scorable unit B1 is linked to letter B (1/2 as fair share) also
stated in the first column, so the student would have to
demonstrate the concept of fair share using a quantity between 1
20. How to Read the Progressions continued...
Slide 13
S TUDENT AS A UTHOR T HE C RITICAL R OLE OF A CCESS TO L
EARNING, C OMMUNICATION & P ERFORMANCE What does independent
performance mean for an alternately assessed student? Directive
prompts violate the learning construct that we are trying to
measure. No credit can be earned when directive prompts are used.
Supportive prompts allow students to show personal authorship of
standards-based tasks. Students can earn credit when supportive
prompts are used because the constructs being tested are honored
and demonstrated.
Slide 14
Two Levels of Score Review 1. Task Level Review Content
Fidelity 2. Item Level Review
Slide 15
Task Level Review Task Level Review determines whether or not
the student performance sample can advance to the Item Level Score
Review, based on sufficiency of match to the required academic
content standards (Content Fidelity). If a student performance work
sample is found not to sufficiently match the required content, the
sample does not pass task review.
Slide 16
TASK LEVEL REVIEW During this review, a scorer must ask the
following two (2) questions: 1. Is the student attempting to apply
knowledge or skills that are clearly within the content area? AND
2. Is the type of skills and knowledge clearly related to the
standards of the progression?
Slide 17
Task Level Review continued If the answer to both of these
questions is YES, then sufficient evidence of content knowledge
exists and scoring may continue to the Item Level Review. If the
answer to either or both of these questions is NO, then sufficient
evidence of content knowledge does not exist and scoring may not
continue. If student performance does not meet this criterion, then
the task sample cannot continue to the Item Level Review and the
applicable comment code must be documented.
Slide 18
Task Level Review Comment Codes 1. Student method of
communication could not be determined. Content Assessment Cover
Sheet information was not submitted and Summary Description of
Student Access to Learning and Performance was not submitted. 2.
Challenge Level and/or Sub-Level was not identified. 3. Performance
task evidence provided, as shown by student work sample, does not
match content progression. (i.e.: Required Reading text type was
not submitted, required Writing genre not submitted, required
Mathematics progression not submitted, or required Science domain
not submitted). 4. Copy or summary of text was not submitted for
Response to Text writing sample. 5. The Science column could not be
determined (last letter in the Science Content ALPs code).
Slide 19
Item Level Review An item is referred to as a scorable unit
within a content standard. There may be one or multiple scorable
units within a content standard. Each scorable unit is considered
to be worth one ( 1 ) point.
Slide 20
ITEM LEVEL REVIEW 1. Is the performance criteria met as stated
in the standard (or in the clarifications)? AND 2. Is the student
clearly the author of this academic performance? If the answer to
both of these questions is YES, then one ( 1 ) point is credited.
If the answer to either or both of these questions is NO, then no
point is credited. If student did not earn credit for any scorable
units within the chosen standard then it is considered Not
Demonstrated (ND) and the applicable comment code must be
documented.
Slide 21
Item Level Review Comment Codes 6. Quality of video impacted
score (i.e., sound quality, camera angle, clarity) 7.Video exceeded
maximum allowable length. Score review ended at maximum time.
8.Student did not demonstrate standard(s), as written, in
identified progression or sub-progression. 9. Directive prompts
prevented observation of student authorship. 10. Directive prompts
impacted observation of student authorship. 11. Science Process
Skill was not indicated. 12. Challenge level of Science Process
Skill did not match Science content challenge level in this entry.
13. Science Process Skill was duplicated; only 1 Process Skill
qualified for score review.
Slide 22
Question 1 Is the Performance Criterion Met? Mathematics
success at the item level is defined as one clear demonstration of
a mathematics scorable unit - as written that also includes clear
evidence of student authorship. One clear demonstration means that
student performance must meet the scoring criterion defined by the
scorable unit(s) with the standard- as written.
Slide 23
Question 2 Is the student clearly the author of this academic
performance? Credit or non-credit of the scorable unit is also
determined by the demonstration of student authorship. There must
be compelling evidence that the student is clearly the author, as
demonstrated in his/her performance of the scorable units (or item
responses) within the standard.
Slide 24
The following questions may serve as guidance to a scorer
judgment in making the determination regarding the presence or
absence of student authorship: Does evidence of student access to
learning, communication, and performance show student engaging in
the task as a willful agent who is effectively communicating
his/her decision-making to indicate personal choice and authorship
of performance of this scorable unit/item? Did the teacher provide
supportive prompts and not directive prompts? (See Training Manual)
Is there a biasing response context? Student as Author
Slide 25
Student As Author continued Is the student able to communicate
choices effectively? If a student has a severe motor impairment and
some responses appear to be a result of the clear willful intention
of the student but some responses seem possibly accidental or
random, is evidence of student response control sufficiently clear
to make authorship compelling? (Requires scorer judgment)
Slide 26
Terminology Used in Mathematics Progressions The words used in
the content standard as written are very important. In addition to
the wording interpretation guide below, the definitions provided in
NH ALPs Master Glossary provide an additional reference to help
clarify the meaning of the words used in the standard.
Understanding the specificity of the Mathematics scorable units as
they are written. Multiple requires demonstration of three (3) or
more unless standard permits less. Quantity requires demonstration
of how many or an amount. Magnitude means demonstration of quantity
NOT size. Or requires demonstration of either one or the other
connected by the word or (minimum of one is required).
Slide 27
Terminology continued And or the use of & when shown within
the Stem concept or when shown between standards (scorable units)
is meant to be read as a complete mathematical sentence. Students
should demonstrate as many standards (scorable units) as possible
within the mathematical sentence. And or the use of & within a
scorable unit requires demonstration of both elements connected by
the word and for credit to be earned.
Slide 28
Terminology continued / is treated as an indication that each
choice separated by the / must be offered to the child. (i.e.
identify objects which are bigger, /smaller, /about the same. Point
is to see child making the comparison and choice among all of these
specified alternatives. The / in Patterns and Change, Emergent,
sub-levels 1, 2, & 3 will be an exception. For these three
sub-levels the / will mean or as it refers to sounds/movement and
colors/textures. Commas in a series with no other connecting words
are treated as an OR statement, as follows: bigger, smaller, same-
the commas will be treated as bigger or smaller or same Commas in a
series with other connecting words are treated as the connecting
word implies, as follows: bigger, smaller, and same- the commas
will be treated as bigger and smaller and same
Slide 29
Clarifications Compare and Order Making Estimates Number
Sentence and Expression (Expression vs. Equation) Verbal
Explanation Finding a Missing Element Repeating (3 iterations) and
Growing Patterns Models (Objects, Materials, Models) Tables,
Graphs, Charts, Etc Number Parameters at different levels. Area,
Set, and Linear Models Ordering by Attributes
Slide 30
Compare and Order Understanding Rational Numbers Comparing
Numbers: Comparing numbers, a and b, means to determine if a is
less than b, if a is greater than b, or if a is equal to b. At
level 17, students are using equality and inequality symbols to
express the comparison (=, ,, ) Compare and Order: These are very
solid examples. Teacher stimulus presents an unordered group of
numbers; credit given if the student puts them in order and
describes why a comes before b. Teacher stimulus presents an
unordered group of numbers; credit given if the student puts them
in order smallest to largest or largest to smallest. Teacher
stimulus presents an unordered set of numbers; credit given if the
student takes just two numbers and finds bigger/smaller (compares),
and then puts them in order (sequence). Teacher stimulus presents
an unordered group of numbers; credit given if the student puts
them in order and demonstrates understanding of one greater than
the stimulus sample.
Slide 31
Making Estimates Operations Makes estimates: Credit is given if
student demonstrates scorable unit A and B congruent or independent
of each other. Teacher stimulus may present A and B together using
the same objects OR present A and B separate using two different
objects on two different days. Students who are nonverbal may be
given a choice. Teacher stimulus presents 7 cubes. Teacher shows
student number cards, one with the number 10 and one with the
number 20; credit given if the student demonstrates estimation by
choosing (per individual means of communication) the number card
with the number 10.
Slide 32
Estimation continued Method of Estimation: An estimate is an
approximation or rough calculation. Estimation requires good
intuition about numbers, good understanding of problem situations,
and a flexible repertoire of techniques. Methods include: Rounding:
replacing a number or numbers in a computation with the closest
multiple of 10, 100, 1000, etc. to make the computation easier to
do Substituting compatible numbers: replacing some or all of the
numbers in a computation with numbers that are easier to work with
Front end estimation: calculating with the leftmost, or front- end,
digit of each number as if the remaining digits were all zeros
Slide 33
Number Sentence and Expression (Expression vs. Equation) Number
Sentence: A number sentence is an equation or inequality such as
10=8+2, 12 n = 4, or 14 > 5. A number sentence has a left hand
side, a relation symbol, and a right hand side. Each side of a
number sentence is a numerical expression. Number sentences can be
true, false, or neither true nor false. A number sentence that is
neither true nor false is called an open sentence (see Equality
Progression).
Slide 34
Verbal Explanation Equality Credit is given for standard A2
describe/describing, if student provides a verbal response, but not
necessarily a spoken response, per individual means of
communication. Starting at standard B3A4, a verbal explanation is
required, (this permits other communication methods consistent with
definition of explanation in glossary) If student provides a verbal
explanation using words, they also receive credit for standard
A4.
Slide 35
Finding a Missing Element Patterns & Change Starting at
standard B1A2 - Finds a missing element - Credit is given if
student identifies a missing element from some pattern. Missing
element can be at the beginning, middle, or end of the pattern
(i.e.: 2 2 3 1 2 2 3 _)
Slide 36
Repeating and Growing Patterns Patterns & Change Concepts
B3 & B4 Growing Numeric Patterns- Credit is given if student
identifies an extension of a pattern, or an extension of a
decreasing pattern. Increasing is defined as growing or extending,
as related to these standards Decreasing is defined as extending a
decreasing linear pattern, as related to these standards
Slide 37
Models (Objects, Materials, Models) Equality Models: At the
Emergent level only, models will be synonymous with demonstrate.
Using Models: A student uses models when he/she uses manipulatives,
pictures, diagrams, graphs, or mathematical symbolism to simulate
real-life situations and understand quantitative
relationships.
Slide 38
Tables, Graphs, Charts, Etc Patterns & Change Tables:
Numbers or quantities arranged in labeled rows and/or columns.
Graphs: A diagram of values, usually shown as lines or bars with
consistent and appropriate scales. Each dimension of the graph
should have its own scale. Graphed values can also be represented
in pie chart, box and whiskers data plot, scatter plot, bubble
plot, or in other graphic forms. Graphed values are diagrammed
along scales that are labeled, consistent, and appropriate to the
data being presented. If there is only one type of value
(variable), the graph is on a single number line (presented in one
dimension). If there are two variables, the graph is on the
coordinate plane. (Shown in two dimensions). If there are three
variables, the graph is shown in three-dimensional coordinates. In
general, for n variables, the graph is in n dimensions. Each
dimension (or axis) of a graph should have its own labeled,
consistent, and appropriate scale.
Slide 39
Charts, Graphs, Tables continued If the scorable unit, as
written, requires a graph, but only a table is presented, then no
credit can be given. If the scorable unit, as written specifies a
certain type of graph, then that type of graph must be shown to
earn credit. If there is a graph in the student performance
evidence, per condition of the scorable unit, and if the graph
shows the coordinates without the values labeled, credit cannot be
given. The graphed values must be represented & appropriately
labeled. One way the graphed coordinate values may be represented
(but not the only way) could be in a table format in addition to
(paired with) the graph. If the scorable unit, as written specifies
a table, but only a graph is presented, then no credit can be
given.
Slide 40
Number Parameters at Different Levels When number parameters
are given, students must demonstrate numbers beyond the previous
level in order to receive credit for scorable units. Rational
Numbers
Slide 41
Area, Set, and Linear Models Area Models: Students may use many
types of manipulatives and/or pictures to develop area models
(e.g., pattern blocks, circular fraction regions, pizzas,
geoboards, spinners). An area model can be used to represent part
to whole relationships for fractions, decimals, and percents. The
entire model may represent the whole where the model is divided
into parts of equal area, the model given may represent a part
where the whole is to be determined, or the model given may
represent a part where another part is to be determined. Shade one
square, partitioned vertically, to represent 3/8 (shown below in
pink): Shade another square, partitioned horizontally, to represent
2/3 (shown below in blue): Superimpose the two squares. The product
is the area that is double-shaded (shown below in purple):
Slide 42
Models continued Linear/Measurement Models: Linear models
include number lines, scales (temperature), and linear
measurements. Linear models can be used in a similar fashion as
area and set models. Students use the number line model found on
rulers or divide fraction strips into the appropriate sections by
length. Teachers may use any of these manipulatives to develop the
measurement or linear model: number line rulers linking cube trains
Cuisenaire rods fraction bars fraction strips
Slide 43
Models continued Set Models: Since a set is a collection of
objects, demonstrating understanding of part to whole relationship
in a set model means to identify a fractional part of a set, or
identify the fraction represented. Additionally, as with area
models, the set given may represent a part where the whole is to be
determined, or the set given may represent a part where another
part is to be determined. Students may use many types of
manipulatives and/or pictures to develop a set model (e.g.,
pictures, cubes, foam shapes, etc.). See glossary for
examples.
Slide 44
Patterns Using cards student orders cards 1(ace) through 10 and
then teacher questions to have student explain why they ordered
that way (this is ordering by attribute of number) 5 triangles of
the same shape but of different sizes could be used to demonstrate
understanding of ordering by size These activities could easily be
turned into patterning by asking the student to continue the
pattern that was already demonstrated through ordering.
Slide 45
Problem Type Operations E1, E2 Joining action is permitted at
levels 1 and 2 since part-part-whole is a simpler version of a
joining action.
Slide 46
Problem Type Chart
Slide 47
Scoring Sample
Slide 48
Template Components Portfolio Validation Form Decision Making
Worksheet Parent Review Statement Informed Consent Communication
Inventory (Must be present. If not, check Content Cover Sheet. If
both are missing, content area cannot be scored.)
Slide 49
Slide 50
Required: Mathematics Assessment Cover Sheet Required: Video
for 4 Tasks Required: Concepts of Rational Numbers Cover Sheet
Required: Solving Problems with Operations Cover Sheet Required:
Equality Cover Sheet Required: Patterns and Change Cover Sheet
Supplemental: A copy of any paper products used/produced (tests,
worksheets, cards, etc) Supplemental: Written Transcript of audio
portions of video that are difficult to understand Math Evidence
Collection Documentation
Slide 51
Scoring Materials Student Score Form Step by Step Scoring
Procedures for Mathematics Scoring Comment Codes Training Manual:
Learning Progressions Glossary
Slide 52
How to View the Video Insert thumb drive Go to My Computer
Select removable disc Choose appropriate content area submission
Double click to view If video doesnt play please notify table
leader
Slide 53
Removing Thumb Drive Close video screen Remove Thumb Drive You
do not need to do a complete safe removal of the thumb drive Place
thumb drive back into envelope as soon as it is removed so it does
not get misplaced among other scoring materials.
Slide 54
Scoring Samples
Slide 55
Scoring Logistics 8:00 am - Start 10:00 - 10:15am Break 12:00
-12:30 - Lunch 2:00 2:15 Break 4:00 End
Slide 56
Organizing Your Space Keep all portfolio components together
Leave unneeded documents in the Tyvek envelope Take only the
paperwork you need for a particular entry
Slide 57
Protocol for Screening/Scoring Questions: Scorer Table Leader
Scoring Leader and Content Expert Marie MP Staff Tina
Slide 58
Finding Samples Videos that clearly and fully show the
demonstration of the chosen standard. Creative activities used to
demonstrate standards. If you find evidence that you feel could be
used as a sample, please fill out an Excellent Example form and
give to your table leader