Upload
tejana
View
37
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Achievement Exam Analysis. ED3604E/F February 5/9, 2010 adapted from materials created by Gerry Varty, Director of Instruction Wolf Creek Public Schools. Achievement Exams. Administered by Alberta Learning yearly to all students in Grades 3, 6, and 9 Purpose: - PowerPoint PPT Presentation
Citation preview
Achievement Exam Analysis
ED3604E/FFebruary 5/9, 2010
adapted from materials created byGerry Varty, Director of Instruction
Wolf Creek Public Schools
Achievement Exams Administered by Alberta Learning yearly to all
students in Grades 3, 6, and 9
Purpose: Serves as a ‘dipstick’ measurement Gauges general performance of a large population Measures effectiveness of curriculum and
initiatives designed to effect learning Should be used to improve teaching and learning
What they are not: An effective measure of individual student ability
An effective measure of school performance in the short-term
A ‘gateway’ or filter to direct individual student promotion.
“High-Stakes” tests. Nobody fails, nobody gets fired, no school loses funding or autonomy over student performance on these exams.
What they are: Data, through which we can see trends indicating
the effectiveness of initiatives (such as AISI) designed to improve student learning.
General measures of Program content and delivery over time
“Large-Scale” assessments – enough students write them to create a basis for consistent appraisals.
Only a piece of the larger assessment puzzle
Those distinctions are important… By using PATs for purposes they were
designed for: Government can use them to determine the
general level of attainment over time Schools can use them to gauge the
effectiveness of program enhancement, or to serve as a springboard for discussion and professional development
Parents can use them to get an idea of how their kids perform (relative to the large group) on that specific sample of outcomes.
Those distinctions are important… Using PATs for purposes they were not
designed creates issues such as: Schoolworks, Fraser Institute, etc
‘ranking’ schools (or teachers) based on aggregate PAT data
PAT data being used to determine grade or program placement for individual kids
This data is only one part of the story.
What to look for: Performance over time that indicates
strengths or weaknesses in Programs. Where there are strengths, look at those, to
seek out Best Practices which might be possible to replicate in other areas
Where there are weaknesses, they can be addressed those through collaboration and Professional Development initiatives.
Strengths of Achievement Testing
Reliable, valid data. Every student in the province writes the same exam. If others are achieving where we are not, we know that we can improve on that area.
Good fit to many curricular objectives. While they do not tell the whole story, they do tell a useful part of it.
The trick is to find the useful parts; sometimes, the very act of searching for those is an opening for great collaboration and sharing of ‘best practices’ among teachers.
PATs aren’t everything… But they’re not nothing, either.
Perhaps the most important function of PATs is to help us figure out which questions we should be asking…
‘Cause any way you cut it, if you teach the curriculum well, the PATs should take care of themselves. When they don’t, we have a problem.
Weaknesses of Achievement Testing You can’t measure everything that’s
important with these. Don’t try. Uber-Security. Not being able to sit
down with the test and look at (and discuss) how the specific outcomes are measured.
Multiple-choice. Questionable statistical practices; cut-
scores, equating, over-analysis…
This data is from one school Did the school show any changes over
this time? What conclusions can you draw, or what
possibilities can you suggest? Does anything need deeper analysis, or
ask for more information?
What else do you need to know about this school?
What do the numbers mean? In their raw form, not very much. Consider these 2 schools:
Sunshine School: 80% acceptable, 20% excellent
Gloomy Academy: 80% acceptable, 20% excellent
Which one is better?
Analyze the Data Distribution for Sunshine School:
Sunshine School
05
1015202530354045
10 20 30 40 50 60 70 80 90
Analyze the Data Distribution for Gloomy Academy
Hmm… they don’t look the same anymore…
Sunshine School
05
1015202530354045
10 20 30 40 50 60 70 80 90
Gloomy Academy
05
1015202530354045
1 2 3 4 5 6 7 8 9
They both have the same statistics…
80% Acceptable20% Excellent
But they’re so different
The problem with ‘cut scores’ Is that the range is too big. Cut scores tell us that a student
with a 54% is the same as one who scored 86%… acceptable.
Cut scores don’t tell you what kids are good at, or help you improve your instruction.
There 3 types of Assessment: Assessment of Learning Assessment for Learning Assessment for Teaching
PATs are two of these. Trying to make them fit the second purpose is a mistake; neglecting them for the third is an even greater error. If they can’t inform our teaching, then they are of no value to us.
Grading/Reporting
AssessmentEvaluation
Assessment of Learning Is what we used to call Summative
Assessment. The process of measuring the Learner’s progress
against the pre-determined standard for knowledge or performance
Reflects a judgment regarding the quality or adequacy of student achievement as determined by curriculum standards
Intent is to describe the student’s standing relative to course outcomes, at this point in time.
Grading/Reporting
AssessmentEvaluation
Proficiency & Outcomes ‘Power Outcomes’ are those critical
outcomes that demand mastery. Assessing for that takes more time and is
more difficult.
You can’t do a good job of that on a PAT … but good assessment during the year should yield concurrent results on the PAT.
Clearing the bar.. in order to ‘get’ credit for an outcome on the
PAT, you only have to answer at the difficulty level of the question.
From that, we can infer ONLY that you can also answer easier questions … for all we know, you may also be able to answer harder ones.
That fact makes ‘equating’ unsupportable; it is based on the premise that the student could not ALSO get the harder question.
Assessment for Learning Is what we used to call Formative
Assessment.
PAT’s don’t do that. One shot. End of year. No do-overs, no learning from mistakes.
But…
Grading/Reporting
AssessmentEvaluation
For kids, this isn’t necessarily a bad thing. Sometimes, our system of using tons of
accumulated marks throughout the whole year works against kids.
Purpose is everything. What are we trying to assess, what outcomes are we watching, what do we look for as evidence?
What poor assessments ‘hang around’ to mess up the final totals for the year?
How would you grade Jared? You all remember
Jared … he’s the Subway guy.
Jared lost a lot of weight by eating Subway sandwiches and walking every day.
Jared … Before and After
How would you grade Jared?
What grade would Jared get in your ‘lifestyle, nutrition, and DPA class?
(assume the ‘before’ and ‘after’ pictures coincide with the timelines for your class)
In most of our schools, Jared … would not get a great mark.
Probably, no more than 50% Our pre-occupation with calculated
‘Averages’ would ensure that at least some of his marks were low. After all, he only got to be the way he is now in the last month.
We would have to ‘average in’ all of his earlier overweight scores.
Assessment for Teaching Is another story…
This is formative assessment for us. Professional Development Program and curriculum development What can we do that impacts the
performance of our students? If you do a good job of teaching, the
achievement tests take care of themselves.