Upload
angelica-thetford
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Planning for the New Measurable Objectives: 2 Critical District Issues with Examples & Background to Help You Address Them
KASB, Topeka, 4 April 2013
Kelly Spurgeon, KSDETony Moss, KSDE
What we hope to cover today:
1. How does the new federal acct. system compare with the old?
2. How can districts best prepare for these changes?
1. An integrated district Measured Objective strategy;
2. Communication strategies for main constituency groups
3. Why do districts need an integrated MO strategy?
4. Nuts & bolts of the Measured Objectives.
Leaving the AYP Mentality:
What we had:• An overemphasis on a state
assessments and a single measure, percent proficient
• Identifying “bubble kids” and getting them over the standards line
• Sometimes teaching to identified test items, not concepts
• Counting the same kids more than once across multiple subgroups
What we will have:• 4 Measured Objectives from
assessments; 5 areas in the new accreditation model, each counting 20%
• Incentives to move every student to their highest possible proficiency level
• Constructed response items that will require deeper conceptual mastery
• One gap measure of the lowest 30%, no duplicate counting
• Overall, an emphasis on improved instruction and instructional support
The big differences:
• A much more nuanced set of measures (good schools can have different profiles)
• College and Career standards—”raise the bar”—so that students are graduating ready for college or a career.
• Why? So they can better compete in a very competitive world.
2 Critical District-Level Tasks:
1. Designing an integrated strategy for making the Measured Objectives; and
2. Developing a communication strategy for each of your 3 key constituencies:
1. Your staff
2. Your board and political stakeholders
3. Parents, the public, and the press.
Why do districts need a Measured Objectives strategy to coordinate school strategies?
• Children’s intellectual and social-emotional development is integrated and hierarchical: each stage depends on the stage that went before;
• Only districts can put all the institutional segments together, from early childhood education and care, through primary, middle, and high school, and keep them all directed toward the same integrated goals.
For example, consider the holes in children’s academic trajectories when measured only by state assessments:
All the other necessary parts—early childhood, student engagement, relationships, school climate and culture, professional development—are missing.
• Collaboration with local pediatricians, early education and child care providers, and social services to improve child development and child-rearing skills before Kindergarten;
• In primary school A, a focus on immediate teacher recognition of any student’s failure to master any competence, including behaviors, and immediate one-on-one work to move the child to competence;
• In middle school B, a focus on weekly career and academic coaching, Individual Plans of Study, and career shadowing;
• In high school C, a focus on improved social climate, social inclusion, continued Individual Plans of Study, project-based instruction, pathway completion, and higher API scores.
• District investments in high-quality, focused, 50-hour PD training.
An Overly Simple Example of a District’s Integrated MO Strategy:
Ages 0 – 5
5 to 9
10 to 14
15 to 18
Let’s now consider districts’ communication needs:
A common question you can expect:
Why all the changes? • To better prepare students for college or a
career, the bar is being raised.• AYP was obsolete and had to be replaced.
In 2015, with the higher standards in the new Kansas Career and College Ready assessments, we can expect for scores to go down substantially.
We have to explain this so it isn’t mis-interpreted.
20102012
20142016
0
100
200
300
400
500
600
700
800
900
1000
reading APImath API
New assessment introduced
Points that Might Be Included in a Staff Communication Plan:
We are in a period of rapid change in assessments and accountability:
o ESEA Waiver replacing AYPo Career pathways assessments comingo School readiness measures comingo A broader set of accreditation measures (state assessments
results only 20%) (rigor, relevance, responsive culture, relationships 20% each)
o KEEP (Kansas Educator Evaluation Project)o Blended assessment in 2014 & Kansas College and Career Ready
assessments in 2015
Staff Communication (cont.)
• The Waiver’s incentives are different from AYP’s—they reward moving every student to the highest proficiency possible;
• Constructed response items will require greater student mastery of concepts; specific test items won’t be identified.
• Constructed response items will have to be checked by staff. Educators will have a view of student proficiency across the grades when they check these items.
• Many MO’s will be re-set in 2015 & 2016 because the assessments will be new.
Points that Might Be Included in a Board or Parent Communication Plan:
• The new assessments are raising the bar, so we should expect that scores will go down.
• This decline is a result of the new assessments, not indicative of worse performance by students or teachers.
• Assessments are designed so that all students miss some items. Some assessments are more difficult than others.
• Different assessments have different plateaus or ceilings.
Example: NAEP Scores
For 17-year olds, NAEP reading scores have only varied 5 points in 42 years.
Now the details of the new MOs:
Academic Performance Index
performance level points per level
# of students total points
exemplary 1,000 262 262,000
exceeds standard 750 268 201,000
meets standard 500 268 134,000
approaching standard 250 60 15,000
academic warning 0 42 0
totals 900 612,000
Assessment Performance Index (API) = 612,000 ÷ 900 = 680
How is the API calculated?
Getting your API data from the Performance by Grade Report:
Let’s compare 2 years of Math API:
level points 2011 # of students
2012 # of students
within category% change
exemplary 1,000 237 262 +10.5
exceeds standard 750 233 268 + 15%
meets standard 500 276 268 -2.9%
approaching standard 250 90 60 -33.3%
academic warning 0 51 42 -17.6%
Result: API went from 645 to 680
Why did Kansas need a new academic performance measure?
• Not many students left below proficient .• The API rewards schools for moving all
students to higher proficiency.• It is more accurate that percent proficient.• It acknowledges the ceiling or plateau—gets
us away from 100% proficient idea.• We could use historic rates of improvement
to set realistic MO goals of improvement.
To differentiate between schools, we looked at the distribution of all schools’ API scores.
Then we divided the distribution into quarters.
Making Progress, Not Making Progress,
Title I Schools
10%Reward Schools
10%Focus Schools
5%PrioritySchools
Title (federal) & Non-Title Categories
4
3
2
1
Reading AMOs are like the Standard of Excellence:
School Category API Range Expected Rate of Improvement / AMO
Cap on % Below Standard
Modeling (Level 4)top 25 percent
API > or = 757
For schools below the 90th percentile, a mean advance of 2 points per year. Above the 90th percentile, whatever improvement is possible.
< or = 5 percent; if not, next lower level
Transitioning (Level 3)3rd quarter
API > or = 703 but < 757
An average yearly advance of 5 points per year
> 5 but < or = 10 percent; if not, next lower level
Implementing (Level 2)
2nd quarter
API > OR = 635 but < 703
An average yearly advance of 10 points per year
> 10 but < or = 15 percent; if not, next lower level
High-Need (Level 1)lowest 25 percent
API < 635
Increments sufficient to enter level 2 or a yearly mean API advance of 15 pts., whichever is greater.
Any school with > 15 percent of its students below proficient is a level 1 school.
Student Growth Percentiles imitate pediatricians’ growth charts.
Normed, percentile bands from the 5th to 95th
Girls’ Length and Weight by Age
Kansas Growth AMO:a relative measure
There are no consequences for not making a building’s growth measure.
Advantages of the Student Growth Percentile Model
SGPs set realistic yearly goals based on each student’s academic peers—students with similar score histories
SGPs map a student’s progress relative to all assessed students
How will growth MOs be displayed?
• How are 270 of my 900 students doing?
• Based on performance categories, not individuals
• No subgroups• See the Performance by
Grade Report• If the LP30 make an API
of 500, also made the gap MO
the Gap: How are the lowest performing 30 % doing?
How is the Gap calculated?
performance level points per level
# of students total points
exemplary 1,000 262 262,000
exceeds standard 750 268 201,000
meets standard 500 268 134,000
approaching standard 250 60 15,000
academic warning 0 42 0
totals 900 612,000
Assessment Performance Index (API) = 612,000 ÷ 900 = 680
900 * 0.30 = 270
performance level points per level
# of students total points
exemplary
exceeds standard
meets standard 500 168 84,000
approaching standard 250 60 15,000
academic warning 0 42 0
totals 270 99,000
API of lowest 30% = 99,000 ÷ 270 = 367
Reducing the Non-Proficient
Reduction of Non-Proficient AMOs
The goal: cut the percentage of non-proficient students (RNP) in half by 2017-2018
Custom RNPs at the district, building, and subgroup levels
Traditional subgroups with an n ≥ 30 will have an AMO determination
No monitored students in the SWDs or ELLs Only the All Students group is merged if less
than 30 (this is still under consideration)
Graduation AMOs
Participation AMOs
Resources
Video clips and fact sheets for Achievement, Growth, Reduction of Non-Proficient and Gap AMOs can be found on the ESEA flexibility waiver webpage: http://www.ksde.org/Default.aspx?tabid=5075
This page can also be accessed by clicking this logo on the KSDE home page
Questions?
Contact the Measurable Objectives Help Desk:Phone: (785) 296-2261Email: [email protected]
Slides that deal with subtopics follow:
The first big hole—early childhood—is where the largest unrealized gains are.
The success of the whole educational enterprise is even more dependent on early childhood than was previously known:
• Working memory and self controls—the ability to delay gratification and control impulses
• Students’ social predilections—empathy and considering others• Students’ behaviors (persistence, engagement, motivation)• Language skills• Some disabling conditions—antisocial behaviors, ADHD—
All have their origins in early interactions and environments.
Some Implications for Districts:
• Large improvements in K-12 education depend on overcoming fragmented services in early childhood and improvements in the quality of family environments.
• The social intelligence and responsiveness of teachers are important--sometimes as important as subject expertise.
• An overemphasis on academics and test scores, to the exclusion of developmentally important events, e.g. fantasy play in the early school years, and social integration, can be developmentally damaging.
Another consequence of ignoring early childhood: large inefficiencies are built into later district and post-secondary results:
returnper $1invested
Why did Kansas need a new academic performance measure?
Relatively few students are available for moving over the proficiency line.
Is the API more accurate than the Percent Proficient?
School T has 91% at Standard or Above, but 75% in the top 2 categories, Exceeds & Exemplary.
School P has 92% at Standard or above, but only 46% in the top 2 categories.
We step away from AYP’s 100% above standard and introduce the concept of a ceiling.
20002002
20042006
20082010
20120
100
200
300
400
500
600
700
800
900
1000
473 477 481525 555
587
618651 661
685 687 701 694
438474 485
523
580617
568621 622
655 653 674 677
reading APImath API
API Average Building Scores, All Students Group, All Public Schools, 2000 to 2012
What is a realistic rate of improvement?
Rates of improvement are larger at the beginning of a new testing cycle. They start to plateau when all of the variables in the system—alignment with standards, student and teacher skills, engagement—begin to reach their limits.
2 pts. / yr
5 pts. / yr
10 pts. / yr
15 or more
19 pts. / yr
Mathematics AMOs
School Category API Range Expected Rate of Improvement / AMO
Cap on % Below Standard
Modeling (Level 4)top 25 percent
API > or = 744
For level 4 schools below the 90th percentile, a mean advance of 2 pts. per year. Above the 90th percentile, whatever improvement is possible.
< or = 6 percent; if not, next lower level
Transitioning (Level 3)
3rd quarter
API > or = 679 but < 744
An average yearly advance of 7 points per year
> 6 but < or = 13 percent; if not, next lower level
Implementing (Level 2)
2nd quarter
API > or = 596 but < 679
An average yearly advance of 13 points per year
> 13 but < or = 19 percent; if not, next lower level
High-Need (Level 1)lowest 25 percent
API < 596
Increments sufficient to enter level 2 or a yearly mean API advance of 15 pts., whichever is greater.
Any school with > 19 percent of its students below proficient is a level 1 school.
How can our measures be made more useful?
• Please tell us what data and charts would help you communicate better with:o Your boardo Your parents o The local press ando Your teachers and staff.
• We need your feedback and suggestions
• To prompt your thinking, some examples follow
Possible Improvements within Current Assessment Reports:
• Population trends • Cohort views of the API & Growth• Grade patterns across years• Adding comparison groups of “schools like
mine” or “students like mine”• Bar graphs that show improvements across
performance levels
How about growth trends?
Will there be comparative growth measures?
What measures would better support instructional improvement?
Emotional Supportclassroom climateteacher sensitivityregard for child perspectives
Classroom Organizationbehavior management / guidanceproductivitymaximizing engagement
Instructional Supportconcept developmentquality of feedbacklanguage modeling
Classroom Assessment Scoring System, Prof. Robert Pianta, et al
What would be meaningful measures to guide reducing the gap?
• Improved early education and child care measures
• Standardized school climate measures (staff collaboration; student trust of teachers)
• Better tools for diagnostic assessments (emphasis on formative assessments)
• Tools to smoothly monitor the effectiveness of interventions