Accountability: Are You Aware? Kim Gilson Senior Consultant Data and Accountability Region 10 ESC...

Accountability: Are You Aware?

Kim Gilson

Senior Consultant

Data and Accountability

Region 10 ESC

Kim.gilson@region10.org

972-348-1480

Index 1: Who PassedIndex 2: Who Grew

• Accountability Reports Tell You:• Performance by Subject• Performance by Student Group

• Accountability Reports DON’T Tell You• Performance by Grade Level, Course,

Teacher…• Too many other things to list!

Sample State ReportsIndex 1

Index 2

Sample Safeguards Report

Student Groups and Minimum Size

• On State Accountability Reports, Student Group Performance is reported on Page 2 of Index 2 as Safeguards• Data is aggregated by subject across the

campus/district• For weak student group performance in

safeguards, break it down by grade level• Where are those students this year?

• Minimum size of 25 means potentially a handful at each grade level, resulting in campus shifts as students progress from one grade to the next

5th 6th 7th 8th

Student Group 2014

9 10 9 4

Student Group 2015

9 10 9

23

28

Phase-In Custom Report

How to Build It

Student Progress Measure Tip

State Report• Combines Students Who

Meet OR Exceeded Growth in the Calculation for 1 Point• Example: If 60% of

students Meet Progress and 30% Exceed Progress, then you will see this:

Aware Report• Separates Students Who

Meet Growth From those Who Exceed• Same Example: 60%

Students meet and 30% Exceed, then Aware will report this:

% Who Met Or Exceeded

90

% Who Exceeded 30

Weighted Score 120

% Met % Exceeds

60 30

Double the Exceeds Score to Give them Their 2nd Point in Index 2

Growth Measure Rule

• Students must test 2 consecutive years on the same version of the test• Example: 2013 STAAR M 2014

STAAR• No Growth Measure can

be calculated• Any student who took a STAAR M last

year will not have a Growth Measure this year

Phase-In Custom Report

Drill Down By Grade and Subject

• District View

How did each grade/subject contribute• Campus View

How did each campus contribute to each grade/subject• Course View

How does performance compare in an intervention course to regular to PAP/AP?

• Teacher View

How do students perform in various teachers’ classrooms?

Drilling Down Continued

• Reports from the Pull-Down Menu:• Reporting Category• Learning Standard• Individual Responses

Important!

• Regardless of what indicators you’re analyzing, Compare Student Group performance to ALL STUDENTS: Is it a “some students” issue or an “everyone” issue?

Other Interesting Views

• Learning Standard Description

Writing Composition Scores

How it’s Built

Other Tidbits…

• Use of the following Student Groups might help in PBMAS and Program Evaluation:

Index 3

• Index 3 Awards: • 1 Point for Students who Meet the

Standard• Index 1, Student Groups Reported

on Safeguards• 2 Points for Students Who Meet

Advanced Level III• Evaluated for Econ Dis and Up to 2

Student Groups• Greatest Opportunity to Raise Index 3?

• Advanced Level III Numerator!!!

Advanced Level III

• Use Your Aware Reports to Evaluate Advanced Level III Performance• Drill Down at Various Levels

• Campus, Grade, Course, Teacher, Reporting Category, Learning Standard, Individual Responses

• Compare Regular to Pre-AP Courses!!!

CRITICAL!!!

• State Reports DON’T Tell The Performance of ALL STUDENTS in Index 3.• Is it a “Some Students” Issue or an “All

Students” Issue?• Do Your 3 Student Groups Represent ALL

Students, or Only a Portion?

What Exactly Does Advanced Level III Mean?

• Students must score in the top 5-6 Raw Scores to Earn Advanced Level III

• Guiding Question:• Can my students successfully answer

the 5-6 most difficult questions on the test?

• We know how many questions the top performers got right, we need to ask….”Which ones did they miss????”

Analyzing Test Items For Students Scoring 70 or Greater

2 students who scored >70 on the test missed this question

Both chose answer C (incorrect)

These 2 students averaged a 70% on the test

How to Build It…

Index 4

• 4 Components• Graduation Rate• Diploma Plan• Post Secondary Indicators:

• ACT• SAT• TAKS (Compare 2013 College

Readiness to 2012)• FINAL LEVEL 2 IN 2+ Subjects!

Sample Index 4 Report for Post Secondary Readiness

This doesn’t tell you:Which Grades?Which Subjects?

How Can the Data Help Me?

• Begin thinking in terms of “How many questions can a student MISS and still earn Final Level 2?”

How Many Can I Miss?

ItemsPhase 1

Number You Can Miss

Phase 2 Number You

Can Miss

Phase 3 Number You

Can Miss

Final Level 2 Number You Can

Miss

Advanced Level 3 Number You Can Miss

Grade 3 40 19 16 12 10 5

Grade 4 44 21 17 13 10 6

Grade 5 46 21 18 14 11 6

Grade 6 48 23 19 14 12 6

Grade 7 50 24 20 16 13 8

Grade 8 52 26 22 17 14 8

English I 92 43 40 36 34 16

English II 93 43 40 37 34 14

Grade 5 44 18 15 11 9 4

Grade 8 54 24 21 17 14 8

Biology 54 34 30 25 21 9

Grade 8 52 26 22 17 14 9

U. S. History 68 40 35 29 24 13

Grade 4 44 21 19 16 14 7

Grade 7 72 33 29 24 21 11

STAAR Raw Score Conversion

Subject

Use the 70% Report to Identify the Hardest Questions on the Test!

• Have your students been challenged at that level consistently in class?

Conclusion

• The State Accountability System has many and varied components. Aware can help you understand the stories behind the numbers to positively impact 2015 ratings.

Questions? Contact Me!

Kim Gilson

Kim.gilson@region10.org

972-348-1480