Using Data to Using Data to Improve Student Improve Student

AchievementAchievement

Summer 2006 PreschoolSummer 2006 PreschoolCSDCCSDC

OutcomesOutcomes Know why we need to look at dataKnow why we need to look at data Identify two types of testsIdentify two types of tests Understand three types of scoresUnderstand three types of scores Understand Summative & Formative Understand Summative & Formative

AssessmentsAssessments Be able to interpret Summative Be able to interpret Summative

Assessment ReportsAssessment Reports Know how to use data in instructional Know how to use data in instructional

planning for increased student planning for increased student learninglearning

OutcomesOutcomes

Understand how students are placed Understand how students are placed into High School Intensive Reading into High School Intensive Reading coursescourses

Review materials published by Great Review materials published by Great SourceSource

Align assessments included within Align assessments included within curriculum to drive instructioncurriculum to drive instruction

Develop lesson plansDevelop lesson plans

Guide to the ProgramGuide to the Program 1) Plan1) Plan

Lesson Plan BooksLesson Plan Books 2) Teach2) Teach

Teacher’s GuideTeacher’s Guide Overhead TransparenciesOverhead Transparencies Website – www.greatsource.comWebsite – www.greatsource.com

3) Practice3) Practice Student Applications BookStudent Applications Book Independent PracticeIndependent Practice WebsiteWebsite

Great Source Material Great Source Material OverviewOverview

Lesson Plan BookLesson Plan Book Teacher’s Guide (all levels)Teacher’s Guide (all levels) Student Application BookStudent Application Book Student Application Book, Teacher’s EditionStudent Application Book, Teacher’s Edition Reader’s Handbook (all levels)Reader’s Handbook (all levels) Test BookTest Book Overhead Transparencies (all levels) Overhead Transparencies (all levels) WebsiteWebsite SourcebookSourcebook Sourcebook, Teacher’s EditionSourcebook, Teacher’s Edition Sourcebook Florida Diagnostic Tests Sourcebook Florida Diagnostic Tests

(optional/available by site)(optional/available by site)

Lesson Plan BookLesson Plan Book

Gives day-by-day and week-by-week Gives day-by-day and week-by-week lesson planslesson plans

Shows how to use Shows how to use Reader’s HandbookReader’s Handbook to set-up a complete reading to set-up a complete reading curriculum curriculum

Curriculum plan suggests year-long Curriculum plan suggests year-long planplan

Individual lesson plans outline weekly Individual lesson plans outline weekly and daily lesson plansand daily lesson plans

Teacher’s GuideTeacher’s Guide

Walks through each lesson in the Walks through each lesson in the Reader’s HandbookReader’s Handbook

Highlights what to teachHighlights what to teach Suggests ways to extend the lessonSuggests ways to extend the lesson

““How to Use a Teacher’s Guide How to Use a Teacher’s Guide Lesson”Lesson”

Pages 16 – 20Pages 16 – 20

Student Application BookStudent Application BookStudent Application Book, Student Application Book,

TETE

Extends the lessons with a new Extends the lessons with a new selection for students to work throughselection for students to work through

Lessons let students apply the reading Lessons let students apply the reading strategies and tools to a new selection, strategies and tools to a new selection, give them guided practice, and help you give them guided practice, and help you assess their understandingassess their understanding

Reader’s HandbookReader’s Handbook

HANDBOOK- NOT A TEXTBOOK!HANDBOOK- NOT A TEXTBOOK! What is the purpose?What is the purpose?

Guides students as they read informational Guides students as they read informational texttext

Students should use this book to:Students should use this book to: Look up information prior to or while Look up information prior to or while

readingreading Develop new strategies to improve readingDevelop new strategies to improve reading Get to know different types of textGet to know different types of text

Test BookTest Book

Contains two types of tests for each Contains two types of tests for each topictopic

Assesses students’ understanding of Assesses students’ understanding of skills and strategiesskills and strategies

Tests can be used as diagnostic, Tests can be used as diagnostic, formative, or summative assessmentsformative, or summative assessments

Overhead TransparenciesOverhead Transparencies

Display key concepts presented in Display key concepts presented in the handbookthe handbook

48 color transparencies48 color transparencies

WebsiteWebsite

www.greatsource.comwww.greatsource.com Great Source Homepage Great Source Homepage

www.greatsource.com/rehandwww.greatsource.com/rehand Reader’s Handbook WebsiteReader’s Handbook Website

www.greatsource.com/floridawww.greatsource.com/florida Florida Diagnostic Sourcebook – Florida Diagnostic Sourcebook –

provides 9provides 9thth grade SR and ER grade SR and ER

SourcebookSourcebookSourcebook TESourcebook TE

Incorporates four approaches:Incorporates four approaches: ComprehensiveComprehensive Strategy IntensiveStrategy Intensive Literature BasedLiterature Based InteractiveInteractive

Can be used as a formative assessment Can be used as a formative assessment tooltool

Each of the 24 selections includes: Each of the 24 selections includes: Before, During, and After Reading Before, During, and After Reading strategies, vocabulary and assessmentsstrategies, vocabulary and assessments

Sourcebook Florida Sourcebook Florida Diagnostic TestsDiagnostic Tests

PretestPretest Interim Test OneInterim Test One Interim Test TwoInterim Test Two Interim Test ThreeInterim Test Three PosttestPosttest

Why Look at Data?Why Look at Data?

The purpose of data is to The purpose of data is to

give educators give educators

INSIGHT!INSIGHT!

Types of TestsTypes of Tests

Norm-Referenced Test (NRT)

Criterion-Referenced Criterion-Referenced Test Test (CRT)(CRT)

What is a Norm-What is a Norm-Referenced Test (NRT)?Referenced Test (NRT)?

A standardized A standardized assessment in which assessment in which all students all students perform under perform under the same conditionsthe same conditions

It compares the performance of a It compares the performance of a student or student or group of students to a national group of students to a national sample of students at the same grade sample of students at the same grade and age, called the norm groupand age, called the norm group

What is a Criterion-What is a Criterion-Referenced Test (CRT)?Referenced Test (CRT)?

An assessment comparing one student's An assessment comparing one student's performance to a specific learning performance to a specific learning objective or performance standardobjective or performance standard and and notnot to the performance of other students. to the performance of other students.

It tells us how well students are It tells us how well students are performing on specific goals or content performing on specific goals or content standards rather than how their standards rather than how their performance compares to a national or performance compares to a national or local norming group.local norming group.

Summary NRT and CRTSummary NRT and CRT

TYPE Norm-referenced

Test (NRT) Criterion-ref erenced

Test (CRT)

DEFINITION

Shows how a student does in relation to a norm group.

Shows how a student does in relation to a standard.

EXAMPLES

FCAT NRT SAT/ 10 TerraNova PSAT/ NMSQT

FCAT SSS Classroom Tests AP Exams District Math

Assessments

Types of ScoresTypes of Scores

Raw Score (RS)Raw Score (RS)

The number of items a student The number of items a student answers correctly on a test.answers correctly on a test.

John took a 20 item mathematics test John took a 20 item mathematics test (where each item was worth one (where each item was worth one point) and correctly answered 17 point) and correctly answered 17 items. items.

His His raw scoreraw score for this assessment is for this assessment is 17.17.

Scale Score (SS)Scale Score (SS)

Mathematically converted raw scores Mathematically converted raw scores based on level of difficulty per based on level of difficulty per question question

For FCAT-SSS, a computer program is For FCAT-SSS, a computer program is used to analyze student responses and used to analyze student responses and to compute the scale scoreto compute the scale score

Scale Scores reflect a more accurate Scale Scores reflect a more accurate picture of the student’s achievement picture of the student’s achievement levellevel

Developmental Scale Developmental Scale Score (DSS)Score (DSS)

ReadingReadingGrade Level

1Level

2Level

3Level

4Level

58 886-1695 1696-

18811882-2072

2073-2281

2282-2790

9 772-1771 1772-1971

1972-2145

2146-2297

2298-2943

10 844-1851 1852-2067

2068-2218

2219-2310

2311-3008

Current Current GradeGrade

DSSDSS

Student Student AA

99 22242224

Student Student BB

1010 18351835

Student Student CC

1111 16751675

Grade Grade EquivalenEquivalen

tt

1010thth

88thth

66thth

High School High School Intensive Reading Intensive Reading

PlacementPlacement

If If LANGUAGE!LANGUAGE! Placement Placement Assessment Assessment

indicates placement indicates placement into into LANGUAGE!LANGUAGE! Levels A/B, C/D or Levels A/B, C/D or

E/F– E/F–

2 periods2 periods

If If LANGUAGE!LANGUAGE! Placement Placement Assessment Assessment

indicates placement indicates placement beyond beyond

LANGUAGE!,LANGUAGE!,

Great Source -Great Source -

1 period1 period

Incoming L1 & L2 students on 2006 FCAT

Note: Incoming 11th & 12th graders who scored above 1926 will be able to take a 10th grade fluency assessment to be placed out of the Intensive Reading requirement..

Gain ScoresGain Scores

Commonly referred to as “Learning Gains”

The amount of progress a student makes in one school year.

Learning Gains: Who Qualifies?

All students with a pre- and post-test, including all subgroups (ESE, LEP, etc.).

All students with matched, consecutive year (i.e. 2005 & 2006) FCAT SSS results, grades 4-10, who were enrolled in the same school surveys 2 & 3 (FTE).

Learning Gains: Which Scores?

Gains apply in reading and math, not writing or science.

Pre-test may be from same school, same district, or anywhere in the state.

Learning Gains:What equals Adequate

Yearly Progress (AYP)? A. Improve FCAT Achievement Levels from 2005

to 2006 (e.g. 1-2, 2-3, 3-4, 4-5)

OR

B. Maintain “satisfactory” Achievement Levels from 2005-2006 (e.g. 3-3, 4-4, 5-5)

OR

C. Demonstrate more than one year’s growth within Level 1 or Level 2 - determined by DSS Cut Points (not applicable for retained students)

Developmental Scale Score

Gains Table (DSS Cut Points)Students achieving within Level 1 (or within Level 2) for two consecutive years

must gain at least one point more than those listed in the table in order to satisfy the “making annual learning gains” component of the school accountability system.

Grade Level Change Reading Mathematics

3 to 4 230 162

4 to 5 166 119

5 to 6 133 95

6 to 7 110 78

7 to 8 92 64

8 to 9 77 54

9 to 10 77 48

Learning Gains: Retainees

A retained student can only be counted as making adequate

progress if he/she:

Moves up one level. (e.g. 1-2, 2-3, 3-4, 4-5)

Maintains a level 3, 4, or 5.

Learning Gains: Activity

Using the data on the following table, determine:

which students made a learning gain

what percentage of the teacher’s students made a learning gain

Data Display for Data Display for FCAT Reading ResultsFCAT Reading Results

Student 04/05 Grade Level

05/06Grade Level

Pre-test Achievement Level

Pre-test DSS

Post-test Achievement Level

Post-test DSS

Learning Gain Determination

A 7 8 Level 1 Level 2 Yes or NoReason: A, B, or C

B 7 8 Level 4 Level 4 Yes or NoReason: A, B, or C

C 7 8 Level 2 1598 Level 2 1743 Yes or NoReason: A, B, or C

D 8 8 Level 1 Level 2 Yes or NoReason: A, B, or C

E 8 8 Level 3 Level 3 Yes or NoReason: A, B, or C

F 8 8 Level 1 1486 Level 1 1653 Yes or NoReason: A, B, or C

G 7 8 Level 5 Level 4 Yes or NoReason: A, B, or C

Teacher Learning Gains Based on Data Display

5 out of 7 students made learning gains. 71% of this teacher’s students made learning gains

and add points towards the school’s grade. No points are given to the school for Student F

because he was retained and stayed within level 1 – even though he made significant gains in DSS points.

No points are given to Student G because he decreased a level.

Total Number of Students with a Pre and Post-test who qualify for learning gain calculations:

Reason AIncreased 1 or more Achievement Levels

Reason BMaintains “satisfactory” levels (3, 4, or 5)

Reason CDSS Target Gain (More than a year’s growth)

7 2 2 1

Class Record Sheet Class Record Sheet for for

Learning GainsLearning Gains

Microsoft Word Document

Types of DataTypes of Data

Results (Summative)

Data used to make decisions about student achievement at the end of a period of instruction.

Process (Formative)

Data gathered at regular intervals during the instructional period; used to provide feedback about student progress and to provide direction for instructional interventions.

Summative DataSummative DataContinuedContinued

FCAT FCAT Great SourceGreat Source

Florida Diagnostic Tests (Sourcebook)Florida Diagnostic Tests (Sourcebook) Post TestPost Test

Reader’s Handbook Test BookReader’s Handbook Test Book Multiple Choice/Short Answer test pages for Multiple Choice/Short Answer test pages for

each topic.each topic. Multiple topic tests can be combined in order Multiple topic tests can be combined in order

to create a semester exam.to create a semester exam. Teacher’s Guide Source BookTeacher’s Guide Source Book

Fourth black-line master page of each story.Fourth black-line master page of each story.

A Closer Look at Results A Closer Look at Results DataData

Examples:

FCAT Parent Report

Group ActivityGroup ActivityHow do parents get these reports?How do parents get these reports?

Did this student pass the 10Did this student pass the 10thth grade test? grade test?

In what grade did this student first achieve In what grade did this student first achieve grade level mastery?grade level mastery?

Which content area had the most questions?Which content area had the most questions?

Using the bar graph, how does this student’s Using the bar graph, how does this student’s achievement compare to grade level?achievement compare to grade level?

A Closer Look at A Closer Look at Formative DataFormative Data

Quizzes

Chapter Tests

DIBELS

District Math Assessments

Formative Data ExamplesFormative Data ExamplesGreat SourceGreat Source

DefinitionDefinition: : Data gathered at regular intervals Data gathered at regular intervals during the instructional period; used to provide during the instructional period; used to provide feedback about student progress and to provide feedback about student progress and to provide direction for instructional interventions.direction for instructional interventions.

Florida Diagnostic Tests (Sourcebook)Florida Diagnostic Tests (Sourcebook) Interim TestsInterim Tests

Reader’s HandbookReader’s Handbook Student Application BookStudent Application Book Lesson Plan BookLesson Plan Book

SourcebookSourcebook Teacher’s GuideTeacher’s Guide Student TextStudent Text

What tools do we have?What tools do we have?

FCAT Inquiry (Summative)FCAT Inquiry (Summative) Teacher Tools for Data Collection Teacher Tools for Data Collection

(Can be Summative or Formative)(Can be Summative or Formative)

•Histogram•Pareto Chart•Run Chart•Scatter Diagram•Item Analysis

HistogramHistogram

Bar chart representing a frequency distribution

of student scores

Heights of the bars represent

number of students scoring

at same level/score

Used to Monitor progress

Histogram: Mastery of FCAT Subtest Content Categories

0

2

4

6

8

10

12

14

16

F1 F2 F3 F4 FA FBSubtests

Nu

mb

er o

f S

tud

ents

Using Data Inquiry to Using Data Inquiry to Determine MasteryDetermine Mastery

FCAT Data InquiryFCAT Data Inquiry

Histogram: Grade Distribution on Final Exam (Summative Test) 9th Grade Reader’s Handbook Classes

0

10

20

30

40

50

60

70

0-10 11-20 21-30 31-40 41-50 51-60 61-70 71-80 81-90 91-100Grade

Fre

qu

ency

Run Chart

Use to:

Monitor progress over time

Display data in simplest form

Run Chart: Words Correct Per Minute on Weekly Fluency Test

100

110

120

130

140

150

160

170

180

1 2 3 4 5 6 7 8 9Week

Nu

mb

er o

f w

ord

s

Class Goal: By the end of 9 weeks, 100% of our class will have an average

of at least 75% on our weekly benchmark reading passages.

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8 9Week

Per

cen

t w

/ avg

. of

at le

as75

%

Class Run Chart: Percent of Students Averaging at Least 75%

Scatter DiagramScatter Diagram

Scatter Diagram: Fluency Rates vs. Gain Points

50

60

70

80

90

100

78 88 98 108 118Gain Points

WC

PM

Scatter Diagram: Number of Siblings vs Grade Point Average

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

0 1 2 3 4 5 6

Number of Siblings

Gra

de

Po

int

Ave

rag

e

Scatter Diagram: Days of Attendance vs # of Mistakes on Sourcebook Diagnostic Posttest

0

3

6

9

12

15

18

0 18 36 54 72 90 108 126 144 162

Days of Attendance

Mis

take

s

180

Item Analysis

Use to:

Determine mastered content

Determine most common mistakes

Classroom Test AnalysisBENCHMARK

ASSESSEDITEM

#NUMBER CORRECT

NUMBERINCORRECT

NUMBERPARTIALCREDIT

NUMBERDISTRACTORA/1

NUMBERDISTRACTORB/2

NUMBERDISTRACTORC/3

NUMBERDISTRACTORD/4

NUMBERNOANSWER

1

2

3

4

5

6

7

8

9

10

Additional Additional Item Analysis ChecklistItem Analysis Checklist

Item Analysis ChecklistItem Analysis Checklist

Classroom Test Analysis

Student q1 q2 q3 q4 q5 q6 q7 q8 q9 q10 q11 q12 q13 q14 q15 q16 q17 q18 q19 q20

1 B C B B D C A A B C D C C A B C A D A D 2 B A A B D A B B A A D A C A B A A D A D 3 B C A B A B D A C A D C C A B C A D A D 4 A C A B D C A A A B D C C B C C A D A D 5 B C B B D A A A B A D C C A C C A C A A 6 B B A B C A D D D B A C C A B B A D A D 7 B B A A D C C D A C D C C C B C A D A D 8 B C A A D C C A C D D C C A B C A D A D 9 B C C B D C D A B D D C C A B C A A A D 10 B D A B B D D A A A D C C A B C A D A C 11 C C A B D B A C B C D B C B B D A D A D 12 B C A B D A C C B B D C C A A C A D A D 13 B A C B D C D A C B D C C A B C A D A D 14 B C A B D C B A A C D C C A B C A B A D 15 B B A B A B A B D A D C C A B C A D A A 16 A C A B D B C A A C D C C A B C A D A D 17 B B B B D C D C C D D C C A B C A D A D 18 B C A B D A B A A B D C C A B B A D A D 19 B C A B D A D A B A D C C A B C A D A D 20 B C A B A C B A C B D C C D B C A D A D

Correct Answer

B C A B D C D A A C D C C A B C A D A D

Questions 1-5 Introduction Questions 6-10 Reading Process Questions 11-15 Reading Know-How Questions 16-20 Improving Vocabulary

Item Analysis Activity

Pareto ChartUse to:

Rank issues in order of occurrence

Decide which problems need to be addressed first

Find the issues that have the greatest impact

Monitor impact of changes

Pareto Chart: Introduction, Reading Process, Reading Know-How, Improving Vocabulary

0

10

20

30

40

50

60

70

80

90

100

Reading Process Introduction Reading Know-How

Improving Vocabulary

Category

Pe

rce

nt

0

10

20

30

40

50

60

70

80

90

100

Cu

mu

lati

ve p

erc

en

tag

e

Data analysis provides:

Insight

and

Questions

Adapted from Getting Excited About Data, Edie Holcombwww.corwinpress.com

• What question are we trying to

answer?• What can we tell from the data?• What can we NOT tell from the

data? What else might we want to

know?• What good news is here for us to

celebrate?• What opportunities for improvement

are suggested by the data?

Questions to Ponder…

Action

Answers!

Provides

What information have I gained from my

data? What interventions can I put

In place?

Implement the plan.

Analyze the results.

Make improvements.

Steps to Improvement

PLAN

DO

STUDY

ACT

Personal Action Plan

What data can I access?

What tools can I use to help me monitor progress toward our class goals?

What/who else do I need to help me?

What is my start date?

How will I evaluate the results?

P

DS

A

Lesson PlanningLesson Planning

2 approaches can be used when 2 approaches can be used when planning your Great Source Reader’s planning your Great Source Reader’s Handbook lessons. You can organize Handbook lessons. You can organize your year by using:your year by using: The Teacher’s Guide and Lesson Plan bThe Teacher’s Guide and Lesson Plan b

ook ook Sunshine State Standards BenchmarksSunshine State Standards Benchmarks