PROGRESS MONITORING
FOR DATA-BASED DECISIONS
June 27, 2007 FSSM Summer Institute
2
Progress Monitoring: Learning Goals for Today
• Determine student’s current level of performance (Baseline data) CBM
• Identify learning goal (norms) • Implement Research-Based Interventions• Continue to measure and monitor students’
performance on a regular basis• Graph the results• Compare expected progress to actual rate• Adjust instruction based on the data
3
What IS Progress Monitoring
Find the in your handout packet.
Work with your TEAM to complete this.
Be ready to share a specific example of Progress Monitoring in your school.
Handout PM 1
4
Progress Monitoring
is a scientifically-based practice that is used to assess students’ academic performance and evaluate the effectiveness of instruction – for an individual student or an entire class.”
National Center on Progress Monitoring
www.studentprogress.org
5
A Scientific Base Supports One Form of Progress Monitoring:
Curriculum-Based Measurement
CBM
6
Curriculum-Based Measurement (CBM) . . .
• Result of 20-30 years of research
• Used in schools across the country
• Uses short “probes” for frequent PM
• Demonstrates strong reliability and validity
• Is sensitive to small gains in progress
• Helps teachers plan instruction
7
Uses of CBMBenchmark
All Students – 3 times a year – F, W, SStrategic Monitor
Monthly check up for students with moderate skill deficits who are receiving supplemental instruction in small group (3-5)
Progress MonitorMonitor once or twice weekly to measure student’s response to more intensive interventions (individual or group of 2). Graph the data for on-going decision making.
8
2006
2007
School Calendar Year (2006-2007):
Benchmarking
(Tier 1)
2-weeks during:September 1 to October 15
January 1 to February 1
May 1 to June 1
9
For Teachers: Classroom ReportBenchmark Data
AIMSweb.com
10
10th %ile10th %ile
90th %ile90th %ile
75th %ile75th %ile
50th %ile50th %ile
25th %ile25th %ile
Student is above the90 %ile and is well above average.
Student is above the90 %ile and is well above average.
TargetTarget
____________________________________Box and Whisker Charts
AIMSweb
11
AIMS web / Harcourt w/ permission 6/07
12
Decision Making RulesStudents who score below the 25th percentile on
general outcome benchmark screening receive targeted intervention, and progress is monitored
on a monthly basis. Progress for those below the 10th percentile is monitored at least weekly, and they receive intensive intervention.
Eight data points over at least 4 weeks are required to determine a trend line.
Change or modify the intervention when the data points are below the aim line (goal line) for 4 consecutive data points, or when the trend line is not on course to meet the goal line.
13AIMSweb/Harcourt w/ permission 6/07
14AIMSweb/Harcourt w/ permission 6/07
15AIMSweb/Harcourt w/ permission 6/07
16
How Does Student Progress Monitoring Work?
• Determine student’s current level of performance (Baseline data)
• Identify learning goal (local norms or accepted research standards)
• Use research based intervention(s) targeting the problem
• Continue to measure performance on a regular basis (CBM probes at same level)
• Graph the results• Compare expected progress to actual rate• Adjust instruction based on the data
17
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Weeks of Primary Prevention
WIF
: C
orr
ec
tly
Re
ad
Wo
rds
Pe
r M
inu
teCalculating Slope: First draw a
Trend Line – Tukey MethodStep 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.)
Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.”
Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement.
www.studentprogress.orgPM 2
18
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Weeks of Primary Prevention
WIF
: C
orr
ec
tly
Re
ad
Wo
rds
Pe
r M
inu
te
Step 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.)
Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.”
Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement.
19
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Weeks of Primary Prevention
WIF
: C
orr
ec
tly
Re
ad
Wo
rds
Pe
r M
inu
te
Step 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.)
Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.”
Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement.
X
X
20
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Weeks of Primary Prevention
WIF
: C
orr
ec
tly
Re
ad
Wo
rds
Pe
r M
inu
te
Step 1: Divide the data points into three equal sections by drawing two vertical lines. (If the points divide unevenly, group them approximately.)
Step 2: In the first and third sections, find the median data-point and median instructional week. Locate the place on the graph where the two values intersect and mark with an “X.”
Step 3: Draw a line through the two Xs, extending to the margins of the graph. This represents the trend-line or line of improvement.
X
X
21
Baseline Data & GoalTOM
• Second grade student, fall assessment, oral reading fluency, second grade probes
• Baseline Data: day 1 =11, day 2 = 13, day 3 =12
• Plot these points on your graph
PM 3
22
Baseline Data & Goal Line
• Baseline Data: 11, 13, 12
• Find the median, or middle number when numbers are rank ordered: 11, 12, 13
• Median = 12 words read correctly per min.
• Average Peer = ? Use norms table, Handout PM 4
• Rate of Improvement = ? Words per wk.
• Use norms table, Handout PM 4
23
Baseline Data & Goal
• Baseline Data: 11, 13, 12
• Median = 12 words read correctly per min.
• Average Peer = 55 wrc per minute
• Average Rate of Improvement = 1.1 words per week.
• Set a realistic but ambitious goal, such as:
• Goal = 2 words per minute more per week
24
Baseline Data & Goal• Goal = 2 words per minute more per week• Select # wks for monitoring: Weeks =12• Calculate: 2 wds./wk. increase, times 12
wks. = 24 words total increase• Add the expected 24 word increase to
baseline (12). Goal= 36 wrc in 12 weeks• Mark the goal point (36) on the last data
collection day, Thursday, at the end of 12 weeks of intervention.
• Connect baseline data median(12) to the goal point (36) = Goal Line (Aim Line)
25
GOAL LINETom's Oral Reading Fluency
0
5
10
15
20
25
30
35
40
Sep-06 Oct-06 Nov-06 Dec-06
Dates
WRC
26
Intervention Data Points
9/16/2006 14
9/18/2006 15
9/23/2006 13
9/25/2006 17
9/30/2006 16
10/2/2006 15
10/7/2006 16
10/9/2006 15
Enter this data on your graph and draw a trend line.
27
Tom's Oral Reading Fluency
0
5
10
15
20
25
30
35
40
9/5 9/15 9/25 10/5 10/15 10/25 11/4 11/14 11/24 12/4 12/14
dates
wrc
Goal Line
Intervention 1 data pts
Linear (Intervention 1 data pts)
28
Data-Based Decision Making
• Were there 4 consecutive data points below the goal line?
• Are there 8 data points for a trend line?
• Is the trend line on course to meet the goal line?
• Is this intervention effective?
• Decision?
29
Intervention #2 Data Points10/14/2006 17
10/16/2006 19
10/21/2006 21
10/23/2006 26
10/28/2006 28
10/30/2006 26
11/4/2006 30
11/6/2006 32
Enter this data on your graph and draw a trend line.
30
Tom's Oral Reading Fluency
0
5
10
15
20
25
30
35
40
9/5 9/15 9/25 10/5 10/15 10/25 11/4 11/14 11/24 12/4 12/14
dates
wrc Goal Line
Intervention 1 data pts
Intervention 2 data pts.
Linear (Intervention 1 data pts)
Linear (Intervention 2 data pts.)
Intervention 1Intervention 2
31
Data-Based Decision Making• Draw a vertical line to show each
intervention change.
• Label “Intervention 1”, “Intervention 2”, etc.
• Keep records of specific intervention protocols and fidelity of implementation.
• What is your decision based on Intervention 2 data?
32
Why is Fidelity Important?
• To demonstrate that measurable changes in behavior are related to systematic & controlled changes in the environment (intervention)
• Without objective & documented evidence that the intervention was implemented as planned, we can’t conclude that inadequate response to intervention was due to a poor intervention or insufficient intensity (ie: inadequate response may be due to poor instruction.)
• Likewise, success can’t be attributed to the intervention if we don’t know how it was implemented
33
3rd Grade Winter R-CBM – Mia - Baseline
Tues. 34 WRC Wed. 40 Thurs. 36
Enter the data.
Mark MEDIAN with X PM 5
34
Graphing: Make Mia’s Goal Line
Set a Reasonable but Ambitious Goal
What is the average Rate of Improvement (ROI) for third grade? – norms – PM 4
Would a gain of 15 words per week be ambitious? Reasonable?
Would a gain of 2 words per week be ambitions? Reasonable?
Calculate Mia’s goal at the end of 12 weeks.
Graph the GOAL LINE (Aim Line)
35
Mia’s Problem Statement
“When given a third grade oral reading fluency probe, Mia reads 36 words correctly in one minute. Her average third grade peer reads 98 words correctly in one minute.”
Intervention: Repeated Readings 2:30 – 2:50 M/W/F
Implemented by: Classroom teacherProgress Monitor: T/Th R-CBM (3rd)
36
Mia’s Progress Monitoring Data
Mia’s R-CBM scores (Tues./Thurs.)
Week 1: 34, 36Week 2: 42, 36Week 3: 36, 38Week 4: 38, 38
Graph the data.
PM 5
37
Make a Trend Line
Use Mia’s data: Weeks 1-4
(Don’t include the baseline data)
Use the Tukey Method to draw a trend line.
Compare Mia’s Trend Line & Goal Line.
What’s your decision about the intervention?
38
To Monitor Student Progress• Determine student’s current level (Baseline)• Identify learning Goal (local/research norms)• Research based Intervention(s) target
problem• Implement with Fidelity• Continue to measure performance on a
regular basis (CBM probes at same level)• Graph the results• Compare expected progress to actual rate• Adjust instruction based on the data
39
Where Do We Go From Here? RtI / SLD Eligibility Determination?
Students who score below the 25th percentile on general outcome benchmark screening receive targeted intervention, and progress is monitored on a monthly basis. Progress for those below the 10th percentile is monitored at least weekly, and they receive intensive intervention.
Eight data points over at least 4 weeks are required to determine a trend line.
Change or modify the intervention when the data points are below the aim line (goal line) for 4 consecutive data points or when the trend line is not on course to meet the goal line. At least 2 intervention changes (ie: 3 interventions) are required before students may be referred for evaluation due to suspected disability.
40
Special Education Regulations Apply at Time of Referral
• Letter to parent informing of referral• Evaluation decisions made by the Evaluation
Team at the Review of Existing Data/Evaluation Plan Meeting.
• Parents are given written notice and must sign consent for the evaluation. “Specialized Instruction” is listed under “other” in the area of functioning (Ex: “Specialized Instruction, 225 minutes per week, special ed resource room.”)
• Specialized instruction must be very focused and targeted at the problem as operationally defined in the problem statement.
41
• CBM is used to collect data in all academic areas, administered frequently (2X/wk)
• The Evaluation must be comprehensive and address all areas of concern.
• Consider norm referenced achievement assessment in addition to CBM (if not indicated in the student’s records). Level of performance must be far below & rate of improvement far below average peers.
• Use cognitive assessment if broad & pervasive concerns with student’s functioning across areas.
• However, do not calculate a discrepancy between IQ and norm referenced achievement scores. This information is not relevant to determining eligibility or interventions.
See Handouts PM 7 & PM 8 for more specific procedues.
42
Progress Monitoring & IEP Goals
Formula for Good Goals Using CBMGiven a (specific grade level and subject
area) probe,STUDENT will (increase/decrease/maintain)
his/her ability to (state skill addressed) by (observable behavior)
from (insert baseline)to (insert goal)
for (insert monitoring period)
43
References• Deno, S. (2003). Developments in Curriculum-
Based Measurement. Journal of Special Education (37) (3), 184-192.
• Fuchs, L. & Fuchs, D. (2002). Curriculum-based measurement: Describing competence, enhancing outcomes, evaluating treatment effects, and identifying treatment nonresponders. Peabody Journal of Education, 77, 64-84.
• Hosp, M. & Hosp, J. (2003). Curriculum-based measurement for reading, math, and spelling: How to do it and why. Preventing School Failure,
48 (1), 10-17.