Data Collection and Analysis. Baseline Data Whenever possible collect baseline data. –This is most important for behavior programs. –Also very useful

Data Collection and Analysis

Baseline Data• Whenever possible collect baseline data.

– This is most important for behavior programs.– Also very useful for academic programs when

student knows some of the skill but is not proficient.

– If student has never demonstrated skill it is not necessary to collect baseline - we know it is 0

• Having baseline data gives greater evidence that intervention/instruction is responsible for the behavior change.– Collect several days of data when possible.

Baseline DataPreschool Student

0

1

2

3

4

5

6

2/19/06 2/26/06 3/5/06 3/12/06 3/19/06 3/26/06

Frequency

Incidents of DisruptiveBehaviorBreaks

Instruction on Requesting a Break

Frequency of Data Collection

• Once or twice per week data collection is sufficient for most academic skills.– Don’t collect more data than you need or

can use.– Determine data collection schedule prior to

start of day or week (e.g., collect data during workbox activities on Wednesday).

• Ensures that student behavior on a specific day does not influence data collection.


• Data collected should be representative of overall student behavior not just when student is demonstrating challenging behavior or exceptionally positive behavior.– If practicable, collect data on each

significant incident of challenging behavior.• Frequency data


• Collect data on instructional trials as often as possible. – Many students will make progress rapidly

frequent data collection will show this progress and lead to a more responsive instructional program.


• When teaching specific skills such as, using a schedule it is best practice to collect data on each trial.– Generally these are task analyzed skills

and taught at relatively low rates (2-5 times a day).

– Graph data on trials in which student was able to complete skill independently.

Event Recording or Frequency

• A tally or count of discrete behaviors as they occur.Examples: dropping to floor, physical aggression,

words spoken, talk outs, directions followed.– Behavior should be discrete, having a definite

beginning and end.– Behavior should not occur at very high rates.– When expressed in a ratio with time it is a

measure of rate.

Rate

• Rate - number of occurrences in a specific period of time.

• Rate is useful measure of fluency or how quickly a student can perform a task.– Most common rate measure is words read per

minute.– Rate is also a useful measure in addition to

accuracy for workbox activities.– Fluency and rate are important for future

employment of many of our students.

Rate

• Assembly or sorting tasks– Collect data on the number of items assembled or

sorted correctly out of items possible.• Provides measure of accuracy.

– Collect data on the time it took to complete task.• Provides a measure of rate.• Many students can be taught to start timer at

beginning of task and stop timer at end of task.

Accuracy

• Percentage of correct responses.Example: – Number of trials in which the student

correctly performed the behavior divided by number of opportunities.

– Very common and easy measure to use.• Can also be used when collecting data on

behaviors that can be broken into steps (task analyzed).

• Independent responding is considered a correct response.

Accuracy

• Be careful when comparing percentages if opportunities for behavior are not the same. – Use percentages or accuracy only to

compare same number of opportunities over time.

– 50% is really not the same if it is a comparison of 1 of 2 trials vs 10 of 20 trials.

Rate and Accuracy

• Flashlight assembly– At end of task 5 of 6 flashlights assembled

correctly and work properly.– Task took student 5 minutes.

Accuracy: 5 assembled correctly out of 6 5/6 = 83%

Rate: 5 completed correctly in 5 minutes

5/5 = 1 per minute

Rate and Accuracy

• Workbox sorting task– Sorting by color (red, blue, green, yellow,

purple)• 5 of each color, 25 possible correct responses• Student places 20 of 25 correctly = 80% correct• Student completes task in 5 minutes - 20/5 =

rate of 4 per minute (20 done correctly).• Rate could also be calculated as overall items

in 5 minutes (25/5 = rate of 5 per minute.)

Rate and Accuracy

• Reading fluency - WRC and Errors is also a measure of rate and accuracy.– One of the most common measures used

in education.

Duration

• The extent of time a behavior occurs during an observation period.Examples: – Length of time student could wait.– Length of time student worked on

independent task.– This measure shows how long a student

engages in a particular behavior.

Duration Per Occurrence• The extent of time of each occurrence of the

behavior.Examples: – Length of time student was able to work at

independent task.– Length of time student could wait in lunch line.– This measure provides data on the number of

occurrences, duration of each occurrence, and total duration.

– Provides more information about the target behavior than total duration does.

Level of Assistance

• Most often used for students that need differing levels of prompts. Provides record of how much external assistance was needed.Examples: full physical prompt, partial

physical prompt, verbal prompt.– Useful descriptive information.– Graph only behaviors/steps that student

can perform independently.

Time Sampling

• Momentary Time Sampling - Behavior immediately following specified time period is recorded. Example: on-task behavior.– It is easy to use and will not interfere with other

ongoing activities of the observer. This method can be used easily for either individual or group behavior.

– Excellent for behaviors that are not discrete.• Socializing/playing at recess• See attached data sheets

Time Sampling

• Variety of time intervals can be used. – 10 second; 5 - 15 minute– 1 minute; 15 minutes– 5 minute; 25 minutes– Use a consistent measure for

comparisons.• Can be presented in a bar graph to

demonstrate a discrepancy in comparison to peers.

Data Collection Considerations

• What type of data will be collected?– Dictated by the goals and objectives or the

ultimate purpose of the intervention.• Reduction in challenging behavior, increase in

independent responding etc.

– Be cognizant of data collection when writing goals.

• When writing the goal or objective ask: – Is this something that can be reliably measured? – Is it observable and quantifiable?– Can I measure this in reasonably efficient

manner?

Data Collection Examples

• Schedules are used to increase independent transitioning.Collect data on:– Student’s progress in learning steps to using

schedule independently.– Frequency of student independently transitioning

from one activity to another.


• Schedules are used to decrease challenging behaviors at transitionCollect data on:– Frequency of challenging behaviors at

transitions prior to and after schedule is implemented.

• Baseline data should be collected.

– Remember to collect data on student’s progress in using skill independently.


• Workbox system is used to develop or increase ability to work independently.Collect data on:– Student’s ability to complete tasks

independently.• Number of tasks completed.• Duration of time student worked independently.• Number of prompts needed.


• Workbox activities are used to maintain skills.

Collect data on:

– Rate and accuracy of task completion.• Number of items completed correctly.• Speed at which task was completed.


• Social Skills• Following instructions, accepting

corrective feedback, etc.– Common to use point sheet to collect data

on behavior goals.• Point sheets, at best, provide general indictor of

skill use.

Data Collection Examples• Social Skills

– Data can then be collected on the the number of times the student demonstrates the the individual skill steps vs the number of opportunities.

Accepting corrective feedback___will increase his ability to accept corrective feedback from refusing to accept feedback on 7 of 10 trials and raising his voice, arguing, screaming, hitting, kicking, or throwing things to calmly accepting feedback on 8 of 10 trials through the use of social skill instruction, visual supports, praise, positive and corrective feedback, logical consequences, and other behavior management techniques.

Data Collection Examples• Count number of times student

accepted feedback divided by opportunities (number of times feedback was given).– Provides a concrete measure of actual skill

use (obtain a measure of accuracy).– Number of times correct = 2, Opportunities = 3

• Student correctly responded on 2 of 3 trials during this observation.

• Next observation do the same and add up total trials and total correct (2/2)

• Total for two observations 4/5

Data Collection Examples• The same procedure can be used to

collect data on social skill objectives.

– Collecting data on scaffolded objectives is simple and easy.

– Data collection can be part of instruction and done at time of instruction.

– Demonstrates that student knows the steps to the skill and how to use skill even if he/she is not yet using the skill in the natural environment.


• 1: When asked, ____ will verbally identify the steps to accepting feedback or a consequence (look in the direction of the person, say "okay," and don’t argue) with 100% accuracy for 3 consecutive trials as measured by teacher observation.

• 2: In a role-play, ____ will practice using the steps to accepting feedback or a consequence (look in the general direction of the person, say "okay," and don’t argue) with 100% accuracy for 3 consecutive trials as measured by teacher observation.


• 3: When given corrective feedback or a consequence ____ will increase his behavior of accepting feedback by using the steps to accepting feedback 6 of 10 opportunities of the time as measured by periodic observation.

• 4: When given corrective feedback or a consequence ____ will increase his behavior of accepting feedback by by using the steps to accepting feedback 8 of 10 opportunities as measured by periodic observation.


• ______ will increase positive response to directions from following 4 of 10 instructions and arguing, hitting, kicking, throwing objects or leaving the area to promptly following 8 of 10 directions through the use of social skill instruction, visual supports, praise, positive and corrective feedback, logical consequences, and other behavior management techniques.

Data Collection Examples• Monitor number of directions given and

number of directions student followed without exhibiting challenging behavior.– Remember 1X per week is usually sufficient

– Collect data when student is given typical directions in resource room or mainstream classroom.

– If provided support in mainstream have support staff monitor directions given by teacher and number followed by student.


• _____ will increase his ability to remain on-task from being on-task an average of 60% of the time to being on-task 85% of the time through the use of skills training, classroom modeling, teacher feedback and praise.

Data Collection Examples• Momentary time sampling

– Remember 1X per week is usually sufficient– Special Ed teacher or para can collect data– Collect data when student is in resource room or

mainstream classroom.• If possible, during both teacher lead instruction and

independent work time.

– If provided support in mainstream have support staff could collect data for 5-10 minutes.

• Remember to communicate with mainstream teacher that support staff is to collect data during this time.

Interpreting and Analyzing Data


• After data is collected, draw a graph and use graph to evaluate student progress and to formulate instructional decisions. – There is no substitute for a graphic display

of data.– Graphic displays do not need to be

complex or fancy.


A simple hand drawn graph is sufficient.


There are two basic methods to interpreting academic data:

1. Decision rules based on the most recent 3-4 consecutive scores.

2. Decision rules based on the trend-line.

Goal-Line Versus Student’s Current Rate of Progress

• Examine both level and rate of student progress to determine whether student is progressing adequately to reach end-of-year goals

• Compare student’s current rate of progress with projected rate of progress (i.e., goal-line):– To judge whether the instructional program needs to be

modified to better meet student needs or

– To determine whether the goal should be raised

General Decision Making Framework

• 4-Point Rule (supersedes the trend-line rule):– If 3 weeks of instruction have occurred and

at least 6 points have been collected, then examine the four most recent data points:

• If all four are above goal-line, then increase goal.• If all four are below goal-line, then make a

teaching change.• If the four data points are both above and below

the goal-line, then keep collecting data until trend-line rule or 4-point rule can be applied.

What Is the Data-Based Decision Rule?

• The 4-point rule may be applied: Data-based decision is to raise the goal.

G

General Decision Making Framework

• Trend-Line Rule:– If 4 weeks of instruction have occurred and

at least 8 data points have been collected, then figure trend of current performance and compare to goal-line:

• If trend of student progress is steeper than goal-line, then raise goal.

• If trend of student progress is less steep than goal-line, then make a teaching change.

What Is the Data-Based Decision Rule?

• The trend-line rule may be applied: Data-based decision is to make an instructional change.

G

T

Benchmark Report - Box & Whiskers Graphs (box plots)AIMSweb commonly uses box plots to report data.This chart will explain box plots:

Consider bell-curve. Consider bell-curve. Box plots are somewhat Box plots are somewhat similar in shape and representation.similar in shape and representation.

7575thth percentile percentile

Median (50Median (50thth percentile) percentile)




outlieroutlier

Average range Average range of population of population includedincluded in in

sample.sample.

Below Average Below Average RangeRange

Above Average Above Average RangeRange

Strategic Monitor Individual Student Report

Progress Monitor Report

Progress Monitor Report

Percent vs Percentiles

• Percent refers to the proportion of the whole thing, as in He got 25% right on the test, or 90% of the kids passed the test.

• Essentially, it answers the question, How much? or What part of 100. A percent is absolute, referring to the exact part of the whole.

Percent vs Percentiles• Percentile is often used as a shortening of the term percentile

rank. In the world of assessment, a percentile rank describes how a score fits in to the distribution or spread of scores in the comparison group. If a student scored at the 40th percentile, he scored better than 40 percent of those in the comparison group. It answers the question How well compared to…?

• A percentile rank is a score relative to the comparison group, and is totally dependent on how everyone in that group performs.

AIMSweb box plots are percentile ranks.

Behavior Decision Rules

• Some time after 5 days (minimum number for evaluating an intervention) but no longer than 15 days (maximum number of days for allowing any program to run before change in behavior is expected), analyze the data and if necessary develop a program change.

• Purpose is to compare data before the intervention to that following the intervention using analyses of (a) change in level, (b) slope, and (c) variability.

Variability

• Variability or bounce in data refers to the range of scores in the data collected.– Can be an indicator of a number of issues

such as • low motivation, • ineffective prompting, • inconsistency in implementation,• inconsistency in expectations across staff.

Indicating finished

Variability

• If the student’s data indicates considerable variability or the program is not effective, check for fidelity of implementation issues before making other changes.

Fidelity of Implementation

• Fidelity of implementation occurs when staff use the instructional strategies and deliver the content of the intervention in the same way that they were designed to be used and delivered.– Frequency and consistency are critical to

fidelity of implementation.

Data Collection Summary

• Keep the data collection simple and useable.

• Instill in your staff that data collection is as important as instruction.

• Always use the data to drive instruction.

• Use ready made tools to simplify the process.

Documents

Data Collection and Analysis. Baseline Data Whenever possible collect baseline data. –This is most important for behavior programs. –Also very useful