PDF® of Powerpoint® Data Collection: An Introduction

Data Collection:

An Introduction

PowerPoint Slides

to be used in conjunction

with the Facilitator’s Guide

Recommended citation:

Wakeman, S., & Henderson, K. (2012). Data collection: An

introduction – A PowerPoint presentation for

professional development. Modules Addressing

Special Education and Teacher Education (MAST).

Greenville, NC: East Carolina University.

This resource includes contributions from the module

developer and MAST Module Project colleagues (in

alphabetical order) Kelly Henderson (Facilitator Guide

Editor), Tanner Jones (Web Designer), Diane Kester

(Editor), Sue Byrd Steinweg (Project Director), Bradley

Baggett (Graduate Assistant), and Sandra Hopfengardner

Warren (Principal Investigator).

Copyright © 2012, East Carolina University.

Session Agenda

• Introduction

• Session Goals and Objectives

• Background Overview

• Legislation

• Data Collection Design

• Response Modes

Session Agenda, continued

• Writing Measurable Skills/Tasks

• Data Collection

• Practice Matching Skill to Sheet

• Reliable Data Collection

• Summary

• Evaluation

Introduction

Brian is a student in Mrs.

Stephens’ 7th grade class. He

works very hard and tries on

every task he is given. One

task in particular is troubling

for Mrs. Stephens. Brian

needs to work on solving for a

variable in an algebraic

equation.

Introduction, continued

He works on this task in his inclusion class

as well as on his homework, therefore Mrs.

Stephens rarely sees him solve a problem.

He can add and subtract with regrouping,

so Mrs. Stephens feels comfortable that he

can complete that part of the task. He

continues to get each question wrong,

however, and she is not sure what to do

about it.


• What ideas do you have to help her

gather information about Brian’s

performance?

• One way Mrs. Stephens could find out

what is happening within Brian’s

performance is to collect data during his

performance using a systematic method

or plan.


• Mrs. Stephens could:

– identify the steps to solve for the variable and create a task analytic data collection sheet and use it to gather data as she observes Brian attempting to solve for the variable.

– Or Mrs. Stephens could analyze work samples to identify error patterns by examining papers from Brian on which he has worked out on paper each step to solve the problem.

• Either approach would allow Mrs. Stephens to collect performance data from Brian.


• Data could be collected by the teacher, paraprofessional, general education teacher, and/or parent using a specified method. Whichever method, the data collected should inform their instructional efforts for supporting Brian.

Session Goal and Objectives

Objectives: Participants will be able to:

1. Identify types of data collection systems

with a focus on the data collection sheets

(e.g., task analysis, repeated trial, repeated

opportunity).

2. Identify measurable skills with a condition,

behavior, and criterion for mastery.

3. Identify the correct data collection sheet for

a given skill.

Overview

• Data collection is the objective and accurate

measurement of a student’s present level of

performance of and progress, or lack of

progress, on a task, activity, or behavior.

• Starting point- the baseline data of what the

student is currently doing or not doing.

• Data collection ends at the point where the

student reaches the criteria for mastery.

Overview, continued

• Accurate data collection allows the teacher or service provider to determine:

(a) a student’s present levels of performance (PLOP);

(b) any changes needed to provide adaptations to the materials or the interventions;

(c) student progress or lack of progress;

(d) program modifications or supports which are necessary; and

(e) any patterns of behavior.

Overview, continued

• Data can be used on a:

– daily basis to inform instructional and

behavioral efforts for individual students.

– monthly, quarterly, or yearly basis to inform

programming efforts (e.g., the need for

additional or tertiary level interventions, the

need for support services, and the plan for

sequential instructional efforts).

Legislation

• The Individuals with Disabilities

Education Act (1997) required schools

to:

– monitor and provide parents with

documentation of a child’s progress towards

mastery for annual Individual Education

Program (IEP) goals and objectives; and

– provide documentation of special education

student participation in state-wide alternate

assessments.

Legislation, continued

• Data collection plays an essential part in

this documentation, particularly in states

that require teachers to provide evidence

of student work as a part of assessment.

• Data collection has been recognized as a

cornerstone for research-based practices.

For example, Response to Intervention

(RTI) is based upon the collection of valid

student performance data.

Data Collection Design

• Data collection of student progress is

designed to be: – Objective: data should be measured

objectively. The skill must be observable and

measurable. It must be written in a manner

that enables multiple observers to see the

same outcome. The use of multiple observers

is an important step to demonstrate that the

data is a valid representation of the student’s

performance.

Data Collection Design, continued

– Systematic: the process of collecting data must

follow a systematic pattern. Random collection

of different expectations or outcomes creates

the chance of a variety of interpretations of the

data. Defining the cues, prompts (including

error corrections), materials, and data

collectors is necessary. A plan should be

identify when data collection should occur;

data would typically be taken at least 2-3 times

a week.

Data Collection Design, continued

– Defensible: a data collection system should

be carefully thought out. Why should the

teacher collect this data and collect it this

way? Data provides teachers with the

necessary information to make defensible

decisions about instruction.

Response Modes

• Data collected must be an accurate

representation of what the student knows

and can do.

• Tasks must be designed and supports

must be utilized in such a way as the

student can independently respond to any

items presented.

Response Modes, continued

– For example, if a task was defined for a

student to identify sight words but the student

used picture symbols and photographs for

receptive and expressive language, the task

would be too difficult for the student.

– However, if the student was asked to increase

his picture vocabulary paired with the written

word, the student could respond within the task

as designed. How the student responds best

must be considered when designing a task in

which data will be collected.

Response Modes, Activity ?

– Review sample data sheets. Examine

the objective for each data sheet to

see (a) what responses students

would have to have to be able to

show their understanding of the task

as well as (b) what supports are

provided to them to make an accurate

response possible.


• Designing the system to collect data

typically falls to the teacher or behavior

specialist. These practitioners are

responsible for:

(a) identifying and defining skill/behavior;

(b) establishing the baseline performance of the

student (what the student can do) and the

criteria, including mastery (what they want the

student to be able to do);

(c) creating a user-friendly data collection

method;


(d) training team members to collect data across

environments;

(e) reviewing and analyzing data weekly; and

(f) modifying programs based on data.

• The skill or target behavior is typically an

IEP objective, a skill related to prioritized

content standards, a behavior that impedes

the student’s learning, or a skill to be

assessed for a formal assessment, such as

the alternate assessment.

Writing Measurable Skills/Tasks

• There are three components to each

skill or task: condition, behavior, and

mastery level.

• The following narrated slides are

available at

http://mast.ecu.edu/modules/dc_intro/lib

/media/slides01/SlideShow.html.

http://mast.ecu.edu/modules/dc_intro/lib/media/slides01/SlideShow.html


Writing Measurable

Skills/Tasks

Data Collection: An Introduction

Condition

• What will be

presented to the

student?

• What materials

will be used (be

generic)?

• Given manipulatives, a

worksheet with 10

problems, and

scaffolded support, if

necessary, and a

pencil, Josh will

independently solve 9

out of the 10 problems

correctly for 3

consecutive trials.

Behavior

• What will the

student be doing?

• Given manipulatives,

a worksheet with 10

problems and

scaffolded support, if

necessary, and a

pencil, Josh will


out of the 10

problems correctly for

3 consecutive trials.

Criteria for Mastery • What level of

increase or

accuracy are you

expecting?

• How many days will

the student have to

reach that criteria

to be considered as

having mastered

the task?

• Given manipulatives, a

worksheet with 10

problems and scaffolded

support, if necessary,

and a pencil, Josh will


out of the 10 problems

correctly for 3

consecutive trials.

What is missing?

• Jane will independently read 4 out of 5 sight words correctly for 3 consecutive trials.

• Given a writing instrument or stamper, Michael will independently mark on paper as an emerging signature.

• Given her phone number and a distractor, Gerry will independently point to her own phone number 4 of 5 trials correctly for 3 out of 5 days.

Non Examples

• What is wrong with the way the following tasks are written?

– Sherrie will participate in an inquiry based science lesson with physical assistance for 4/5 trials.

– Given a book of her choice, Lynn will read a book independently.

– Given a crosswalk sign, Joel will cross the street safely on 8 of 10 occasions.

Activity suggestion

- What unusual experiences have you had

with writing measurable skills and

objectives and the results?

Data Collection

• Creating a user-friendly data collection

method starts with the skill or task and

deciding how to document progress. Options

include data sheets, permanent products

(e.g., work samples), anecdotal notes, or a

combination.

• Data collection sheets should be determined

prior to instruction or assessment. Selection

is guided by the purpose the data and the

way the task is written.

Data Collection, continued

• Some examples of different types of data

collection sheets follow.

a) Task Analysis- Outlines the steps

necessary to complete a task. When the

student is presented the task to complete,

the number of steps correct is scored. The

teacher decides on the number of steps

presented in each trial (total task versus

forward or backward chaining).


• For example, a task analysis data sheet

would likely be used to record the steps for

a student to use money in a vending

machine or the steps in a science

experiment.

• An example of a task analytic data sheet

follows and is available at

http://mast.ecu.edu/modules/dc_intro/lib/do

cuments/Task_Analysis_Data_Sheet.pdf .

http://mast.ecu.edu/modules/dc_intro/lib/documents/Task_Analysis_Data_Sheet.pdf




b) Repeated Trial- One of the most common

data sheets. Teacher delivers teaching

trials in a set (e.g., present a sight word,

then another word, then another word, etc.)

Responses given by student are charted on

the data sheet (either correct/incorrect or

independent/prompted depending upon if

the task is instructional or assessment


• An example of a repeated trial data sheet

is provided here. A copy of the repeated

Trial Data Sheet follows and is available

at

http://mast.ecu.edu/modules/dc_intro/lib/d

ocuments/Repeated_trial_data_sheet.pdf.

http://mast.ecu.edu/modules/dc_intro/lib/documents/Repeated_trial_data_sheet.pdf

http://mast.ecu.edu/modules/dc_intro/lib/documents/Repeated_trial_data_sheet.pdf


• Note on the data sheet that items

presented to the student are listed on the

left side of the data sheet and each unique

item has its own row for data collection.

• It’s easy to see if a pattern emerges for

specific items (e.g., the student can

identify snowy consistently but not cloudy).


• This is important as specific items may

be too difficult for the student and

prevent mastery of the overall task

• For example- the task is 8 out of 10 sight

words and the student consistently gets

7 correct, the teacher can decide what

instructionally to do next to support

student acquisition of the 3 words that

are preventing mastery of the task.


c) Repeated Opportunity- appropriate for

skills that are taught throughout day,

i.e., the trials are spread out. The

student’s responses are charted as they

are made. Examples of skills that could

be recorded on a repeated opportunity

data sheet are using an object schedule

or telling clock time at start of each

lesson.


• An example of a Repeated Opportunity

Data Sheet with data in it follows and is

available at


/documents/Repeated_Opportunity_dat

a_sheet.pdf.

http://mast.ecu.edu/modules/dc_intro/lib/documents/Repeated_Opportunity_data_sheet.pdf




• On the previous data sheet, the who,

where, and potentially with what the

student makes the response for the task

or item is recorded for each opportunity.

• In this case, the student is learning to

independently transition when given a

prompt during different parts of his

schedule.


• Some students may respond differently to

different people or in different locations

depending upon what distractors may be

present.

• This type of repeated opportunity data

allows the teacher to see what patterns, if

any, exist that prevent or promote the

students response.


d) Frequency- typically used to measure

the degree to which there is an increase

or decrease in number of times the

student uses a new response or refrains

from an unwanted response. This skill

or task may be measured throughout

day (e.g., hand raising instead of calling

out) or in one lesson (e.g., activating a

communication device to respond).


• An example of a frequency data sheet

follows and is available at


/documents/Frequency_data_sheet.pdf.

http://mast.ecu.edu/modules/dc_intro/lib/documents/Frequency_data_sheet.pdf

http://mast.ecu.edu/modules/dc_intro/lib/documents/Frequency_data_sheet.pdf


• The previous sheets is an example of

how frequency data can be taken. The

student is sorting a series of pictures into

two categories. The frequency of how

many pictures were sorted correctly is

recorded.

• Another example of how frequency data

could be collected is to simply record a

number for each date the items or prompt

is given.


• For example, students could record how

many sit ups they do each day, the

number of correct math facts in a minute,

or the number of out of seat occurrences

in a class period. A simple table can

include date and number of responses.

Date Response

3/16/10 14

3/17/10 16

3/18/10 18


e) Duration- used to record a skill that is

measured in time, specifically, the total

amount of time the student engages in

task. The purpose of the instruction may

be to increase the amount of time (e.g.,

attending to task) or decrease the

amount of time (e.g., length of

tantruming behavior). Time could be

recorded in seconds or minutes.


• A duration data sheet may be appropriate

when the student is expected to work for

30 consecutive minutes on a vocational

task or to indicate the length of time for a

student to transition between tasks.

• An example of a duration data sheet with

data follows and is available at

http://mast.ecu.edu/modules/dc_intro/lib/d

ocuments/Duration_data_sheet.pdf.

http://mast.ecu.edu/modules/dc_intro/lib/documents/Duration_data_sheet.pdf

http://mast.ecu.edu/modules/dc_intro/lib/documents/Duration_data_sheet.pdf


• This duration data sheet has four

columns, but a sheet could have an

additional column in which the prompting

level (i.e., physical, model, or

independent) is listed on its own.

• In the example, mastery was described

in the objective as an independent

performance of the task under 30

seconds of the given prompt for 3

consecutive days.


• It is possible that the time may also be

increased (e.g., actively engaged in

completing a given task by 10 seconds

each day until student completes a

given task without prompting up to 5

minutes).


f) Cumulative- appropriate when one

discrete response is to be used

throughout the day or over a period of

days. Responses are totaled over days

to reach a criterion. An example of a

skill for which a teacher may use a

cumulative data sheet is when a student

is expected to independently ask for

help when appropriate.


• An example of a cumulative data sheet

with data follows and is available at


/documents/Cumulative_data_sheet.pdf

.

http://mast.ecu.edu/modules/dc_intro/lib/documents/Cumulative_data_sheet.pdf

http://mast.ecu.edu/modules/dc_intro/lib/documents/Cumulative_data_sheet.pdf


• Notice the objective on the previous data

sheet. The expectation for mastery is

such that the student will acquire a set of

information over a longer period of time

rather than within a given response set

(like the repeated trial data sheet).

• The student is to independently

recognize 10 new science terms within

context over the school year.


• Student responses are considered in the

context of natural cues or when the

chance is given to elicit the response

that may not occur on a daily or even

weekly basis.


• One important consideration when

designing a data sheet is to first

determine the purpose for collecting the

data.

– Is the data collected being used for

assessment purposes or is it being used

within instructional trials?


• The video included later is an

assessment. When presenting

assessment trials, the teacher will

typically not provide correct responses

unless the student replies, “I don’t know”

as instructed prior to the beginning of

the assessment.


• If it was an instructional video rather than

an assessment video, the teacher would

correct errors and support student

acquisition of the words within each trial,

likely using a prompting system. Note the

difference when you watch the video.

• The table that follows helps illustrate the

type of scoring that would occur

depending upon the purpose or use of the

data.


• For an assessment probe (a set of items

presented to test what the student knows

and can do independently) the first row

shows what would be recorded on the data

sheet- accuracy of responses.

• The repeated trial data sheet in this module

is an example of an assessment probe.


• For an instructional probe, typically the

data sheet will list a prompting system

in the key. The duration data sheet in

this module is an example of an

instructional probe.


Response is

correct/incorrect

(ASSESSMENT OR NON

INSTRUCTIONAL)

+ = Correct

– = Not correct

x = Does not attempt

Type of prompt or

highest level of prompt

used

I = Independent

G = Gestural

NV = Non-Specific Verbal (e.g.,

“What’s next?”)

V = Verbal

M = Model

P = Partial Physical

F = Full Physical

Practice Matching Skill to Sheet

• Watch the slide show at


/media/slides02/SlideShow.html to

practice matching skills to the correct

data sheet. Copies of the slides follow.



Matching skills to data sheets

• Data collection: An Introduction

Which is this?

Student:

Task: Given a computer and website, Student

will complete all steps to locate chosen website

independently at 100% accuracy for 3 websites.

Steps: Dates: 9/02 9/07 9/09 9/12 9/16 9/18 9/20

1. Find and double-click

Internet Explorer icon I I I I I I I

2. Wait for Home page to

appear V V V V I I V

3. Find and click “Search” icon

in toolbar M M V V M M V

4. Type in search topic of

choice V V V V V V V

5. Find and click “Search”

button M I I M M M I

6. Click desired web page I V M M V V I

Total Independent Correct: 2 2 2 1 2 2 3

Where: CL H MC CC CL CC H

With Whom: T PA T CTS A CTS PA

And this?

Student:

Task: When given a book on tape and switch,

student will activate the switch to start the book

in less than 30 seconds for 3 consecutive trials.

Date Student Response/Time

on Task

Where With Whom

10/01 M/65 secs. CL T

10/02 M/62 secs. LG A

10/06 M/60 secs. CL T

10/09 P/59secs. LG P

10/10 P/60 secs. CL A

And this?

Student:

Task: Given sight words written on index cards, Student will identify 5 out

of 7 weather related words for 3/5 trials.

Date: 3/03 3/04 3/07 3/10 3/12 3/13 3/18 3/20 3/21 3/25 3/26

1 Snowy - + + + + + + - - + +

2 Cloudy + + - + - - + - + - +

3 Sunny + + + + + + + + + + +

4 Rainy + - + + + + + + + + +

5 Windy + - + - - - - + - + -

6 Hot + + + + + + + + + + +

7 Cold + + + + + + + + + + +

Where: H MC CL CL H CL MC H CL MC H

With: PA A T P PA T A PA P A PA

Material Used: HWK N W W HWK W N HWK W N HWK

Total Independent

Correct:

6 5 6 6 5 5 6 5 5 6 6

How about this one?

Student:

Task: Given her AAC device and a story, Student will activate a voice

output device to indicate a repeated story line 4/5 times requested for 3

stories.

Date: 3/03 3/04 3/07 3/10 3/12 3/13 3/18 3/20 3/21 3/25 3/26

1 - + + + + + + - - + +

2 + + - + - - + - + - +

3 + + + + + + + + + + +

4 + - + + + + + + + + +

5 + - + - - - - + - + +

Where: H MC CL CL H CL MC H CL MC H

With: PA A T P PA T A PA P A PA

Material Used: HWK N W W HWK W N HWK W N HWK

Total Independent

Correct:

4 3 4 4 3 3 4 3 3 4 5

What about this one?

DATE

Given a reminder card on his

desk, Marcus will get out of his

seat less than three times a day

without asking for 5 consecutive

days.

4/11 llll=4

4/12 lllllllll=9

4/15 lll=3

4/16 lll=3

4/17 lll=3

How about which sheet would you

use for these skills?

• Given a jig and materials, Matt will set the table correctly (follow 5 steps) each day for 3/5 days.

• Given 15 minutes of leisure time, Jerry will stay in the leisure area leaving no more than once without asking for 5 trials.

• Given familiar people, Kitty will initiate a greeting as she approaches for 3 or more people each day.

Reliable Data Collection

• One important aspect of data collection is

ensuring that the data collected is

objective:

– Some measures in place to avoid biased or

incorrect data being included in the data set.

– Biased data is not objective.

– Data collectors who have preconceived

ideas about how the data should be may not

be able to see what is actually happening,

particularly in behavioral issues. here.

Reliable Data Collection, continued

– Incorrect data can happen when the person

collecting data does not recognize the prompt

or does not understand what a correct

response looks like for the student.

– Data should be apparent and available to

stakeholders.

– The data should be defensible in its accuracy

and should be a valid representation of what

the student is actually doing as it plays a key

role in making decisions about the student’s

education.

•


• Watch the video at

http://mast.ecu.edu/modules/dc_intro/lib/

media/MOV051.html. The video

illustrates a child reading sight words:

Galaxy, Saturn, Jupiter, Earth, Venus,

Mars, Mercury, Uranus, Pluto, Neptune.

• Use a repeated trial data sheet and take

data with the assessor.

http://mast.ecu.edu/modules/dc_intro/lib/media/MOV051.html

http://mast.ecu.edu/modules/dc_intro/lib/media/MOV051.html


• After completing your own data

collection, view the answers in the data

sheet available at

http://mast.ecu.edu/modules/dc_intro/lib/

documents/Data_Sheet_answers_solar_

system_sight_words_data_collection.pdf

once you have watched the video. A

copy of the data sheet follows.

http://mast.ecu.edu/modules/dc_intro/lib/documents/Data_Sheet_answers_solar_system_sight_words_data_collection.pdf




• Check your answers with the answer sheet

provided. A few of the incorrect responses

are worth discussion:

– Emma mispronounces Uranus in trial two so it

is first checked by the teacher and then

recorded as an incorrect response.

– Emma also says in trial 2 that she does not

know a word (Mercury), so that word is also

marked as incorrect.


– When reviewing the mastery level set in

the objective for the task, Emma does not

master this task as she was required to get

9 out of 10 correct for three consecutive

trials and Emma only has one trial at the

mastery level.

Summary

• Data collection is one of the most important

aspects of instruction and assessment.

• If a plan to collect observable, measurable,

valid data is in place, practitioners can

identify student response patterns and

support student progress towards mastery.

• Skills or tasks must be written with a

condition, behavior, and criteria for mastery.

Summary, continued

• Data collection systems can include

student work samples, anecdotal records,

data sheets, or a combination of all three.

• Data can be collected using a data sheet

that best matches the purpose or use of

the data and the type of skill to be

measured.

Summary, continued

• Examples of data sheets include task

analytic, repeated trial, repeated

opportunity, frequency, duration, and

cumulative.

• Finally, reliable data collection is a must

before analysis of the data can occur.

Focus and Reflection Questions

• What do you think is the most

difficult part of creating and using

data collection strategies?

• Why?

• What can you suggest to make the

task easier?

Application and Extension

Activities

1. Data can be collected using a data

sheet that best matches the purpose or

use of the data and the type of skill to

be measured. Examples of data sheets

include task analytic, repeated trial,

repeated opportunity, frequency,

duration, and cumulative.

Application and Extension

Activities, continued

Design their own data sheets for specific

purposes and types of skills for other tasks

and/or objectives, keeping in mind your

classroom and students.

• Task analytic

• Repeated trial

• Repeated opportunity

• Frequency

• Duration

• Cumulative

Self-Assessment

• A self-assessment with response feedback

is available at

http://mast.ecu.edu/modules/dc_intro/quiz/

. Participants may take this assessment

online to evaluate their learning about

content presented in this module.



Session Evaluation

• A form for participants to evaluate the

session is available in the Facilitator’s

Guide.

Documents

PDF® of Powerpoint® Data Collection: An Introduction