Getting the Most out of Data Collection and Analysis: Part ...€¦ · > Data collection is the process of gathering and measuring information on targeted variables in an established

>www.dpw.state.pa.us >www.dhs.pa.gov

Getting the Most out of Data Collection and Analysis:

Part 1- Creating Individualized Data Sheets

Amy Alford, M.Ed., BCBAClinical Consultant

September 28, 2017


• Purpose of data collection• Determining needs to create the tool• How to refine/individualize your data sheet• How to ensure others can/will use your data sheet

Agenda


PURPOSE OF DATA COLLECTION


Data collection is the process of gathering and measuring information on targeted variables in an established systematic fashion, which then

enables one to answer relevant questions and evaluate outcomes. Data collection is a component of research in all fields of study

including physical and social sciences, humanities, and business. While methods vary by discipline, the emphasis on ensuring accurate

and honest collection remains the same.

The goal for all data collection is to capture quality evidence that allows analysis to lead to the formulation of convincing and credible

answers to the questions that have been posed.

www.reference.com


The 5 “Ws” of Data Collection• Who collects data? Behavior Specialist, Direct Support

Professionals, Job Coach, family members, participant (for self-monitoring)

• What data do you collect? Frequency, Duration, Latency, Intensity, etc… Problem Behaviors and Desired Behaviors, Progress toward goals/objectives, Skill Deficits

• Where do you collect data? Wherever the behavior/skill is occurring or needs to occur, natural setting

• When do you collect data? As often as possible, but only when necessary

• Why is data collection important? Confirms or refutes hypotheses, identifies skill deficits/areas to target, shows progress over time, informs future changes to services/supports

How can data be measured and described?How to

Measure and Describe Data

Definition Example

Anecdotal Written narrative/summary of all behaviors over a period of time, including info about antecedents/consequences

ABC charts: On Saturday Chris was watching TV and his staff came over to ask him to take a shower. He began to yell inappropriate comments. Staff turned off the TV.

Count/Event/ Frequency

# times a behavior occurs Chris made 5 inappropriate comments on Saturday.

Rate # times a behavior occurs over a certain amount of time

Chris made 5 inappropriate comments in 10 minutes, for a rate of .5/minute.

Duration How long a behavior occurs Chris’ behavior on Saturday lasted for 30 minutes.

Latency How long after stimulus does behavior occur

After staff asked Chris to take a shower, he began making inappropriate comments after 30 seconds.

Inter-response Time (IRT)

Amount of time between when one behavior ends and another begins

The first behavioral episode ended at 1:05pm and the 2nd

began at 2:15, for an IRT of 1 hour, 10 min.

How can data be measured and described, cont…?How to Measure

and DescribeData

Definition Example

Percentage A ratio, usually a # expressed out of 100

Chris plays video games 75% of his day.

Trials to Criterion

How many attempts (trials) were needed to master a skill

When learning to cook dinner, Chris needed 25 trials to make a meal with 90% independence.

Topography What the behavior looks like

The inappropriate comments began as curse words, but now calls others “mean, stupid”

Magnitude/ Intensity

The strength or force of a bx

When Chris makes comments, he is often waving hands wildly, in close proximity to others’ faces

Interval/ Time-Sampling

Presence or absence of bx over certain amount of time

Every 30 seconds, staff will look at Chris to determine if he was or was not making a comment for 15 minutes. In 30 intervals, Chris was making an inappropriate comment in 18 out of 30 intervals.

Permanent Record

Written, recorded, or completed work assessed as a whole

After putting bicycle parts together at the store, Chris’ supervisor checks to make sure all steps were completed accurately


DETERMINING NEEDS TO CREATE THE TOOL


Needs > Goals/Objectives > Data Collection Tool


• Do you need to understand when Bryan’s engaging in problematic behaviors and how others are responding?

• Do you need to understand if Stephanie knows how to use public transportation to get to work?

• Do you need to understand if Nolan is taking his medications as prescribed? And what time?

• Do you need to understand how long it takes Tyler to respond to a question? And whether he knew what the question meant?

• Do you need to understand if Chris is using his time management skills effectively?

What are the needs?


• A well-written goal/objective can set you up on the path to developing a useful data collection tool Use context clues Dissect each component

• As the data tool is developed, revisions to the goal/objective may be needed- be flexible and think realistically

Goals and Objectives


ExampleObjective Data Collection Considerations

When Bob arrives at work, ready to start his shift,

-Will Bob self-monitor? Does he have on the job support who will collect data?-Data sheet needs to be at work and available as soon as he is ready to start his day

He will go to his desk, turn on the computer, open/respond to emails, and then complete at least 3 tasks received from supervisor before lunch,

-Should this be a task analysis?-Does he know how to do each task? Or does he need reminders to complete each task within a certain amount of time?

with less than 5 prompts to remain on task each day, 5/5 days for one consecutive month.

- Data sheet should include a place to mark off going to desk, turning on computer, checking emails, completing 3 tasks, a place to record number of prompts, at least 5 days worth of data on one sheet, but maybe more.

Bob’s Sample Data SheetDate Go to Desk? Turn on

computer?Open/Respond to email?

Tasks completed before lunch

# total prompts needed to stay on task

Monday___/___/___

____ yes ____ no ____ yes ____ no ____ yes ____ no __ 1__ 2__ 3

Tuesday___/___/___


Wednesday___/___/___


Thursday___/___/___


Friday___/___/___


End of week review:

Did he complete 3 tasks before lunch each day?____ yes ____ no

Did he require 5 or less prompts each day?___ yes ___ no


HOW TO REFINE/INDIVIDUALIZE YOUR DATA SHEET: TIPS AND TRICKS


Let’s take a step back…


• Anecdotal and/or Frequency data, only • A generic data sheet• A data sheet that is hard to understand/use

What do we commonly see?


A Antecedents

BBehavior

CConsequences

Generic ABC Chart


Antecedents Behaviors Consequences__ Undesired task __Yelling/ cursing __Demand removed

__Preferred activity __Ignoring request __ Verbal explanation/counseling

__ Prompt to clean apartment

__ Grabbing/ Scratching staff

__ Remove attention

How to Refine an ABC Chart- make a checklist

Another ABC Checklist- even more refined

Behavior Checklist Name: ________________Dates/Week: _____________ Treatment Plan: Sept. ‘07 Morning Session F=Frequency (mark with a tally for each occurrence) D=Duration in minutes (rounded)

Inappropriate Behaviors (Function)

Monday Tuesday Wednesday Thursday Friday Totals(Avg) F D F D F D F

D

F D F D

Self Injurious 3’, 5’, 2’, 9’,

Grabbing/hitting others

Elopement

Generic

Behavior Checklist Name: ________________Dates/Week: _____________ Treatment Plan: Sept. ‘07 Morning Session F=Frequency (mark with a tally for each occurrence) D=Duration in minutes (rounded)

Inappropriate Behaviors (Function)

Monday Tuesday Wednesday Thursday Friday Totals(Avg) F D F D F D F

D

F D F D

Self Injurious 3’, 5’, 2’, 9’,

Grabbing/hitting others

Elopement

How Could We Refine This?

Time Activity Monday Tuesday Wednesday Thursday Friday

Interval Recording/Scatter Plot FormUsed for: ___ Frequency count (tally each time behavior occurs within each interval)

___ Scatter plot (Key: o = 1 occurrence; *= more than 1; x = crisisParticipant: ________________________________________ Dates: ___________________Behavior(s): __________________________________________________________________Weekly Total = ______________________________ Average per day = __________________

Generic

Time Activity Monday Tuesday Wednesday Thursday Friday

Interval Recording/Scatter Plot FormUsed for: ___ Frequency count (tally each time behavior occurs within each interval)

___ Scatter plot (Key: o = 1 occurrence; *= more than 1; x = crisisParticipant: ________________________________________ Dates: ___________________Behavior(s): __________________________________________________________________Weekly Total = ______________________________ Average per day = __________________


Time Frame

Date_______ Date_______ Date ______ Date ______ Date ______ Total Responses

Time Periods

Rate of Behavior

Start_______ Start ______ Start ______ Start ______ Start ______

Stop_______ Stop ______ Stop ______ Stop ______ Stop ______

Total Responses

Time Periods

Rate of Behavior

Duration and ScatterplotInterval Size _____________Name: _____________________________________ Behavior: ___________________________________

Generic

Time Frame

Date_______ Date_______ Date ______ Date ______ Date ______ Total Responses

Time Periods

Rate of Behavior

Start_______ Start ______ Start ______ Start ______ Start ______

Stop_______ Stop ______ Stop ______ Stop ______ Stop ______

Total Responses

Time Periods

Rate of Behavior

Duration and ScatterplotInterval Size _____________Name: _____________________________________ Behavior: ___________________________________


Another Individualized ScatterplotMon Wed Fri Total

Time: 10 12 2 4 10 12 2 4 10 12 2 4

Hands/feet to self ______ / 24

Had all materials ______ / 24

Required less than 3 prompts to stay on task

______ / 24

Used appropriate words

______ / 24

Remained in work area

______ / 24

Completed activities

______ / 24

Total _____ / 48 _____ / 48 _____ / 48 ______ / 144

Scoring Rubric: 0- None of the time, 1- Most of the time, 2- All the timeGoal: 80%- for one quarter (39/48 points per day, 115 points out of 144 per week)

Generic

28

Individualized

Task Analysis Cue Date

10. When crock pot alarm rings, scoop dinner in serving bowl

Alarm sounds

9. Set time and temp on crock pot

Time and temp known

8. Verbally state time and temp on recipe card

Lid on

7. Put lid on crock pot Sauce in

6. Put sauce in crock pot Veggies in

5. Put veggies in crock pot Meat in

4.Put meat in crock pot Crock pot on counter

3. Get out crock pot Ingredients on counter

2. Get out ingredients Read recipe card

1.Choose a recipe After breakfast

% Independent/Correct

Obj Criteria: 80% for 1 mo.

Code:G- GestureIV-Indirect verbalDV- Direct verbal

Individualized Task Analysis


HOW TO ENSURE OTHERS CAN/WILL USE YOUR DATA SHEET


• Provide written directions On the data sheet, easily accessible

• Frequent check-ins; Model how data should be collected. Observe Initially and ongoing

• Ask questions (of the person who collected the data) To check they understand the tool and will collect data as intended

• Ask for input for how to revise More buy-in = more accurate (most of the time)

• Identify methods to record data short-hand and then transfer to data sheet Make phone calls, emails

Suggestions

31


• Consider use of technology• Consider what data is already being collected Sleep charts Medication refusals Mood charts Seizure records

• Keep the analysis in mind- how will you transfer this to Excel to analyze? Put variables in same order/location Keep things clean (vertical and horizontal) One-word or short phrase; abbreviate

Suggestions

32


35

Original

36

Revised- After we received input

Technology

37


Does someone already track this?

Date Setting Did he usecoping

strategy?

Duration ofYelling

# Hitting Behaviors

Comments

1/7 Restaurant no 5 min 1 Children’s birthday party, had to leave restaurant to calm down

1/8 Hobby Shop

no 6 min 3 Josh realized he did not have enough money for what we wanted to buy

1/10 Bookstore yes 1 min 0 Small child was crying, quickly redirected to different section of store

1/15 Restaurant no 4 min 0 Did not want to eat at this restaurant

1/17 Bank no 30 sec 1 Large noise from office as we were leaving

1/18 Grocery Store

yes 1 min 0 First time using strategy with only 2 prompts from staff

1/21 Mall no 5 min 1 Music was very loud, lots of crowds

1/23 Restaurant yes 1 min 0 Josh’s volume of yelling decreased

1/28 HobbyShop

yes 0 0 Great Day- group of kids were in store, used strategy independently

Keep the analysis in mind! (More about this topic in part 2 of the data series)

40

Keep the analysis in mind!


FINAL THOUGHTS

>www.dpw.state.pa.us >www.dhs.pa.gov 42

1. Determine the purpose for data collection. What are the needs?

2. Define the behavior/skill specifically and measurably.

3. Decide where, how often, and who will collect data.

4. Design a data collection tool(s) to fit your needs – simple, but functional with clear coding system/directions.

5. Train the data collectors.

6. Transfer the data to a visual representation (graph).- We’ll review this step in Part 3 of the Data Series.

Reminder- Steps for Data Collection Process


• Simply not collecting data is not ok!• We cannot make informed decisions to support these

individuals if we do not have information to back it up! Would you want your doctor to give you medications before

running tests and taking vitals? Would you buy a car if it was never tested by safety experts or

went through a multi-point inspection?

• Tools and Resources are available- use them, stay current, ask questions

You have a responsibility to collect data- No more excuses!


Come back for the rest of the Data Series!

Documents

Getting the Most out of Data Collection and Analysis: Part ...€¦ · > Data collection is the process of gathering and measuring information on targeted variables in an established