experience samplingdesign, data collection & analysis

Ben Richardson

experience sampling• a form of moment-to-moment

data collection• increased ecological validity• minimise retrospective bias• participant burden• different kinds of questions

experience sampling• a form of moment-to-moment

data collection• increased ecological validity• minimise retrospective bias• participant burden• different kinds of questions

experience sampling• a form of moment-to-moment

data collection• increased ecological validity• minimise retrospective bias• participant burden• different kinds of questions

experience sampling• a form of moment-to-moment

data collection• increased ecological validity• minimise retrospective bias• participant burden• different kinds of questions

experience sampling• a form of moment-to-moment

data collection• increased ecological validity• minimise retrospective bias• participant burden• different kinds of questions

Jeffrey S. Simons, Raluca M. Gaher, Matthew N.I. Oliver, Jacqueline A. Bush, Marc A. Palmer

An Experience Sampling Study of Associations between Affect and Alcohol Use and Problems among College Students

example

a quick note; I am focused on self report studies but passive data collection is also

possible

design considerations• appropriate measurement

resolution

couple satisfaction

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium

blood glucose monitoring

design considerations• appropriate measurement

resolution

event-based

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium interval-based

design considerations• appropriate measurement

resolution

!

Mass & Hox (2005) Sufficient Sample Sizes for Multilevel

Modeling

• rough rule of thumb: 50 individuals

• although power depends on many factors and is often most usefully estimated based on power analysis

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium

design considerations• appropriate measurement

resolution

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium

• honorarium

• usability

• length / frequency

• feedback

design considerations• appropriate measurement

resolution

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium

design considerations• appropriate measurement

resolution

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium

design considerations• appropriate measurement

resolution

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium

design considerations• appropriate measurement

resolution

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium

design considerations• appropriate measurement

resolution

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium PDAs

design considerations• appropriate measurement

resolution

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium

web surveys

resources for optimising web forms for mobile

• detecting whether participant is using mobile

• optimise webpage for iOS

design considerations• appropriate measurement

resolution

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium

mobile application

design considerations• appropriate measurement

resolution

• event-based versus interval-based response cues

• sample size and power

• engaging participants

• response medium

mobile application

analysis• main difference between ‘regular’ analysis and

analysis of ESM data is the hierarchical structure of the data

level 1: time points

analysis• main difference between ‘regular’ analysis and

analysis of ESM data is the hierarchical structure of the data

{ { { { { {

level 1: time points

level 2: individuals

analysis• multilevel modeling (MLM) addresses the lack of

independence between the observations

• can also use regression with robust standard errors

• in addition, MLM opens up some possibilities for some novel questions not so easily answered in single level analyses

example• using ESM to study risky single occasion drinking

• presentation that follows is mostly visual, do not take the diagrams too literally. for more comprehensive / technical overview of MLM as applied to ESM data please see

• Intensive Longitudinal Methods: An Introduction to Diary and Experience Sampling Research

• Models for intensive longitudinal data

intercept only model

risky drinking

fun seeking

level 1 variable

level 2 variableclustering variable = participant id

positive moodeveningpositive mood

intercept only• Intraclass correlation (degree of variance explained

in the outcome variable by the clustering / nesting variable)

intercept only• Intraclass correlation (degree of variance explained

in the outcome variable by the clustering / nesting variable)

rsod on positive mood

risky drinking

fun seeking

level 1 variable

level 2 variableclustering variable = participant id

positive mood

evening

positive mood

level 1 variables• level 1 variables actually capture two sources of

variance: • within participant variation (e.g., fluctuations around

an individual’s average level of mood) • between participant variation (e.g., individual

differences in level of positive mood)

• these are often usefully represented using separate variables in the model • achieved by person mean centring

level 1 variables

level 1 positive mood = score - person’s mean !

!

level 2 positive mood = individual’s average across time points

level 1 variables• fixed component of an effect

• average relationship between variables for all participants

• e.g., on average, how does positive mood relate to drinking? !

• random component • between participant variance in relationship • e.g., how much variation is there in the relationship

between positive mood and drinking? Does positive mood more strongly associate with drinking for some participants compared to others?

rsod on positive mood

risky drinking

fun seeking

level 1 variable

level 2 variableclustering variable = participant id

positive mood

evening

positive mood

level 2 moderators• can we explain variation in level 1 relationships

using level 2 variables?

• E.g., does an individual’s fun seeking explain variation in the relationship between positive mood and drinking?

some extensions

piecewise regressionsome resources

• http://www.ats.ucla.edu/stat/stata/faq/piecewise.htm

• http://www3.nd.edu/~rwilliam/stats2/l61.pdf

dose-response model• Hunt & Rai (2003). A threshold dose-response model with random

effects in teratological experiments. doi: 10.1081/STA-120021567

4.5

9

13.5

18

1 2 3 4 5 6 7 8 9 10 11 12 13 14

ControlDose

4.5

9

13.5

18

1 2 3 4 5 6 7 8 9 10 11 12 13 14

ControlDose

risk versus time to onset

photo credits

Couple photo: https://flic.kr/p/4SDwWz !"Couple in Covent Garden" by Mark Hillary (https://www.flickr.com/photos/markhillary/)!!

Diabetes photo!"My "kit"" by Jessica Merz (https://www.flickr.com/photos/jessicafm/)!!

Alcohol photo!"Alcohol and Ulcerative Colitis" by Kimery Davis (https://www.flickr.com/photos/117025355@N05/)!!

Timer photo!"Microwave Timer" by Pascal (https://www.flickr.com/photos/pasukaru76/)!!

PDA photo!"I Used To Be Cool..." by H. Michael Karshis (https://www.flickr.com/photos/hmk/)!!

Piecewise regression graph http://www3.nd.edu/~rwilliam/stats2/l61.pdf