Toward Fully Automated Person-Independent Detection of Mind Wandering

Robert Bixler & Sidney D’Mellorbixler@nd.eduUniversity of Notre DameJuly 10, 2013

mind wandering indicates waning attention

occurs frequently 20-40% of the time

decreases performance comprehension memory

solutions proactive

mindfulness training Mrazek (2013)

tailoring learning environment Kopp, Bixler, D’Mello (2014)

reactive mind wandering detection

our goal is to detect mind wandering

related work – attention Attention and Selection in Online Choice Tasks

Navalpakkam et al. (2012)

Multi-mode Saliency Dynamics Model for Analyzing Gaze and Attention Yonetani, Kawashima, and Matsuyama (2012)

distinct from mind wandering

mind wandering detection neural activity

physiology

acoustic/prosodic

eye movements

neural activity

Experience Sampling During fMRI Reveals Default Network and Executive System Contributions to Mind Wandering

Christoff et al. (2009)

physiology

Automated Physiological-Based Detection of Mind Wandering during Learning

Blanchard, Bixler, D’Mello (2014)

acoustic-prosodic

In the Zone: Towards Detecting Student Zoning Out Using Supervised Machine Learning

Drummond and Litman (2010)

eye movementsmindless reading

mindful reading

research questions1. can mind wandering be detected from eye

gaze data?

2. which features are most useful for detecting mind wandering?

4 texts on research methods self-paced page-by-page 30-40 minutes difficulty and value

auditory probes 9 per text inserted psuedorandomly (4-12s)

data collection

type of report

yes no total

end-of-page

209 651 860

within-page

1278 2839 4117

total 1487 3490 4977

tobii tx300

1. compute fixations OGAMA (Open Gaze and Mouse Analyzer)

(Voßkühler et al. 2008)

2. compute features

3. build supervised machine learning models

data analysis

global

local

context

features

global features eye movements

fixation duration saccade duration saccade length

fixation dispersion reading depth fixation/saccade ratio

local features reading patterns

word length hypernym depth number of synonyms frequency

fixation type regression first pass single gaze no word

context features positional timing

since session start since text start since page start

previous page times average previous page to average ratio

task difficulty value

supervised machine learning parameters

window size (4, 8, or 12) minimum number of fixations (5, 1/s, 2/s,

or 3/s) outlier treatment (trimmed, winsorized,

none) feature type (global, local, context,

combined) downsampling feature selection

classifiers (20 standard from weka)

leave-several-subjects-out cross validation (66:34 split)

1. can mind wandering be detected using eye gaze data?

End-of-page Within-page0

0.050.1

0.150.2

0.250.3

best model kappas

report type

kapp

a

1. can mind wandering be detected using eye gaze data?

End-

of-pa

ge

Within-

page

4045505560657075

AccuracyExpected Accuracy

accu

racy

%

1. can mind wandering be detected using eye gaze data? confusion matrices

end-of-page within-pageactual response

classified response

prior

yes noyes .54 .46 .23

no .23 .77 .77

actual response

classified response

prior

yes noyes .61 .39 .36

no .42 .58 .64

2. which features are most useful for detecting mind wandering?

End-of-page Within-page0

0.1

0.2

0.3

average kappa values across feature types

GlobalLocalContextGlobal + Local + Con-text

report type

kapp

a

2. which features are most useful for detecting mind wandering?

rank

end-of-page within-page

1 previous value saccade length max2 previous difficulty saccade length

median3 difficulty fixation duration

ratio4 value saccade length

range5 saccade length

maxsaccade length mean

6 saccade length range

saccade length skew

7 page number fixation duration median

8 saccade length sd fixation duration mean

9 saccade length mean

saccade duration mean

10 saccade length skew

saccade duration min

summary mind wandering detection is possible

kappas of .28 to .17 end-of-page models performed better

global features were best exception: context features highest ranked

for end-of-page

enhanced feature set global

pupil diameter blink frequency saccade angle

local cross-line saccades end-of-clause fixations

enhanced feature set

End-

of-pa

ge

Within-

page

0.1

0.15

0.2

0.25

0.3

OriginalEnhancedka

ppa

predictive validitymw rate post

knowledge

transfer learning

end-of-page predicted -.556 -.415 actual

(model)-.248 -.266

actual (all data)

-.239 -.207

within-page predicted -.496 -.431 actual

(model)-.095 -.090

actual (all data)

-.255 -.207

self-caught mind wandering

End-

of-pa

ge

Within-

page

Self-

Caug

ht0

0.10.20.3

self-caught vs. probe caught

report type

kapp

a

what does mind wandering look like? saccades

slower shorter

more frequent blinks

larger pupil diameters

limitations eye tracker cost

population validity

self-report

classification accuracy

future work multiple modalities

different types of mind wandering

mind wandering intervention

acknowledgements Blair Lehman Art Graesser Jennifer Neale Nigel Bosch Caitlin Mills

questions

?