Counting How Many WordsYou Read

Reading Habit

• Quantifying reading habit– Provides insights on language skills, effective

learning and critical thinking– Identifying a reading log, time spent on reading at

different time of day may provide these insights• Electrooculography and optical eye tracking

provide the opportunity to track reading

Motivational Applications• Quantifying reading habit– Might help to identify the children with reading

disabilities– People can increase their reading speed without

any problemQuantifying self

Contributions

• Quantifying how much words a user reads working for both optical eye tracking and electrooculography

• Lowest error rate between 5 – 15% for word count estimation

• Initial hint that these method can work with commercial smart glasses

Fixation

• Fixation or visual fixation is the maintaining of the visual gaze on a single location

https://www.youtube.com/watch?v=kiFpMbfj_08

Saccade• Saccade– quick, simultaneous movement of both eyes

between two phases of fixation in the same direction

– can be associated with a shift in frequency of an emitted signal or a movement of a body part or device

• Alternate saccades and visual fixations,

https://www.youtube.com/watch?v=kiFpMbfj_08

Eye Tracking• Electrooculography

– Uses electrodes– Measures change in potential when eye moves as eye is considered as

a diphole between cornea and retina– Cheap, but gives relative eye movements

Eye Tracking• Optical Tracking– Uses cameras and infrared light to track eye gaze– Binocular gaze estimates at a sampling freq. 30Hz– Video resolution with resolution 1280X960 pixel– Provides higher accuracy but requires high power to infer

eye position, motion and gaze based on iris shape

Provides raw eye gaze data (Saccade and fixation data)

Proposed Approach

Proposed Approach

• Preprocessing• Reading detection• Line-break detection• Word count estimation

Preprocessing

• Optical tracking– Combines several close by fixations into larger

duration fixation

Reading Detection

• Calculate these features in 3 sec framing window • Apply a SVM classifier• For EOG, blink duration and frequency is also used as a

feature• For optical system mean fixation duration and

variance of fixation count is used

Line-break Detection• Detection of the dynamic line breaks using the distribution of

the horizontal component of saccade

• Reading is dominated by two types of saccades – short one in reading direction – Longer saccade against the reading

• Two steps– Combine consecutive saccade against reading

direction– Horizontal Saccade Direction (HSD) histogram

Line-break Detection• Dynamic line break detection• Saccade amplitude sa• Horizontal direction component sdh-1 for the against main reading and +1 for the in reading

• Horizontal saccade direction componentHSD = sa * sdh

• HSD distribution is noted next

Line-break Detection• Combine all saccades, calculate HSD and combine them in

HSD histogram• Fit a mixture of 2 Gaussians and take distance between two

maximas

Line-break Detection• Two types of saccades– Short– Long

• larger one is the average reading saccade direction * amplitude

• Second maximum average line break saccade direction

Line-break Detection• Two steps– Combine consecutive saccade against reading

direction

Determine line break

Word Count Estimation• Basic Word Count– (1)Avg. word count from the document read time – (2) Estimate from line breaks • Uses the information – avg. words per line

– Words per line – easy for electronic system– For paper--------image retrieval technique

Word Count Estimation• Basic Word Count

– (1)Avg. word count from the document read time – (2) Estimate from line breaks

• Uses the information – avg. words per line

– Words per line – easy for electronic system– For paper- image retrieval technique

• SVR count– Uses features

• total time read, • sum of all saccades distance, • sum of the line break saccade distances, • number of line breaks and • sum of the reading saccade distances

– Train a SV regression

Experimental Setup• Mobile optical eye tracking– Mobile eye tracker 2.0– Binocular gaze estimates at a sampling freq. 30Hz– Video resolution with resolution 1280X960 pixel– Can capture saccades of 33 ms or slower

Experimental Setup• Baseline Experiment• Eye tracking glass– Office scenario, 9 subjects– 14 documents to read– 10 simple text (PET) , 4 difficult (SAT)– Word length 135 to 414 (mean 245)– Eye gaze and scene camera are recorded– Comprehensive questions • Text understanding

Experimental Setup• Devices Experiment– Impact of device type and document length on word

count accuracy– 5 documents of different length read on different

devices (size)– paper, smartphone, computer screen, tablet---same font 12

– Varying line length – Office scenario, 10 subjects– 5 documents to read– Word length 135 to 414 (mean 245)– Perform other activities (solving sudoku, talking,

playing games)

Experimental Setup• Electrooculography• 8 participants• Active electrodes • Sampling rate 1kHz

Results – Optical Tracking• Evaluation- Leave one out (of n users)• Reading detection is accurate • Line break detection

– HSD Histogram for line break detection on different devices– For baseline dataset, line break method reduces error from 15%

to 6% – Device dataset – 62% to 8%

• Word-count estimation

Results – Optical Tracking

Change in line length Line break

Reading from the devices is challenging – high error

• Line break and Word-count estimation

• EOG performs better than Mobile tracker

Results – EOG

Commercial eyewear – JINS MEME4 participants, 3 days

https://jins-meme.com/ja/

Application