Embed Size (px)
Citation preview
Evaluating the Practical Applications
of Eye Tracking in Museums
Evaluating the Practical Applications
Ed Bachta & Silvia Filippini-Fantoni
Exploring Visitors’ Engagement
With Artworks
Research results
Research so far has indicated that visitors spend little timelooking at artworks:
• Hein, 1998
• Smith, 2001
• Worths, 2003 (‘grazing’)
• Viewing Project, IMA 2012
• Use of observational rubric is labor-intensive and not always precise
• Eye tracking technology has the potential of being more precise and less time-consuming
• More recent development of less intrusive devices (non head mounted)
Observation vs. eye tracking
Photo from Milekic MW 2010
• Gauge the practicality of using such devices in a museum gallery setting.
• Assess the ability of current eye tracking technology to reveal what visitors are looking at and for how long.
• Explore the potential use of this equipment in a practical setting (e.g. VTS discussion)
Sparks! grant objectives
Infrared Emitters
Camera
EyeTech VT2
• Eye tracker range limitations.
• Too much variation in height when the person is standing.
• For the experiments the viewer has to be seated, with the tracker placed in a fixed position between the tracker and the painting.
Device initial testing
Experiment 1© Edward Hopper.
• Distinguish when the participant looks inside/outside of the painting.
• Measure time spent looking inside/outside of the painting.
• Track where looking inside the field of the artwork.
Calibration performed once
Experiment 1: objectives
The device was installed on a cart between the work of art and the seated participant and calibrated to the first participant.
Participants’ standing and seated height were measured and distance from mid-eye to floor.
• 22 participants were asked to look in and outside the painting for 1 minute.
• First 10 participants could not adjust their chair position to optimize eye tracking, while the next 12 were asked to do so.
• Participants’ gazes (inside the field of the painting) were tracked by 2 research assistants with stopwatches.
• The times were averaged and compared to the time tracked by the device
Experiment 1: part 1
• A subset of participants (8 of the 22) were asked to look (over a period of 60 seconds) at 6 different areas of the work for 10 seconds each in sequence prompted by a research assistant.
• Tracker data was logged in the same manner as the previous experiment.
Experiment 1: part 2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22-80
-70
-60
-50
-40
-30
-20
-10
0
10
20
diffe
renc
e (%
of s
essi
on ti
me)
Relative quantity of valid gaze dataMissing data for >25% of session time for 6 participants
Fixed seat position Adjusted seat position
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22-80
-70
-60
-50
-40
-30
-20
-10
0
10
20
diffe
renc
e (%
of s
essi
on ti
me)
Relative quantity of valid gaze dataPoor performance for 4 of 6 participants at low eye level (<50”)
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22-80
-70
-60
-50
-40
-30
-20
-10
0
10
20
diffe
renc
e (%
of s
essi
on ti
me)
Relative quantity of valid gaze dataMissing > 10% for 7 of 10 glasses wearers
Comparison against manual measurement
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 220
102030405060708090
100
Participant
erro
r (%
of s
essi
on ti
me)
Comparison againstmanual measurement
Fixed seat position Adjusted seat position
Within 5% 0 5
Within 10% 1 7
• Allowing the participant to adjust the seat position produces better results
• We were still hoping for better accuracy
Gap Handling
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 220
102030405060708090
100
Raw100ms500ms1s
Participant
erro
r (%
of s
essi
on ti
me)
Gap Handling
• Applying a gap algorithm appears to improve results with a 500ms threshold
• Accuracy was within 10% for two thirds of participants when allowing the seat position to be adjusted and using the algorithm
Raw 100ms 500ms 1s
Within 5% (fixed) 0 1 1 1
Within 10% (fixed) 1 3 3 4
Within 5% (adjusted) 5 5 8 7
Within 10% (adjusted) 7 7 8 8
Gaze locations (calibrated)
Gaze locations (calibrated)
Gaze locations (typical uncalibrated)
• Device was not able to continuously track the gaze of a seated viewer.
• An attempt to improve results by handling gaps in the data were successful, but only to a degree.
• Vertical gaze location was not accurate for uncalibrated viewers.
Experiment 1: Summary
Experiment 2
• Measure whether the device could be more precise when calibrated for each participant
• We repeated experiment 1 (part 1 and 2) with 12 participants but calibrated the devices individually
• This second experiment was set in a lab, where the exact size of the painting was reproduced on a board
Experiment 2: objectives& methodology
Relative quantity of valid gaze data
1 2 3 4 5 6 7 8 9 10 11 12
-20
0
20
40
60
80
100
diffe
renc
e (%
of s
essio
n tim
e)
Gaze DurationDelta
(% of session time)Max. calibration
“score”Glasses Seated eye
elevation
7.719 6.70 did without glasses 4915.159 2.76 yes 51.56.306 13.90 no 546.728 6.13 yes 525.164 3.30 no 51.25
10.071 6.78 no 480.007 5.13 no 53.756.082 4.07 did without glasses 47.754.572 4.36 yes 50.52.732 10.10 no 52
12.240 12.15 yes 49.53.790 3.77 no 51
Gaze DurationAccuracy Raw data 100ms threshold 500ms threshold
Within 2% 1 4 6
Within 5% 4 7 8
Within 10% 9 12 12
• An improvement over the first experiment
• One third of participants were in the 5-10% range
Gaze LocationBest session
Gaze LocationWorst session
ComparisonsAverage error
(degrees of FOV)Max. calibration “score” Glasses
0.96 6.70 no1.60 2.76 yes1.72 13.90 no2.47 6.13 yes0.88 3.30 no1.61 6.78 no0.90 5.13 no1.24 4.07 no1.18 4.36 yes1.76 10.10 no3.00 12.15 yes2.22 3.77 no
• Applying the gap handling algorithm brought all sessions within 10% of the manual measurement
• Gaze duration results were better than in the uncalibrated study, but still not what we hoped for
• Gaze location results were also better than in the uncalibrated study, but not as accurate as expected
Experiment 2: Summary
• Experiment 3 will make use of the tracker during a VTS session
• We will evaluate whether the data recorded assists in understanding what VTS participants look at during a session
Future Work
Photo from PAAM.org
