Upload
emerson-tibbs
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Eye Tracking Analysis of User Behavior in WWW
Search
Laura GrankaThorsten Joachims
Geri Gay
why use eye-tracking for information retrieval?
• Understand how searchers evaluate online search results
• Enhanced interface design
• More accurate interpretation of implicit feedback (eg, clickthrough data)
• More targeted metrics for evaluating retrieval performance
Figure: popular regions are highlighted through shadow-intensity
key research questions• How long does it take searchers to select a
document?• How many abstracts do searchers look at
before making a selection?• Do searchers look at abstracts ranked lower
than the selected document?• Do searchers view abstracts linearly?• Which parts of the abstract are most likely to
be viewed?
what is eye-tracking?
• Device to detect and record where and what people look at
• Multiple applications: reading, usability, visual search, in both physical and virtual contextsEye tracking device
Figure: Cornell HCI eye-tracking configuration
View of subject’s pupil on monitor; used for calibration
ocular indices for www tracking
• Fixations: ~200-300ms; information is acquired• Saccades: extremely rapid movements between fixations • Pupil dilation: size of pupil indicates interest, arousal
Aggregate eye-tracking graphs depict viewing intensity in key regions
“Scanpath” output depicts pattern of movement throughout screen. Black markers represent fixations.
experimental search tasks • Ten search tasks
given to all participants
• Search topics included travel, science, movies, local, television, college, and trivia
• Searches evenly split between informational and navigational tasks
experimental procedures
• Users conducted live Google searches
• Users allowed to search freely, with any queries
• Script removed all ad content
• Proxy stored all pages and log files
Figure: Specific “zones” were created around each result, enabling eye-movements to be analyzed specific to the rankings
sample eye-tracking output
sample eye-tracking output
sample eye-tracking output
sample eye-tracking output
sample eye-tracking output
overall searching behaviorHow long does it take users to select a document?
Overall mean: 5.7 seconds, St.D: 5.4
Time spent before a result is clicked
less difficultmore difficult task
overall viewing behavior
Total number of abstracts viewed per page
0
20
40
60
80
100
120
1 2 3 4 5 6 7 8 9 10Total number of abstracts viewed
fre
qu
en
cy
Mean: 3.07 Median/Mode: 2.00
Most likely to view only two documents per results set
How many abstracts do we view, and in what order?How many abstracts do we view, and in what order?
Notice dip after page
break
overall viewing behaviorHow many abstracts do we view, and in what order?How many abstracts do we view, and in what order?
Instance of arrival to each result
0
5
10
15
20
25
1 2 3 4 5 6 7 8 9 10
Rank of result
mea
n f
ixat
ion
val
ue
of
arri
val
Results viewed linearly
overall viewing behavior
Time spent viewing each abstract compared with the frequency that each rank is selected. Error bars are 1 SE
How much time do we spend viewing each abstract?How much time do we spend viewing each abstract?
overall viewing behavior
Number of abstracts viewed above and below selected link
How thoroughly do we view the results set?How thoroughly do we view the results set?
overall viewing behaviorWhat information in the abstract is most useful?
Title: 30%Snippet: 43%Category: 0.3%URL: 21%
Other: 5% (includes, cached, similar pages, description)
Percentage of time spent viewing each part of abstract
Other: 5.3%
Title: 30.5%
Category: 0.3%Snippet: 42.8%
URL: 21.1%
overall searching behavior
*Difficulty and satisfaction are ranked on a 1-10 scale; 10 meaning very difficult and very satisfied, respectively
Search task difficulty and satisfaction with Google Search task difficulty and satisfaction with Google
less difficultmore difficult
overall searching behavior
Mean rank of selected doc: 2.66 Median/ Mode: 1.00
Task difficultyinfluences rank of selected document and number of abstracts viewed
less difficultmore difficult
overall searching behavior
1. Michael Jordan statistician 20
2. Thousand acres dude ranch 11
2. One thousand acres dude ranch 11
3. 1000 acres dude ranch 9
4. Time machine movie 7
4. Carnegie mellon university graduate housing 7
5. Imdb 6
5. Emeril lagasse 6
5. First modern antibiotic 6
5. Greyhound bus 6
5. Carnegie mellon graduate housing 6
Top Query Terms Frequency
conclusions Searching Trends: Popularity of specialized,
vertical portals
Several students preferred conducting a Google search from the cmu.edu homepage
Majority of students preferred an internal imdb.com search over a general Google search
conclusions
• Document selected in under 5 seconds• Users click on the first promising link they see• Results viewed linearly• Top 2 results most likely to be viewed• Users rather reformulate query than scroll• Task type and difficulty affect viewing
behavior• Presentation of results affects selection
future research Impact on advertising
With such fast selections being made, will searchers even view ads?
?
Ads most likely to be seen:• Difficult task• Ambiguous info need• Informational query• Low searcher expertise
future research
Relevance judgments
• Do we spent more time viewing relevant abstracts?• Do we click the first relevant abstract viewed?• Does pupil dilation increase for more relevant
documents
If results were re-ranked, wouldviewing behavior differ?
If results were re-ranked, wouldviewing behavior differ?
Cornell University Computer Science &
Human-Computer Interaction
Thorsten [email protected]
Laura [email protected]
Matthew [email protected]
Geri [email protected]
Helene [email protected]