35
Using Contextual Information to Understand Searching and Browsing Behavior Julia Kiseleva Eindhoven University of Technology Eindhoven, The Netherlands, June 2016

Using Contextual Information to Understand Searching and Browsing Behavior

Embed Size (px)

Citation preview

Page 1: Using Contextual Information to Understand Searching and Browsing Behavior

Using Contextual Information to Understand

Searching and Browsing BehaviorJulia Kiseleva

Eindhoven University of Technology

Eindhoven, The Netherlands, June 2016

Page 2: Using Contextual Information to Understand Searching and Browsing Behavior

Using Contextual Information to Understand

Searching and Browsing Behavior

Page 3: Using Contextual Information to Understand Searching and Browsing Behavior

Searching Behavior

Want to go to CIKM

conference

QUERY SERP

Page 4: Using Contextual Information to Understand Searching and Browsing Behavior

Browsing Behavior

User Preferences

Page 5: Using Contextual Information to Understand Searching and Browsing Behavior

Using Contextual Information to Understand

Searching and Browsing Behavior

Page 6: Using Contextual Information to Understand Searching and Browsing Behavior

Contextual InformationExplicit Context Implicit Context

Page 7: Using Contextual Information to Understand Searching and Browsing Behavior

Contextual InformationExplicit Context Implicit Context

Page 8: Using Contextual Information to Understand Searching and Browsing Behavior

Contextual InformationExplicit Context Implicit Context

Contextual Situations

(Android Tablet, Weekend)

Photo credit: Delwin Steven Campbell via Visualhunt.com / CC BY

Page 9: Using Contextual Information to Understand Searching and Browsing Behavior

Using Contextual Information to Understand

Searching and Browsing Behavior

Page 10: Using Contextual Information to Understand Searching and Browsing Behavior

Our Main Research GoalHow to

usecontextual information

in order tounderstand

users’ searching and browsing

behavior on the web?

Improve Online User Experience

Page 11: Using Contextual Information to Understand Searching and Browsing Behavior

Applied StudiesBrowsing Behavior

Page 12: Using Contextual Information to Understand Searching and Browsing Behavior

Destination Finder

Chapter 3 ‘Contextual Profiles’. L. Bernardi et al. The continuous cold start problem in e-commerce recommender systems. CBRecSys. 2015J. Kiseleva et al. Where to go on your next trip? optimizing travel destinations based on user preferences. SIGIR. 2015

Page 13: Using Contextual Information to Understand Searching and Browsing Behavior

Destination Finder

Chapter 3 ‘Contextual Profiles’. L. Bernardi et al. The continuous cold start problem in e-commerce recommender systems. CBRecSys. 2015J. Kiseleva et al. Where to go on your next trip? optimizing travel destinations based on user preferences. SIGIR. 2015

Page 14: Using Contextual Information to Understand Searching and Browsing Behavior

Destination Finder

Chapter 3 ‘Contextual Profiles’. L. Bernardi et al. The continuous cold start problem in e-commerce recommender systems. CBRecSys. 2015J. Kiseleva et al. Where to go on your next trip? optimizing travel destinations based on user preferences. SIGIR. 2015

Page 15: Using Contextual Information to Understand Searching and Browsing Behavior

Destination Finder

Optimized Ranking of DestinationsUsing Contextual Situations

Increased User Engagement (Click Trough Rate +3.7%)

Chapter 3 ‘Contextual Profiles’. L. Bernardi et al. The continuous cold start problem in e-commerce recommender systems. CBRecSys. 2015J. Kiseleva et al. Where to go on your next trip? optimizing travel destinations based on user preferences. SIGIR. 2015

Page 16: Using Contextual Information to Understand Searching and Browsing Behavior

Applied StudiesBrowsing Behavior

Page 17: Using Contextual Information to Understand Searching and Browsing Behavior

Applied StudiesBrowsing Behavior Searching Behavior

&

Page 18: Using Contextual Information to Understand Searching and Browsing Behavior

Changes in User Satisfaction

Want to go to CIKM

conference

QUERY SERP

Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015

Page 19: Using Contextual Information to Understand Searching and Browsing Behavior

Changes in User Satisfaction

QUERY SERP,Dynamic over Time

Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015

Page 20: Using Contextual Information to Understand Searching and Browsing Behavior

Changes in User Satisfaction

Time

Sati

sfac

tion

Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015

QUERY , SERP

Page 21: Using Contextual Information to Understand Searching and Browsing Behavior

Changes in User Satisfaction

Time

#

Refo

rmul

atio

ns~

Sati

sfac

tion

Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015

2013Oct NovSepAugJul

QUERY , SERP

Page 22: Using Contextual Information to Understand Searching and Browsing Behavior

Changes in User SatisfactionBefore November 2013

After November 2013

Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015

QUERY= ‘flawless’

Page 23: Using Contextual Information to Understand Searching and Browsing Behavior

Changes in User SatisfactionBefore November 2013

After November 2013

Chapter 7 ‘Query Reformulations’ and Chapter 8 ‘Failed SERPs’J. Kiseleva et al. Modelling and detecting changes in user satisfaction. CIKM. 2014J. Kiseleva et al. Behavioral dynamics from the SERP’s perspective: What are failed SERPs and how to fix them? CIKM. 2015

QUERY= ‘flawless’

Page 24: Using Contextual Information to Understand Searching and Browsing Behavior

Applied StudiesBrowsing Behavior Searching Behavior

&

Cortana:“What can I

help you do now?”

Page 25: Using Contextual Information to Understand Searching and Browsing Behavior

Q1: how is the weather in ChicagoQ2: how is it this weekendQ3: find me hotelsQ4: which one of these is the cheapestQ5: which one of these has at least 4 starsQ6: find me directions from the Chicago airport to number one

User’s dialogue with

Cortana:Task is

“Finding a hotel in

Chicago”

Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

Page 26: Using Contextual Information to Understand Searching and Browsing Behavior

Q1: find me a pharmacy nearbyQ2: which of these is highly ratedQ3: show more information about number 2Q4: how long will it take me to get thereQ5: Thanks

User’s dialogue with

Cortana:Task is

“Finding a pharmacy”

Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

Page 27: Using Contextual Information to Understand Searching and Browsing Behavior

Cortana: “Here are

ten restaurant

s near you”

Cortana:“Here are ten restaurants

near you that have good reviews”

Cortana:“Getting you direction to the Mayuri

Indian Cuisine”

User:“show restaur

ants near me”

User:“show

the best ones”

User:“show

directions to the second one”

Page 28: Using Contextual Information to Understand Searching and Browsing Behavior

Cortana: “Here are

ten restaurant

s near you”

Cortana:“Here are ten restaurants

near you that have good reviews”

Cortana:“Getting you direction to the Mayuri

Indian Cuisine”

User:“show restaur

ants near me”

User:“show

the best ones”

User:“show

directions to the second one”

No Clicks ??

?

Page 29: Using Contextual Information to Understand Searching and Browsing Behavior

Cortana: “Here are

ten restaurant

s near you”

Cortana:“Here are ten restaurants

near you that have good reviews”

Cortana:“Getting you direction to the Mayuri

Indian Cuisine”

User:“show restaur

ants near me”

User:“show

the best ones”

User:“show

directions to the second one”

SAT?

SAT?

SAT?

Overall SAT? ? SAT

?SAT

?SAT

?

Page 30: Using Contextual Information to Understand Searching and Browsing Behavior

Acoustic Similarity

Phonetic Similarity

Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

Page 31: Using Contextual Information to Understand Searching and Browsing Behavior

Tracking User Interaction

Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

Page 32: Using Contextual Information to Understand Searching and Browsing Behavior

3 seconds

6 seconds33% of

ViewPort 66% of

ViewPort

View

Port

H

eigh

t

2 seconds20% of ViewPor

t

1s 4s 0.4s 5.4s+ + =

Tracking User Interaction

Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

Page 33: Using Contextual Information to Understand Searching and Browsing Behavior

Quality of Interaction Model

Method Accuracy (%) Average F1 (%)Baseline 70.62 61.38

Interaction Model 80.81*(14.43)

79.08*(28.83)

* Statistically significant improvement (p < 0,05 )

Chapter 4 ‘Intelligent Assistants’ and Chapter 5 ‘Search Dialogues’J. Kiseleva et al. Understanding user satisfaction with intelligent assistants. CHIIR 2016J. Kiseleva et al. Predicting user satisfaction with intelligent assistants. SIGIR 2016

Page 34: Using Contextual Information to Understand Searching and Browsing Behavior

• Contextual information should be taken into account to understand web and mobile users’ behavior

• Analyzing behavioral signals over time is needed to detect changes in user satisfaction with web search

• Touch signals are crucial for inferring user satisfaction with intelligent assistants on mobile devices

Conclusion

Page 35: Using Contextual Information to Understand Searching and Browsing Behavior