Web Search ChallengesFebruary 2007David Rashty, david.rashty@gmail.com

Web Search Challenges Web Search Challenges • Where web search fails ?Where web search fails ?

• Search engines user interfaceSearch engines user interface

• Search engines trendsSearch engines trends

(1)

Where Web Search Fails ?Where Web Search Fails ?

How People Search How People Search • NavigationalNavigational – (find out what is the address of a – (find out what is the address of a

website)website) ‘How do I find the website of CNN’ ‘How do I find the website of CNN’

• FactualFactual – – (find exact information)(find exact information) “ “Population of Population of China; President Bush's email; Flights from NY China; President Bush's email; Flights from NY to Detroitto Detroit““

• ComprehensiveComprehensive – – (build a picture of a new world ) (build a picture of a new world ) ‘I need to understand the market around ‘I need to understand the market around wireless networking’, ‘I need to know more wireless networking’, ‘I need to know more about Leukemia about Leukemia

(2)

Search Skills Vary Significantly Search Skills Vary Significantly between Peoplebetween PeopleSome may succeed and some may fail, in locating what Some may succeed and some may fail, in locating what

they are looking forthey are looking for

Web +/- refers to Web expertise, Econo +/- refers to domain knowledge

From(Christoph Hölscher & Gerhard Strube, 2000), http://www9.org/w9cdrom/81/81.html

(3)

Only users who could rely both on high web expertise and high domain knowledge ("double experts") were able to solve an average of 3.2 out of the 5 tasks

(Christoph Hölscher & Gerhard Strube , 2000)

Scatter Nature of Information Scatter Nature of Information • Despite the existence of huge websites and Despite the existence of huge websites and

powerful search engines, novice users powerful search engines, novice users have have difficulty finding comprehensive informationdifficulty finding comprehensive information about even common topics. about even common topics.

• Users often retrieve incomplete information Users often retrieve incomplete information because of the because of the complex scatter of relevant complex scatter of relevant facts about a topicfacts about a topic across web pages across web pages (Bahavnani 2006)(Bahavnani 2006)

(4)

Information Density Information Density • General pagesGeneral pages contained many facts with contained many facts with

medium amount of detail (portals)medium amount of detail (portals)

• Specific pagesSpecific pages contained few facts with high contained few facts with high amount of detail (articles, expert sites)amount of detail (articles, expert sites)

• Sparse pagesSparse pages contained few facts with little contained few facts with little detail (references)detail (references)

(5)

What are Search Strategies ? What are Search Strategies ? • Online ResearchersOnline Researchers visit a combination of visit a combination of

sources, often in recognizable sequences, to sources, often in recognizable sequences, to find comprehensive information. Some of them find comprehensive information. Some of them are unreachable through the leading search are unreachable through the leading search engines.engines.

• The modus operandi of online researchers is The modus operandi of online researchers is determined by the fact thatdetermined by the fact that information is information is spread unevenlyspread unevenly; a large number of sources ; a large number of sources have very few facts, while a few sources have have very few facts, while a few sources have many (but not all) facts about a topic many (but not all) facts about a topic (Bhavnani, 2005)(Bhavnani, 2005)

(6)

Terms for Online Researchers Terms for Online Researchers • Searchers Searchers • InformationistInformationist• Advanced searcherAdvanced searcher• Information specialistsInformation specialists• Information professionalsInformation professionals• Search expertSearch expert• Search guruSearch guru

(7)

Searching for relevant information on the World Wide Web is often a laborious and frustrating task for casual and experienced users (Christoph Hölscher, Gerhard Strube, 2000)

Search ChallengeSearch Challenge• If users don't find the result with their first If users don't find the result with their first

query, they are progressively less and less query, they are progressively less and less likely to succeed with additional searches. likely to succeed with additional searches. Many users don't even bother… (Nilsen, 2002)Many users don't even bother… (Nilsen, 2002)

• Novice users rarely manage to perform a Novice users rarely manage to perform a comprehensive online researchcomprehensive online research

• They don’t understand the nature of They don’t understand the nature of information and they lack the strategies to help information and they lack the strategies to help them navigate thru information sourcesthem navigate thru information sources

(8)

JupiterResearch found that 71% of online consumers use search engines to find health-related information, but only 16% find the information they are looking for(ZDNet Research, June 2006)

Search UISearch UI

AltaVista 1995 AltaVista 1995

(9)

Google 1998 Google 1998

(10)

Google 2007 Google 2007

(11)

KartOO 2007 – Advanced UI ???KartOO 2007 – Advanced UI ???

(12)

Search UI ChallengeSearch UI Challenge

• Search engines UI didn’t change much in the Search engines UI didn’t change much in the last 10 years (web did change…).last 10 years (web did change…).

• Search engines UI does not reflect what is Search engines UI does not reflect what is known about user behavior.known about user behavior.

• 1,000,000……. results but only 30 are 1,000,000……. results but only 30 are currently useful.currently useful.

• Too much noise !!Too much noise !!

(13)

Search Engines TrendsSearch Engines Trends

Clusty 2007 (Clusty 2007 (clusteringclustering) )

(14)

Grokker 2007 (Grokker 2007 (clustering + visualizationclustering + visualization))

(15)

Rollyo 2007 (Rollyo 2007 (tailor made searchtailor made search))

(16)

MetaCrawler 2007 (MetaCrawler 2007 (combined searchcombined search) )

(17)

ChaCha 2007 (ChaCha 2007 (expert/community searchexpert/community search) )

(18)

Trexy 2007 (Trexy 2007 (strategiesstrategies))

(19)

Snap 2007 (Snap 2007 (improved UIimproved UI))

(20)

SearchMash 2007 (SearchMash 2007 (Google playgroundGoogle playground))

(21)

Swiki 2007 (Swiki 2007 (social searchsocial search) )

(22)

Search Trends ChallengeSearch Trends Challenge

• How do we combine all the relevant features How do we combine all the relevant features together without complicating the user together without complicating the user interface ?interface ?

• Will Google add more advanced features ?Will Google add more advanced features ?

(23)