EVALUATING USERS EXPLORATORY INFORMATION SEEKING …dagda.shef.ac.uk/dispub/dissertations/2010-11/External/... · 2012. 3. 27. · i ABSTRACT Background The increase in number of

EVALUATING USERS EXPLORATORY INFORMATION

SEEKING BEHAVIORS

A study submitted in partial fulfilment of the requirements for the degree of Master

of Science in Information Systems

At

THE UNIVERSITY OF SHEFFIELD

By

MADHUSUTHANAN NITHYANANDHAM

September 2011

i

ABSTRACT

Background The increase in number of internet users searching through the web for their

various information needs has led to increase in number of exploratory search activities due to

which users face the problem of uncertainty of their search goal. Therefore there is need of

search system to support efficient information retrieval by users for their exploratory search

tasks. Previous researchers have identified and developed some systems to support users’

exploratory search but the search system should support and be part of daily search activities of

people.

Aims The aim of this research study was to identify and analyze the exploratory search behavior

of users, evaluate the existing search systems to identify the support provided by those systems

for users exploratory search, analyze users exploratory searching behavior to identify the role of

domain knowledge for exploratory search and to design an alternative interface that supports

better and easier exploratory search.

Methods A survey questionnaire was designed and piloted with two of my friends. The final

survey questionnaire was distributed in-person, online through email and social networking site

Facebook to selected sixteen participants. Experimental search task was conducted followed by

distribution of evaluation questionnaire and interview session. Finally evaluation of designed

alternative interface was carried out followed by interview session.

Results Every internet users have an experience of exploratory search. Most of them are not

aware of exploratory search systems and they use only search engines for their exploratory

search. But neither of the systems supports their exploratory search better. Most users preferred

query suggestions and view-based search and among the techniques to support exploratory

search most of the users preferred tags and faceted interface. The alternative interface was found

useful by users and support better exploratory search.

Conclusions The analysis of results and findings and development of alternative interface based

on users’ feedback and future suggestions will support better and easier exploratory search.

ii

ACKNOWLEDGEMENT

I would like to express my gratitude to my supervisor Prof. Nigel Ford for providing me support,

encouragement and guidance throughout the course of this dissertation. Also I would be grateful

to Dr. Paul Clough who provided me motivation to take up this study.

I would like to thank all participants who participated in various research methods of this study

by dedicating their precious time.

I am thankful to my parents and friends for their constant support and encouragement throughout

my studies.

I dedicate my dissertation to my “Parents, Friends and Teachers”.

iii

TABLE OF CONTENTS

S. No. Contents Page No

1 INTRODUCTION 1

1.1 Introduction 2

1.2 Background and Motivation 3

1.3 Aim and Objectives 4

1.4 Research Questions 5

1.5 Outline 5

2 LITERATURE REVIEW 6

2.1 Introduction 7

2.2 Exploratory Search 8

2.3 Exploratory Search Systems (ESSs) 9

2.3.1 Social Tagging Systems 9

2.3.2 Visualization Techniques 12

2.3.3 Surrogates 14

2.3.4 Faceted Interfaces 15

2.3.5 Semantic Web 20

2.3.6 Relational Browsers 23

2.3.7 Clustering Systems 25

2.4 Exploratory Search Methods 28

2.4.1 Iterative Methods 28

2.4.2 Query by Example 28

2.4.3 Keyword Refining Through Feedback 28

2.4.5 View Based Search 29

2.5 Role of Domain Knowledge for Exploratory Search 29

2.6 Analysis of Literature Review 30

iv

3 METHODOLOGY 32

3.1 Introduction 33

3.2 Research Methods 33

3.2.1 Experimental Research Methods 33

3.2.2 Optimal Based Research Methods 34

3.2.3 Observational Research Methods 35

3.3 Research Approaches 36

3.4 Research Strategies 38

3.5 Methods Used in this Research Study 38

3.6 Procedure 39

3.7 Questionnaire Design 40

3.7.1 Self-completion Questionnaire 40

3.7.2 Question Styles 41

3.7.3 Presentation of Questionnaire 42

3.8 Pilot Study 43

3.9 Sampling 43

3.10 Participants Recruitment 44

3.11 Distribution of Survey Questionnaire 44

3.12 Survey Questionnaire Questions 45

3.13 The Experiment 46

3.13.1 The Exploratory Search Task 48

3.13.2 Evaluation Questionnaire 49

3.13.3 Evaluation Questionnaire Questions 49

3.13.4 Interview 50

3.13.5 Interview Questions 50

3.14 Alternative Interface for Exploratory Search 50

3.14.1 Evaluation of Alternative Interface 51

v

3.15 Data Analysis 51

3.16 Ethical Aspects 51

3.17 Practicalities 52

4 RESULTS AND FINDINGS 53

4.1 Survey Questionnaires 54

4.2 Evaluation Questionnaires 71

4.3 Interview 75

4.4 Role of Domain Knowledge for Exploratory Search 77

4.5 Summary of the Findings 77

4.6 Alternative Interface for Exploratory Search 80

4.6.1 General Layout 80

4.6.2 Search Options 81

4.6.3 Keyword Suggestions 81

4.6.4 Search within Results 81

4.6.5 Advanced Search Options 82

4.7 Evaluation and User Feedback on Alternative Interface 83

4.8 Limitations of the Study 84

5 CONCLUSION 85

5.1 Conclusions 87

5.2 Implications of the Study 90

5.3 Suggestions for Future Work 90

BIBLIOGRAPHY 91

APPENDICES 98

APPENDIX 1 98

APPENDIX 2 100

APPENDIX 3 102

vi

LIST OF FIGURES

Page No

Figure 2.1 Exploratory Search Activities 8

Figure 2.2 Dogear social bookmarking service 11

Figure 2.3 Mr Taggy interface 11

Figure 2.4 TagSphere 13

Figure 2.5 Informedia Storyboard- TRECVID 2006 15

Figure 2.6 mSpace faceted column browser 18

Figure 2.7 SERVICE 18

Figure 2.8 PunchStock 19

Figure 2.9 Flamenco – Hierarchical Facet navigation 19

Figure 2.10 FacetBrowser 20

Figure 2.11 Bletchley Park Text 22

Figure 2.12 LED interface 23

Figure 2.13 Relational Browser 24

Figure 2.14 Relational Browsers ++ Interface 25

Figure 2.15 IGroup 27

Figure 3.1 The Research Wheel 33

Figure 3.2 Google 47

Figure 3.3 Delicious 48

Figure 4.1 Usage of Internet 55

vii

Figure 4.2 Experience of Exploratory Search 56

Figure 4.3 Problems faced by users for their Exploratory Search 57

Figure 4.4 Users awareness of Exploratory Search Systems (ESSs) 59

Figure 4.5 Preference of systems by users for Exploratory Search 60

Figure 4.6 Ability to use Search Engines 62

Figure 4.7 Ability to use Exploratory Search Systems 63

Figure 4.8 Preference of Search Engines by users for Exploratory Search 64

Figure 4.9 Approach to support user’s exploratory search 65

Figure 4.10 Support by Search Engines or ESSs for exploratory Search 67

Figure 4.11 User’s awareness of various techniques to support exploratory search 69

Figure 4.12 Technique to support exploratory search 70

Figure 4.13 Alternative interface: General Layout 81

Figure 4.14 Alternative interface: Search Options 82

Figure 4.15 Alternative interface: Advanced Search Options 83

viii

LIST OF TABLES

Page No

Table 4.1 Usage of Internet 54

Table 4.2 Experience of Exploratory Search 55

Table 4.3 Problems faced by users for their Exploratory Search 57

Table 4.4 Users awareness of Exploratory Search Systems (ESSs) 58

Table 4.5 Preference of systems by users for Exploratory Search 60

Table 4.6 Ability to use Search Engines 61

Table 4.7 Ability to use Exploratory Search Systems 62

Table 4.8 Preference of Search Engines by users for Exploratory Search 63

Table 4.9 Approach to support user’s exploratory search 65

Table 4.10 Support by Search Engines or ESSs for exploratory Search 67

Table 4.11 User’s awareness of various techniques to support exploratory search 68

Table 4.12 Technique to support exploratory search 70

1

CHAPTER 1

INTRODUCTION

Evaluating Users Exploratory Information Seeking Behaviors

Madhusuthanan Nithyanandham

2

1.1. Introduction

The development of World Wide Web and internet has led to increase in number of people

searching web for different daily purposes and database storage and retrieval ranges from

database to search engines. There is search for information needs by professionals in their

workplace to develop knowledge for their work practice to common people for their various

daily activities. As a result, problem of effective retrieval of information is a concern for various

research studies and various search systems are developed during these days to satisfy people

information needs.

According to Internet World Stats (2010) by Miniwatts Marketing Group there are

1,966,514,816 (estimated) internet users in the world which is 28.7% of overall world

population. Every internet users search the web through various search systems to gather their

desired information. Some search activities will have undefined goal and searchers does not have

proper domain knowledge about search subject in which they does not aim to obtain specific

information but aims to gather related information about a subject from various sources to

develop their knowledge. This form of searching behavior leads to exploratory search. Examples

of exploratory searches are finding information about foreign trip, users finding particular

information in website for first time and browsing best book seller lists to buy a book. Also

domain knowledge is important for searchers to find exact information they need.

Traditional search systems require users to have a definite search goal and to submit a

definite query relevant to their search to initiate a search process and to get desired results. These

types of search activities are not possible for exploratory search which arises when users are

unfamiliar which keywords to use for search, unaware how to find the information and

unfamiliar where to find the information. Mostly the submitted keywords may not produce

expected results. Also this will be time consuming as searchers need to search through different

ways and when they search in particular website they need to learn about that website better

before searching process. There is broader scope of research in this due to ability of some users

to perform only keyword search and when the user is unaware about which keyword to use for

search.



3

1.2. Background and Motivation

Exploratory searchers will have uncertainty of their search goal as they aim to collect

broad information about subject. Exploratory searchers submit tentative query to collect any

general information about subject and they explore search results for their next step by refining

their search terms and by modifying their search goals. They need to collect, analyze and

integrate information from various sources to get the information they need. Exploratory

searchers are unsure about the quality of information they obtain from search systems as there is

need for them to assure and interpret documents returned by search systems for their search

query. Exploratory search is iterative process that requires users to search through series of

queries and it depends on the ability of user to analyze and navigate the search results to number

of new and previously visited pages to obtain the desired results (Bates, 1989). But these types of

search activities such as refining queries for iterative search process by users, processing the

search results by analyzing and examining the documents returned by search systems is not

possible by exploratory searchers as they lack knowledge about their search subject and have an

ill defined search goal.

Researchers have identified and developed some search systems and techniques like social

tagging systems, exploratory search systems, visualization technique, exploration map, faceted

interface which will help exploratory search. Each technique has advantages and limitations.

Also there is need for search systems to support effective retrieval of desired information by

users for their exploratory search task. Therefore there is necessary for development of new

systems or technique that will support better exploratory search and that satisfy exploratory

searchers.

As a result, this project aims to study the query methods, search options and presentation

and organization of results provided by existing search systems and to identify strengths and

improvements needed in those systems and design an effective interface system to satisfy users

exploratory search needs. The interface will be designed based on the users’ preference of search

systems, evaluation of existing search systems by users for their exploratory search task and

review of existing literature on search systems to support exploratory search.



4

1.3. Aim and Objectives

The aim of this dissertation is to analyze the users’ exploratory information seeking

behavior, evaluate existing search systems and the support provided by those systems to user’s

exploratory search task to identify whether it satisfies users information exploratory search needs

and by identifying possible improvements to design an alternative interface that supports better

and easier exploratory search even for common users. Therefore during the literature review

process we review at all existing search engines and exploratory search systems and the proposal

by some researchers and evaluate existing search systems using survey questionnaires,

experimental evaluation and interviews. We try to identify the benefits of each proposed system

and also limitations of those systems. Therefore by this research we try to recognize the best

technique that will support exploratory search by identifying changes and improvements in

existing search systems and propose and design an interface to support better and easier

exploratory search. The alternative interface that will be designed based on existing literature and

based on analysis of data collected from users will be evaluated by users to provide feedback on

the support provided by alternative interface for their exploratory search tasks. Also we try to

identify the role of domain knowledge in exploratory information seeking. There were many

researches going on to identify the role of domain expertise in the information seeking behavior

which is the level of education about domain of searching process. The objective of this

dissertation is to identify and analyze users’ experience of exploratory search, their awareness of

present search engines and exploratory search systems, their experience with those systems and

their preference of those search systems, techniques and approaches for their exploratory search

task. Also this study evaluate the existing search systems to identify the support provided by

those systems to users exploratory search and to analyze users exploratory searching behavior in

search engine and ESS to identify the role of domain knowledge for exploratory search and

design an alternative interface that will support better exploratory search task and to evaluate the

support provided by alternative interface design for users exploratory search.



5

1.4. Research Questions

1. What will be the searching behavior of users for exploratory search task?

2. To what extent the existing search systems/tools support exploratory search?

3. What is the role of domain knowledge in exploratory search, how did it influence the

search and the information collected?

4. Would the proposed interface technology make change and support exploratory search

behavior?

1.5. Outline

Chapter 2 describes the background of exploratory search and its challenges. It also describes the

previous studies of various search engines and exploratory search systems, its merits and

demerits along with approaches and techniques.

Chapter 3 describes the methodology used in this research study along with justification of

choosing adopted methodology. It also describes the data collection methods, data analysis

procedures, ethical issues and practicalities associated with this research study.

Chapter 4 describes the results and findings from data collected through various adopted data

collection methods. The results and findings were analyzed in the context of existing literature

and findings were summarized. It also describes the alternative interface design and summaries

users’ feedback on alternative interface design.

Chapter 5 presents the conclusion of this research study. It also explains the limitations of this

study and suggestions for future work with respect to this research study.



6

CHAPTER 2

LITERATURE

REVIEW



7

2.1. Introduction

Most of the daily activities of people involve searching for information through the

internet. The internet provides communication and exchange of information between people.

Some of the issues rose due to growth of internet are information overload, information

unavailability, navigation problems and diversity of different users. The user’s web search

session involves submitting a query, selecting suitable link from returned results and browsing

the link page, navigating the web page and modifying their query if they are unsatisfied with the

acquired information. Some of the search of the people will have clear goal to achieve but most

of them will have unclear goal and unaware about the exact location to find the information and

unsure about how to find the exact information. The most used keyword-based search is offered

by many search engines such as Google, Bing etc. in which users query through set of keywords

for their search activity along with advanced search option. But the keyword-based search and

advanced search option is difficult and complex due to difficulties in effective query formulation,

modification using suitable keywords to obtain desired result and unsuitable to explore

information space. JcsCholtes (2010) differentiated search in to two main types – Focalized and

Exploratory. According to him Focalized search also known as web-search or portal search, in

which users will have well defined goal for their search behavior and looks for particular best

result. Focalized search are suitable only for personal search and does not help in investigation of

a topic as focalized search lack navigation of relevant results. In Exploratory search, users will

be unfamiliar with the goal, unsure about goal and how to achieve the goal.

Marchionini (2006) explained the three kinds of search of search activities such as

Lookup, Learn and Investigate as shown in Figure 2.1 and outlined the process carried out during

each activity. Lookup activity is supported by most of the search engines in which users are

aware of their search goal and they give specified query to yield selected discrete results.

Searching through search engines displays results ranked in any order but exploratory search

which is less directed requires search results to be well organized to obtain better information

about search topic. Cove and Walsh (1988) classified browsing in to search browsing in which

browsing is directed and users have clear goal to achieve, general purpose browsing in which

browsing is based on specific topic or area of interest and serendipity browsing in which

browsing is a unstructured activity. Exploratory browsing involves general purpose and



8

serendipity browsing in which users will have specific area of interest but does not have specific

goal to achieve. Marchionini (2006) explained exploratory search as search to learn and

investigate in which users browse and investigate information in multiple search sessions to

acquire broader information which cannot be satisfied by the relevant page result that will be

acquired through focalized search. Exploratory search combines querying and browsing that help

will help users to investigate particular topic by navigation and discovery of relevant results.

Figure 2.1 – Exploratory Search Activities (Marchionini, 2006)

2.2. Exploratory Search

Exploratory search grown from the field of Information Retrieval and Information Seeking

and covers various activities such as investigating, evaluating, comparing and synthesizing.

Exploratory search are carried out in everyday life by exploring new information space to gather

the knowledge about a domain which help them make a decision. Exploratory search task

requires using more queries to search, remembering and revisiting previously visited web pages,

visit new domain areas thus requires more workload to be carried out for the task (Park et al.,

2008). Thus exploratory search requires people to submit a tentative query to navigate a

collection of related information, to explore the obtained information to obtain cues about their

next step that helps to perform multiple query iterations to obtain a broader class of information.



9

2.3. Exploratory Search Systems (ESSs)

Exploratory Search Systems (ESSs) such as information visualization systems, document

clustering and browsing systems, and intelligent content summarization systems support decision

making through and maximize information gain through information foraging (White et al.,

2008). ESSs help users to make decisions about which hyperlinks to follow for the next step of

their exploratory search process and help them to refine their searching for the shortest path to

achieve the solution for the search topic. According to White et al. (2006), effectiveness of

exploratory search systems should be evaluated based on measurement of interaction behavior,

cognitive load and learning that provided by the systems.

2.3.1. Social Tagging Systems

Social bookmarking or tagging systems support simple search where the users are aware of

their search goal and domain to exploratory search where users are unaware of their search goal

and domain. Tags are associated with every data objects such as photos, URLs, blogs, articles,

audios and videos that help users to organize, access the data objects and retrieve them

effectively when shared between different users. In simple or known-item search users search

through specific queries and analyze the search results to find appropriate bookmarks to gain the

specific information needed to them. Millen et al. (2007) explained the support provided by

social bookmarking applications for known-item search in which users browse through

bookmarks or query through tags for personal bookmark browsing and the support provided by

social bookmarking applications for exploratory search in which users browse through

bookmarks according to time and bookmarks which is popular. Social tagging systems are most

desirable sometimes because social tags from other user’s acts as navigation cues that help in

information acquisition and facilitate exploratory search (Millen, et al., 2007). Social tagging

systems provide better social information environment in which tags also interpreted as

folksonomies provides information cues from others user’s that helps to learn and acquire

knowledge to find the correct information. Customized tags help individual for future research

and social tags helps other users to search information with similar interest (Kammerer, et al.,

2009). The information contained in the tags helps the users to find the relevant concepts and

resources and the inferred topics helps to find the related contents. Thus social tagging systems



10

help in topic inference process and topic extraction process thus acts as semantic imitation model

as explained by Fu (2008). Therefore various features such as creating bookmarks and sharing it

with others, use of tags or keywords for bookmarks that helps users to identify the bookmark and

classify the bookmarks in to collections and social use of system are supported by number of

social bookmarking systems. The problem in social tagging systems is that social tags generate

noise as users have freedom to choose any keyword as tag (Chi and Nairn, 2009). The users may

choose wrong keywords when tagging, some words used as keywords to tag may have different

meaning and tags used in different languages are the problems associated with tagging data

items.

Kerr et al. (2006) proposed a social bookmarking system for large scale organizations

called Dogear social bookmarking service shown in Figure 2.2 that offers number of distinct

characteristics or features such communication between colleagues of organization, bookmarking

intranet resources, slide control to view tag cloud that contain histories of tag and tag index,

navigation of individual people bookmark collection through people links, navigation of

bookmark for which tags associated with other tags through tag clusters, bookmark search

through query search in search box or through drop down search option of people name, tag or

bookmark and community browsing mechanism. The web search tools such as DeeperWeb and

Search Cloudlet helps the users in exploratory search by suggesting keywords that helps the

users to refine their query with the new terms related to the query and also the tools which are

available as plug-ins can be integrated with search engines such as Google that related the results

of users search query from search engines with tag clouds so that user can refine their query with

terms in tag cloud. Kammerer et al. (2009) suggested user interface for tag browser called Mr

Taggy shown in Figure 2.3 that identifies relationship between tags and suggest new tags and

concepts by browsing through documents. It also helps the user to provide Relative tag feedback

and related search feedback that helps to identify related concepts and URL for keyword used to

search. Krause et al. (2008) analyzed the use of queries in search engines and tags in social

tagging systems by people for exploratory search and found that tags are useful than search

results produced by search engines. Also they argued that social tagging systems provide an

information seeking environment and help in effective exploratory search. However there is no

definitive evidence to show that social tagging systems are better than search engines and



11

important role of domain knowledge when searching through social tagging systems for

exploratory search.

Figure 2.2 - Dogear social bookmarking service (Kerr et al., 2006)

Figure 2.3 – Mr Taggy interface (Wilson et al., 2010)



12

2.3.2. Visualization Techniques

Visualization of web space helps to classify the documents and present the results from

web search in structured format to help determine the relevance of different pages and allows

users to select desired pages. Information Visualization is the process of transforming three

different types of data according to Stevens (1946): nominal data which helps to identify things,

Ordinal data which are ordered data and Quantitative data which helps to compute ratio or

interval between data values in to graphical attributes which helps to recognize patterns and

reduces the cognitive workload. Visualization in web applications offers navigation cues that

help users to explore interesting items and make correct decision based on the interest of others

and based on the attention of most other users (Indratmo, 2010). As explained by Mercun and

Zumer (2009) Visualization helps in exploratory information seeking and used in many types of

search systems such as Library systems, Eyeplorer, Liveplasma, Tuneglue, Musicovery and

Touchgraph but most of those lack information needed to represent visual data, textual data and

require proper architecture and interaction design that will help in proper understanding by users

of system.

Digital Library systems will contain vast amount of information items and therefore

retrieval of information item from digital library system will be time consuming unless users

have proper domain knowledge about the item they are looking for. Relational visualization

technique used for HighWire Digital Library of Stanford University provided with Topic maps

of nodes and edges with nodes represents topics structured according to search query and edges

represent relationships between different topics (nodes). Seifert and Kruppa (2010) developed a

system called Digital Library Assistant (DILIA) using Relational Visualization technique in

which results of user search query is visualized as topics as round blobs labeled with topic names

with number of documents present under each topic. The size of blobs represents the number of

documents and edge between topics blobs represents the documents shared between topics with

thickness of edge represents the number of documents shared between topics. Visualization

technique support exploratory search through graphical display of items and queries in document

space and depicts the relationships among data that helps users in easy navigation through

document space to acquire information they require. Dork et al. (2008) proposed a system called

VisGets for coordinated visualizations that help to visually filter the data from more than one



13

dimension such as time, location and topic that cannot be expressed by textual data and construct

dynamic queries to explore large information space. Aurnhammer et al. (2006) developed a

system called TagSphere shown in Figure 2.4 for content-based image retrieval and visualization

of entire search results corresponding to user’s selection of image from examples. TagSphere

combines tagging along with visualization in which tags associated with images are visualized

using different circles that helps users to retrieve images easily. Also users can perform search

using tag where the corresponding image results are displayed in suggestion display and images

selected by users are added to user collection area.

Figure 2.4 - TagSphere (Aurnhammer et al., 2006)



14

2.3.3. Surrogates

Document surrogates are representations about contents of information items which helps

to review the information items quickly which reduces time to process documents. Document

surrogates such as titles, abstracts, summaries, storyboards and sections helps searchers in

making decision about their search activity and thus aid in exploratory search by providing cues

about next step during search. According to Marchionini (1997), there are two types of

surrogates such as descriptive surrogates represent the attributes of the document which helps to

sort the documents and semantic surrogates represent the contents of documents in the form of

thumbnails. Surrogates helps in examination of search results thus helps in faster lookup and also

helps to provide detailed overview of information objects thus helps in easy information

extraction.

The surrogates of documents such as titles are fixed representations of document and

surrogates such as summaries are dynamic representations of documents. Summaries are used for

text retrieval, music videos and media file retrieval and also used for web retrieval. The

storyboard surrogate example of which is shown in Figure 2.5 is used for representation of video

segment in time order that maintains traditional flow. Ding et al. (1999) studied the use of video

surrogates and found that combined surrogates such visual images of key-frames and verbal

information of keywords and phrases provide better surrogate for videos that helps in better

sense-making and information integration of video objects. The interface in Open Video Digital

Library (www.openvideo.org) provides number of surrogates such as storyboard, fast forward

and excerpt for visual representation and textual representation for the video collections that

helps the users to effectively explore the videos for various purposes.



15

Figure 2.5 - Informedia Storyboard- TRECVID 2006 (Christel, 2008)

2.3.4. Faceted Interfaces

The user interface plays a vital role as an exploratory search tool. The user interface is a

Human Computer Interaction tool which includes both software and hardware that helps efficient

interaction between users and computers. User interface is effective tool because of graphical

interface which will make searching process much easier and adopt more browsing and display

options. Faceted search interface is used for number of exploratory search process due to broad

category of search options and results produced by faceted search. Faceted technique helps to

reduce and provide alternative for query reformulation that provide flexible information seeking

strategies which influence searcher tactics and also helps searcher with unclear domain

knowledge to refine their search for efficient information seeking. Faceted interface is useful for



16

exploratory search in the way it helps users without prior knowledge of the item to browse and

retrieve information about item by providing number of alternative choices to browse through

number of possibilities and select items based on their constraints and preferences.

Wilson et al. (2008) explained two main approaches of faceted browsing such as

directional column-faceted browsers and non-directional or traditional faceted browsers. In

directional column-faceted browsers, facets are presented as horizontal so that selection of items

by users in any facets filters the facets to right of the facet. In non-directional or traditional

browsers, selection of items in any facet filters the items in subsequent facets to show items

relevant to selected item. mSpace shown in Figure 2.6 is an interaction model explained by

Schraefel et al. (2006) is a directional column- faceted interface in which facets are represented

by columns from left to right and consists of features such as backward highlighting that helps to

know the structure of collections in facets, order, add or remove the facets in collection, dynamic

info view to know the results of the selection of columns and helps to compare the items in the

collection by change of selection. The log analysis of mSpace by Wilson and Schraefel (2008)

showed that easy to use for complicated queries along with basic keyword query search and

produced subjective measures. There are also web search interfaces systems that help to classify

unknown web collections in to facet categories such as Search Result Visualization and

Interactive Categorized Exploration (SERVICE) shown in figure 2.7, Dyna-Cat, Northern Light,

Exalead and PunchStock shown in Figure 2.8 that automatically generate facets for web search

results. Flamenco system proposed by Hearst et al. (2003) shown in figure 2.9 consists of

hierarchical faceted search interface and provided with menu choice for navigation searching of

collection of art, architecture and tobacco documents. Flamenco interface provide category

overview of collections by providing facets and helps the users to know about the number of

documents present under each category. Hearst et al. (2003) conducted usability test for

Flamenco system and evaluated the system based on subjective measures for exploratory tasks in

which the Flamenco system found more useful for users to retrieve images for structured task

and produced high subjective measures. The other faceted search systems that are similar to

Flamenco are Epicurious, a commercial recipe site developed by a company called Endeca and

Exhibit developed by Huynh et al. (2007). The drawback in these types of faceted interfaces is

that they do not help to visualize or identify relationships between different facet categories. To



17

rectify this problem, Villa et al. (2008) developed a faceted interface called FacetedBrowser

shown in Figure 2.10 that helps in retrieval of multimedia objects and in performing complex

search tasks in which multiple searches are split in to sub-searches all of which can be visually

seen at same time through different facets. FacetedBrowser supports multiple searches to be

performed in parallel through multiple paths by identifying facets during sub-searches thus

providing relationship between facets. Shneiderman and Kules (2008) studied the faceted search

interface using large-scale log analysis that helps to determine use of interface elements by

searchers and comparative user studies to determine the use of interface by searchers and proved

that faceted search interface helps the searchers in better information access and retrieval. They

also shown that searchers those use faceted interface have broad category of search results which

they can access easily than those who doesn’t use faceted interface to search.

The interface in Open Video Digital Library (www.openvideo.org) provides number of

alternative textual and visual browsing options to search the visual segments in the collection and

provides users with more detailed search results. Also it helps the users to download the video in

their desired format (Marchionini, 2006). The interface used in NCSU OPAC that supports

exploratory keyword search consists of three facets such as subject, region and time period to

search all catalog fields in NCSU system (Kules et al., 2009). According to research of Park et

al. (2008), a new interaction tool that provides interface between user and search engine called

SketchBrain helps in information seeking activity by conceptualizing the trails in workspace,

perform implicit operations and helps users to navigate next informative page. The disadvantages

of faceted interface are the faceted classification of information does not provide information

according to ranking or popularity and construction of faceted interface requires predefined

schema and stable structure.



18

Figure 2.6 - mSpace faceted column browser (Schraefel et al., 2006)

Figure 2.7 – SERVICE (Wilson et al., 2010)



19

Figure 2.8 – PunchStock (Wilson et al., 2010)

Figure 2.9- Flamenco – Hierarchical Facet navigation (Hearst, 2006)



20

Figure 2.10 - FacetBrowser (Villa et al., 2008)

2.3.5. Semantic Web

The semantic web provides detailed and additional information in the form of metadata

about resources and their relations present in the web. Semantics may be lightweight semantics

which involves sharing information between users through web and strong semantics which

involves sharing information through ontologies. A semantic web search engines consists of a

centralized data repository that contains metadata about documents and resources that are

interlinked and distributed throughout the web. Semantic web technologies have a better

potential to support exploratory search as semantic web search tools provide search results in

organized structures and help to specify new search queries for users to select for their

navigation of content. Semantic web technology provides better search and exploration of

collection of different museums as museums contains semantically rich collections of metadata.

Hyvonen et al. (2004) developed a system MuseumFinland Portal for publishing artefacts from

museum collections of three different museums of Finland on the semantic web. The main goal



21

of this semantic web portal is to provide global view to museum collections, provide content-

based search for browsing the interested content and to create national publication channel for

publishing semantically enriched contents of museums on the semantic web.

Mulholland et al. (2008) developed exploratory semantic search applications called

Bletchley Park Text (BPT) shown in Figure 2.11 for museum called Bletchley Park museum in

United Kingdom. BPT consists of newspaper style homepage that provides links to stories that

are generated by matching metadata concepts to topic selected by users and composite pages that

contains organized collection of stories. The metadata in semantic web was also utilized by

Hemel et al. (2003) who developed a prototype called OmniPaper to enhance searching in online

newspaper archives using Topic Maps. OmniPaper makes use of both navigation and querying as

it supports two types of navigation and querying: hierarchical and relational navigation, simple

and advanced querying in which enhanced relational navigation and querying is supported by

Topic Maps of semantically rich keywords or concepts related to user query produced using

knowledge map. Using hierarchical traditional navigation users can navigate through categories

of news subjects and relational navigation helps to navigate related news subjects produced using

knowledge map. Simple querying produces results of news subjects corresponding to user search

query whereas advances querying helps to include constraints in their querying which returns

filtered results of news subjects.

mSpace which is interaction model and directional column-faceted interface is based on

semantic web technology that used metadata of content to construct terms that helps to explore

the content. Mirizzi and Noia (2010) proposed a web search tool called Lookup Discover

Explorer (LED) shown in Figure 2.12 that helps in exploratory search and enhances semantic

web technology Linked Data. The LED takes account of the semantics of tags in the tag cloud

that are related to search query used by searchers and thus helps in exploratory search by

generating tag cloud with tags or keywords that are semantically related to keywords used by

searchers for their exploratory search, suggest keywords for search query used by searchers and

helps to refine search results from search engines by refining the query used by searchers by

semantically relating the keywords to keywords in a tag cloud. Semantic versions of Wikipedia

that help to exploit Wikipedia include systems such as DBPedia which helps to visualize



22

semantic data and OntoWiki which provides lightweight semantics and full text semantic search.

Berners-Lee et al. (2006) proposed a generic data browser similar to DBPedia called Tabulator

project that helps to explore Linked data and visualize Linked data by table, map, calendar and

timeline. The main challenge in the semantic web is the construction of semantic queries by

users as this requires knowledge of query languages and domain knowledge related to respective

URIs.

Figure 2.11 - Bletchley Park Text (Mulholland et al., 2008)



23

Figure 2.12 - LED interface (Mirizzi and Noia, 2010)

2.3.6. Relational Browsers

An interactive overview that depicts the structure and content of website is necessary for

large scale websites such as government websites, corporate websites and other portals which

can help users in easy navigation of website. Such interactive overview is provided by generic

interface called Relational Browser that helps to understand the structure of collections in the

web space and provides effective searching by examining relationship between attributes in the

web space. The first prototype of Relational Browser adopted mouse over control mechanism

and used to identify relationships between topic attributes which is used for Fedstats portal

(Marchionini, 2000) shown in Figure 2.13 and the interface developed from the prototype after

usability testing is used for Library of Congress National Digital Library. Marchionini (2006)

suggested Relational Browser for Open Video Digital Library that provides relationship using

mouse brushing between facets such as topic, time, space or data format and their attributes.

Relational Bowser interface has unique features such as displaying results by intersection of

documents based on user’s selection and graphical representation on each facet to show the

number of documents that can be found by users by clicking the facet.



24

Zhang and Marchionini (2005) developed the Relational Browser++ shown in Figure 2.14

overcoming the limitations in earlier relational browser in which they provide visual overview of

information space with help of multiple facets with facet values represented by graphic bars,

helps partition of information items and to identify relationship between them and provides

dynamic filtering option for search results by which users can filter the search results to obtain

desired results by searching with related keywords. Also Relational Browser++ provides

overview of updated results using result set panel that helps users to explore the information item

for their needs and helps to locate information items easily by matching search string patterns

with information items anywhere in the information space. Capra and Marchionini (2008)

proposed Relational Browser 07 that consist number of features to unique features compared to

previous versions of Relational Browser such as multiple view and facet cloud to display facets,

grid or list view to display search results, display of search query at the top of the screen along

with history of facets and search queries, search through keyword and search within results.

Figure 2.13 - Relational Browser (Marchionini, 2000)



25

Figure 2.14 - Relational Browser ++ Interface (Zhang and Marchionini, 2005)

2.3.7. Clustering Systems

Clustering involves grouping search results into groups with specific topic so that users

can choose topic of interest instead of refining their query for their next step. The search results

produced by conventional search engines are organized according to their ranking based on user

query. The users have to navigate the every irrelevant item to find the desired information among

search results returned by conventional search engines which is list according to some ranking

relevance to user query and contains list of mixed topics. Clustering systems organize the results

in to cluster groups and helps to summarize the cluster names to the users which help users in

easy navigation and retrieval of documents from the desired clusters. Thus Clustering systems

help to avoid navigation through multiple search results produced by conventional search

engines and it acts as a complementary to those by providing entire search result lists as a single

view of clustered results which helps users even without proper domain knowledge about search



26

topic to acquire better information by following specific cluster relevant to their search query.

Document clustering systems cluster documents in advance in to hierarchical tree structure and

return results that best matches the user search query from the cluster. But clustering engines

help to cluster search results from conventional search engines thus clustering engines should

require efficient algorithm and design to acquire and process search results and efficient

graphical user interface to display the relationship between clusters.

Hwang et al. (2011) proposed a clustering system to cluster documents based on explicit or

implicit feedback from users that helps to identify the similarity between documents that helps to

avoid problem faced by traditional clustering systems to extract contents of documents and

cluster the documents by finding similarity between them. The image search engine called

IGroup shown in Figure 2.15 proposed by Wang et al. (2007) overcomes the limitation of

different style image search results produced by other image search engines as IGroup groups the

image search results in to clusters with name of cluster groups related to query used to search by

users. The image search through IGroup displays general view results as clusters at the left side

of the interface in the navigation panel with the cluster name at the top of each cluster group and

each cluster displays preview of images present under each cluster which can be expanded to

cluster view results of images under each cluster. The other clustering systems are Lexxe which

uses linguistics approach to extract the meaning of search query and generate results, Biznetic

and Accumo which presents search results in a tree view of clusters and meta-search engines

such as Clusty, Qksearch and Polymeta that collects documents from various systems and

organizes in to clusters. The problem with clustering is to cluster documents in response to user

query and to identify suitable label for cluster of documents.



27

Figure 2.15 - IGroup (Wang et al., 2007)

As pointed out by Rasmussen (2008), there are many tools developed for exploratory

search and evaluated based on some metrics and user generated tasks but the search system

should be developed such that it should not support exploratory search only on the specific set of

task or user- generated task but it should also support every real-world search activity and the

system should be part of users daily information seeking activity. According to White et al.

(2006), ESSs should be evaluated based on metrics and methodologies. Metrics defines the

evaluation of ESSs during the searching process and after the search process which are the

results produced by ESSs. The searching process involves interaction, behavior of user’s and

decision made by users during search process. Methodologies define the evaluation of ESSs

based on experiments such as interviews, questionnaires and tasks. The users should have

acquired and learnt more information of their search topic. Spink (2002) explained the evaluation

of exploratory search systems using system and task centered approaches. A system approach

deals with evaluating the system based on standard function such as testing precision and recall

capabilities of system to improve document retrieval corresponding to search query. A task



28

centered approach focus on the interaction of users with the system on how well the users

perform while using the system. Task centered approach involves assigning tasks to be

performed by users and takes account of effectiveness, efficiency and satisfaction of the users

while performing task in the system. The tasks to evaluate ESSs should be based on the domain

knowledge of users involved in the task and the interviews and questionnaires should be

conducted with at-most care to collect the information needed from the users.

2.4. Exploratory Search Methods

2.4.1. Iterative Methods

Each iteration refines the search query and reduces the size of domain. Also keyword used

to search can be refined through feedback from users after each iteration which can produce

quality queries that can be used to search. But both of these iterative methods will be time

consuming as process need to carried for more iterations until desired result is achieved.

2.4.2. Query by Example

In Query by example, users construct query through navigation by clicking on suitable

example documents related to their information needs. Query by example is a content-based

search that helps to order results based on the examples related to the results. Query by example

are supported in system developed by Aurnhammer et al. (2006) called TagSphere that helps in

content-based image retrieval and visualization of search results corresponding to tags associated

with selected example image by user. Another image search engine IGroup developed by Wang

et al. (2007), also supports query by example search in which the search results of keyword

search performed by users are grouped into clusters related to the query and users can refine their

query by selecting images from the clusters. The drawback of using query by example for image

search engine is that some users may unable to find suitable example image required for their

search.

2.4.3. Keyword Refining through Feedback

User feedback can help to refine and improve the quality of search query. Moussa et al.

(2006) conducted a research to study the potential of user feedback for refining queries in image

retrieval system and found that system utilizing user feedback perform faster than normal system



29

and avoids the need to reinitiate search if users are not satisfied with their search results. Social

tagging systems help the users to refine their search based on the feedback from other users

suggested in the form of tags that provide information cues to users. Kammerer et al. (2006)

utilized user feedback to refine the search query and search results for their tag search browser

Mr Taggy in which user feedback helps to refine search results and search query by including or

excluding tags corresponding to search query.

2.4.4. View Based Search

In view-based search, users construct query through navigation of search results and

following path suitable to their information need among many available options. Also users can

refine their query by navigating through suitable path among many successive options in

information space. View-based search is supported by faceted browsers which help the users to

choose facets suitable for their information need with restrictions among multiple facets

available and it is used for applications such as online shopping and job searching.

2.5. Role of Domain knowledge for exploratory search

Domain knowledge plays an important role in information seeking and those with high

levels of expertise in a domain engage in efficient searching and acquire better information. The

person with good domain knowledge will tend to select the best links that will lead his or her

target, input short and optimal search queries and also spend less time searching (Duggan and

Payne’s, 2008). White et al. (2009) investigated the role of domain expertise on search behavior

and found that the domain experts use domain specific vocabularies and own terms in their

search queries and are more successful in their search than non-experts. In case of social tagging

systems, domain experts can perform better than domain novices in assigning relevant tags to

every documents that will be easy for other users to predict the documents based on tags and also

in extracting related contents from documents based on tags. As exploratory search involves

more learning and investigating process, the person with good domain knowledge can perform

better exploratory search and acquire information efficiently.



30

2.6. Analysis of Literature Review

Exploratory search arises when users searching for information through the

internet have unclear goal and when users are unaware about the exact location to

find the information and unsure about how to find the exact information.

Exploratory search combines both learn and investigate as explained by

Marchionini (2006) and involves general purpose and serendipity browsing as

classified by Cove and Walsh.

According to White et al. (2008), various exploratory search systems such as

information visualization systems, document clustering and browsing systems,

and intelligent content summarization systems are available that supports decision

making.

Social tagging or bookmarking systems supports exploratory search in the form of

tags or bookmarks also interpreted as folksonomies from other users that act as

navigation cues that help users learn and acquire knowledge to aid better

information acquisition and thus facilitate exploratory search.

Visualization systems helps in graphical display of items and results from web

search that help users in decision making and to explore the information space in

desired path to select interesting items.

Document surrogates supports exploratory search by providing cues in the form

of representations about contents of information items to review them quickly

which helps in decision making and reduces time to process documents.

Faceted interface provides view-based search with more browsing, search and

display options to make searching easier and also helps in query formulation to

help even the users with unclear domain knowledge to refine their search query

for better information acquisition.



31

The semantic web has potential to support exploratory search as it contains

metadata about documents and resources to organize search results and help to

specify new search queries for users to select for their navigation of content.

Relational browser is a generic interface that helps to understand the structure of

collections in the web space and provides effective searching by examining

relationship between attributes in the web space which helps users in easy

navigation of website.

Clustering systems provides search results as single view of clustered group

results that help users to select the desired cluster group topic relevant to their

search query and it also help users even without proper domain knowledge about

search topic in easy navigation and retrieve documents from the desired clusters.

According to White et al. (2006), ESSs should be evaluated based on metrics and

methodologies and Spink (2002) explained the evaluation of ESSs using system

and task centered approaches.

The different types of exploratory search methods are iterative methods, query by

example, keyword refining through feedback and view based search.

As exploratory search involves more learning and investigating process, domain

knowledge plays an important role in exploratory search tasks and users with

high level of expertise engage in efficient searching and acquire better

information than novices.



32

CHAPTER 3

METHODOLODY



33

3.1. Introduction

Research involves finding solution to scientific and social problems by analysis of

information collected through various methods and procedures called research methods.

Research methods involves theoretical procedures, experiments, observation, reasoning and

statistical analysis to collect information on particular topic to find solution to problem. The

research wheel shown in Figure 3.1 describes the different phases of research process.

Figure 3.1 - The Research Wheel (Rudestam, 2007)

3.2. Research Methods

According to Shuttleworth (2008), there are three basic research methods as follows

Experimental Research Methods

Opinion Based Research Methods

Observational Research Methods

3.2.1. Experimental Research Methods

Experimental research methods involve conducting laboratory based experiments to

evaluate or test the performance or usability of application or hardware. Experimental research

methods involves testing of research hypothesis using experimental design methods such as true

experiments, laboratory experiments, quasi-experiments to study the cause and effect of design

or situation corresponding to hypothesis (Trochim, 2006). It also aims at providing solution to



34

research problem and generation of theories based on manipulation of experimental research

methods. True experiment involves methods such as pre-post test design, random assignment of

participants to research study and evaluation using experimental group. Quasi-experiments

involves naturally occurring treatment groups and control groups that does not set up or assigned

for research which differs it from true experiment that involves assigning participants to

treatment groups or experimental groups.

3.2.2. Opinion Based Research Methods

Opinion based research methods involves collection of data through questionnaire or

survey, interview or focus groups. Survey involves researchers to gather information by sending

series of research questions to subjects to respond through which more information can be

collected within short period of time. Survey is of two basic types: cross-sectional survey and

longitudinal survey. Cross-sectional survey involves collecting information at a specified time

that may help to gather data about particular factor or compare and identify relationship between

two factors. Longitudinal survey involves collecting information over a period of time to make a

detailed analysis about a factor for that period of time. Survey can be conducted online through

email or through telephone. Telephone survey will be expensive to conduct than email survey

but email survey will be time consuming as it requires subjects to respond to acquire information

and also detailed information could not be obtained through mail survey.

Interview involves direct contact with the subject to acquire in-depth or detailed

information. Interview will involve recording answers from subjects for pre-defined

questionnaires or it will be discussion about list of topics. The person to be interviewed will be

the expert in the field of research or person affected in the field of research. According to

Bryman (2004) there are three types of interview such as un-structured or informal interview and

semi-structured interview which helps in analysis of qualitative data and structured interview

which helps in analysis of quantitative data. Informal interview are flexible and involves open

questions by the interviewer to acquire in-depth information from interviewee. Semi-structured

interview also helps to collect in-depth information from interviewee where interviewer has set

of topics or list of framed questions to be asked to interviewee known as research guide but the

interviewer do not follow the same order of questions as in research guide to every interviewee



35

and also interviewer can ask questions related to list of topics which are not listed in research

guide depending on response from interviewee. Structured interview involves interviewer to ask

same set of standard and predefined questions to ask to every interviewee. Focus groups help to

reveal the response of participants for particular situation. It involves more than one participant

to discuss number of topics or theme and interviewing specific participant for particular

situation. Focus groups are a form of group interview in which the participants are allowed to

interact with each other to share their ideas and experiences with researcher simply acting as

moderator or facilitator to collect data based on interaction between participants by monitoring

and recording the interaction.

3.2.3. Observational Research Methods

Observational research method involves observation or case study. Observation is the

simplest research methods and involves observation of group of people by researcher which

mainly helps to provide qualitative data and more detailed information. Observation may be

participant observation or direct observation (Punch, 1998). Participant observation involves

researcher to become participant to monitor the entire context and interact with the subject.

Direct observation involves researcher to be focused on subject to acquire specific information

from subject by monitoring and observing the interaction. Case Study involves detailed study or

analyzes the problem of subject or case through observation or interview. Case study involves

detailed study of real situation of subject or analyzing the situation of subject during different

times to make comparative study. Stake (1994) classified case study in to three types as intrinsic

case study which helps to understand the particular case or subject better, instrumental case study

which involves detailed study of particular subject or case to get insight in to issues associated

with it and to refine a theory based on study and collective case study which involves detailed

study of different cases or subjects to make general or comparative study.



36

3.3. Research approaches

There are two broader methods of reasoning (Trochim, 2006) as follows

Deductive approach

Inductive approach

Deductive approach informally called top-down approach works from more general

theories to more specific results. Deductive approach begins with thinking about theory related to

any topic or research interest, then hypotheses related to research theory is formulated and

specific hypotheses are tested, then observations are made to collect data related to hypotheses

and finally testing of hypotheses is done by analyzing the collected data to confirm the research

theory. Thus deductive approach is based on deductive thought that allows the researcher to

formulate hypothesis from theory and hypothesis are tested through data collected through

research methods such as surveys and interviews to confirm or reject the theory if the analysis of

data supports hypothesis.

The waterfall representation of deductive approach (Burney, 2008) is

Theory

Hypothesis

Observation

Confirmation

Inductive approach informally called bottom-up approach works from more specific

observations to more general theories (Trochim, 2006). Inductive approach begins with specific

observation, then patterns are discovered or developed, then hypotheses are framed and finally

general theories are framed based on analysis of hypotheses. Thus inductive approach is based



37

on inductive thought that allows the researcher formulate tentative hypothesis based on observed

patterns and define a theory by verifying hypothesis through data collected by observation or

experiments.

The hill-climbing representation of inductive approach (Burney, 2008) is

Theory

Hypothesis

Pattern

Observation

This study involves both deductive and inductive approaches.

The deductive approach of this study starts with general about exploratory search, identify

the various techniques suggested by some previous researchers that will support exploratory

search, investigate various existing search engines, exploratory search systems (ESSs) and

techniques to support ESSs and find the specific suitable technique that will support better

exploratory search. The inductive approach of this study intends to start with evaluating search

engines and ESSs using experiment and investigate problems associated with search engine,

ESSs for exploratory search and observe the behavior of participants to find the role of domain

knowledge for exploratory search. Then propose and develop interface design that will support

better exploratory search. Then evaluate interface design by experiment and investigate the

difference between existing system and new interface design to see if interface system support

better exploratory search. Finally frame a theory on use of proposed interface technique for

better exploratory search.



38

3.4. Research Strategies

According to Bryman, (2004), research strategies can be as follows

Quantitative research

Qualitative research

Quantitative research involves testing of theory by collection of data and uses deductive

approach to derive relationship between theory and research. It involves techniques such as

structured interviews and questionnaires such as closed or open questionnaires (Bryman and

Bell, 2003). According to Punch (2008) quantitative data are information about world collected

through large-scale survey research that are converted in to numbers to generate statistics

through measurements such as counting or scaling. Thus Quantitative research which is

primarily deductive process and objective in approach in which the data collected through facts

and figures are converted in to numerical data which are statistically or mathematically analyzed

to test pre-specified concepts and to draw conclusion on theory or hypothesis based on numerical

data.

Qualitative research involves generation of theory by observation of respondents and uses

inductive approach to derive relationship between theory and research. It involves techniques

such as participant observation, un-structured or semi-structures interviews and focus groups

(Bryman and Bell, 2003). According to Punch (2008) qualitative data are information about

world in the form of words collected through qualitative empirical materials such as observation,

interviews, documents and audio-visual materials. Thus Qualitative research which is primarily

inductive process and subjective in approach attempts to explore behavior and experience of

participants to get in-depth information and generate theory by analyzing data.

3.5. Methods Used in this Research Study

This study involves both quantitative and qualitative research strategies.

Quantitative research of this study uses survey questionnaires to collect data from

participants regarding their various exploratory search tasks, their experience, usage and

preference of various search engines, exploratory search systems (ESSs) and techniques to



39

support their exploratory search tasks. Quantitative research of this study also involves

participants in search task in search engines, ESSs and collects data from them using evaluation

questionnaire. It helps to analyze the data gathered from participants in the form of charts, graphs

or tables and find a suitable technique that will support better exploratory search task.

Qualitative research of this study uses interview to collect data from participants regarding

their experience with search engines and ESSs during exploratory search task, the problems

faced by them during exploratory search task, comments and suggestions about search engines

and ESSs. Also observe the behavior of participants to find the role of domain knowledge for

exploratory search. Based on review of existing literature, by analyzing data collected by survey

questionnaire, experiment evaluation questionnaire and based on opinions from participants

during interview to develop an alternative interface to support better exploratory search. Then

interview participants to collect qualitative data from them about alternative interface after

involving participants in the evaluation of alternative interface. It helps to enhance the proposed

interface based on feedback from participants and propose a theory based on analysis of

information.

3.6. Procedure

A number of participants were engaged in survey about their experience of exploratory

search task, problems faced by them for exploratory task, their experience, usage and preference

of various search engines, exploratory search systems (ESSs) and techniques to support their

exploratory search tasks. The participants also surveyed about their opinion about techniques

suggested by previous researchers that will support exploratory search to find from them which

technique will support better exploratory search task.

The participants also engaged in exploratory searching process in search engine Google

and social tagging system (ESS) Delicious. The domain knowledge of participants related to task

was analyzed by engaging participants in knowledge questionnaire session about domain of

taken task. Then all the participants were engaged in keyword searching process of particular

exploratory task. Some of them lack in domain knowledge of task that was taken to search.

Remaining had good domain knowledge of search task. After completing the search task, the

participants were engaged in evaluation questionnaire session about their feedback on Google



40

and Delicious. The participants were also interviewed about their experience with Google and

Delicious during exploratory search task, the problems faced by them during exploratory search

task, comments and suggestions for enhancements in Google and Delicious to support better

exploratory search. The difference in searching behavior of participants with and without domain

knowledge of topic was identified by analyzing the number of URLs they have visited during the

search task, time they have taken to complete the task, way of search behavior, whether they are

able to complete the task. The participants were engaged in evaluation of alternative interface

developed after analyzing the results gathered from previous study methods. The participants

were interviewed for their feedback about alternative interface to find how far the designed new

interface made their exploratory keyword search better compared to the former two search

systems.

3.7. Questionnaire Design

Questionnaire can be used for range of activities or research which helps to form

background of research or activity by collecting data from large group of people to address the

research problem and objective which can be conducted through face-to-face questionnaire or

through self-completion questionnaire such postal or online surveys. Questionnaire helps to

provide, gather and analyze data efficiently and accurately.

3.7.1. Self-completion Questionnaire

Self-completion questionnaire also called as self-administered questionnaire involves many

forms of sending questionnaire such as postal questionnaire or mail questionnaire to large

number of people at relatively low cost (Bryman, 2004). Self-completion questionnaire are cheap

to conduct which can be distributed to wide geographic area in less time and more respondents

can answer the questions themselves by completing the questionnaire in convenience of their

time.

The questionnaire used in this research is self-completion questionnaire through online

surveys as this research involves gathering data from participants regarding their opinions,

awareness and perceptions about exploratory search, exploratory search systems (ESSs) and

techniques that support exploratory search and ESSs. Each participant can complete the

questionnaire in their own time which is easy for them to understand and answer the questions.



41

3.7.2. Question Styles

Questions can be presented in following two formats relevant to self-completion

questionnaire and structured interview research design (Bryman, 2008)

Open Questions

Closed Questions

Open Question allows the participant to use their own words for questions which helps to

collect qualitative data. Participant will take more time to answer open questions but they are

free to express their own desire and interest so that wide range of data responses can be

collected. Thus knowledge level of participants and new areas can be explored through open

questions.

Closed questions allows the participant to answer the questions from the options provided

which helps to collect quantitative data. Closed questions will be easy for participant to answer

but they are restricted to options provided so that responses may not be true. Closed questions

are hard to design but it is easier to administer data collection through closed questions and the

data collected can be easily processed and analyzed. Trochim (2006) explained different

approaches of closed questions such as:

Dichotomous questions – Questions involving two way responses such as yes/no,

true/false and agree/disagree response.

Questions based on Level of Measurement – Nominal questions which have numbering

response option and Ordinal questions that are based on ranking the response based on

preference or importance of participant.

Questions based on interval level Measurement – Likert response scale that involves

questions based on 5-points scaling response, Semantic differential scale that involves

questions based on scaling with bipolar adjectives response pairs and Guttman scale

which involves participants to checklist each response they agree.

Filter or Contingency Questions – Questions that involves filtering subsequent questions

and avoid subsequent questions by participants based on response to previous question.



42

3.7.3. Presentation of Questionnaire

Better organization of questionnaire will help to gain information needed from

participants. Questions should be presented in professional and attractive format to make easy for

the participants to fill the answer. Questionnaires should be framed relevant to research objective

with simple language which will be easy for participants to read and understand. The font of

questions should be large enough to read by participants, instructions accompanying the

questions should be clear, questions and pages should be numbered and if possible should be

grouped in to sections and should be accompanied by pictures to support questions. The survey

questionnaire should have introduction about the research objective, should start with questions

that are important to research objective, start with questions that will attract interest of

respondents and should end up with greeting to participants. Byman (2008) pointed out rules for

designing questions such as researcher while framing the questionnaire should bear in mind the

research questions, what information they need to know or obtain and how the participant will

answer the question. Also the researcher while framing questionnaire should avoid ambiguous

terms, avoid long questions and be specific, avoid jargons, double-barreled questions and

colloquialisms, avoid very general questions, avoid leading questions, avoid questions that

include negatives, minimize bias, avoid technical terms and ensure options are mutually

exclusive.

The self-completion survey questionnaire used in this research study are revised several

times and presented in well structured format that will be easy for respondent to understand and

answer. Questions are ordered using numbers and clearly spelled in simple language to make

easy for participant to answer. Closed questions such as multiple choice questions are used in

which participants need to select a single answer from number of choices and some questions

require participant to answer straight forward among two choices provided such as yes or no

questions.



43

3.8. Pilot Study

Pilot Study is preliminary investigation that helps to test the research methods whether the

research methods fit for the research to be carried on, whether the research is possible and can be

conducted without any difficulties. Pilot Study helps to find the needs of research, any changes

or amendments to be done to research methods to ensure smoothness during research. In case of

questionnaire method, pilot study helps to test the design of questionnaire whether the

questionnaire and instructions can be clearly understand by respondents, how long it will take for

them to respond the questionnaire and whether any difficulties faced by them while answering

the questionnaire. Also it will help to assess the questionnaire design and it will help to

understand the difficulties in any question and whether the response to questions will be as

expected. Thus pilot study guides to frame efficient questionnaire that will help to collect quality

data. In case of evaluation using experimental design, pilot study helps to ensure the

effectiveness of design and to ensure that the design working as expected. Pilot Study should be

conducted similar to face-to-face interviews to observe the reaction of participants and comments

of the respondent for each question.

Pilot study of this research study was conducted after designing survey questionnaire,

experiment evaluation questionnaire and alternative interface design. Questionnaire and interface

design was tested to ensure smoothness in research by finding efficiency of questionnaire, design

and find whether all work as expected and planned. Two of my friends attended the pilot study

and pointed out problems faced by them in understanding and responding both the survey and

evaluation questionnaire and in using interface design. According to pilot study and based on

valuable suggestion and comments of my supervisor, modifications were done to questionnaires

and interface design.

3.9. Sampling

All research study involves selection of samples from population to collect data to

investigate research topic. Blaikie (2004) explained the use of sampling for research as the

sampling reduces the cost and time of research study as studying whole population will be

tedious and expensive and ideal sample should reflect the whole population to help analyze and

make judgment about whole population. According to Bryman (2004) there are two main types



44

of sampling such as probability sampling and non-probability sampling. Probability sampling

gives every unit or element in a population a known chance of being selected but non-probability

sampling does not provide chance of being selected. Also Bryman (2004) classified probability

sampling in to four types such as simple random sampling, systematic sampling, stratified

random sampling and multi-stage cluster sampling and classified non-probability sampling in to

three types such as convenience sampling, snowball sampling and quota sampling.

This research study has employed snowball sampling. As this research study involves

selection of participants relevant to research topic and based on research questions, snowball

sampling approach was used which involves selection of people relevant to research and making

contact with those people to collect data (Bryman, 2004).

3.10. Participants Recruitment

This research study aims to provide better interface for users of internet to support their

various search activity. So the participant of this study should include both common less

experienced users and also experienced users of internet so that data can be collected from them

regarding problem faced by them during their search activity and can be analyzed to provide

solution in the form of better interface common to all types of users. A total of sixteen

participants were recruited for this study with different age, gender including postgraduate

students with major in information science related subjects, students with major in other subjects,

professionals working related to information field and other professionals working in other

fields. The sixteen participants are invited to complete online survey questionnaire, attended

experiment search task and took part in interview.

3.11. Distribution of Survey Questionnaire

Prepared survey questionnaire was distributed in-person, online through email and social

networking site Facebook (www.facebook.com) to selected sixteen participants. Brief

description about research to respondents and they are provided with information sheet.

Information sheet contains details about research and regarding ethics policy of research which

was classified as ‘low risk’ according to university ethics policy. Consent sheet was provided to

every participant for their approval in the form of signature to participate in the research by



45

reading conditions and agreement of participation. Data collected does not contain any

confidential information about respondents and details were presented in information sheet about

how the responses will be processed and published. Responses were obtained from sixteen

participants which was sufficient to analyze data for quantitative study of this research.

3.12. Survey Questionnaire Questions

After careful review of existing literature related to research study, possible questions were

framed and analyzed. After revising the survey questionnaire according to research objective,

existing literature, based on pilot study conducted using my friends and based on comments of

my supervisor, the final questionnaire was prepared and distributed. The structure of survey

questionnaire is as follows:

Question 1: This question asks the participants about how often they use the internet. It helps to

know the usage of the internet by users.

Question 2: This question asks the participants about their experience of exploratory search. It

helps to know whether users experienced any exploratory search.

Question 3: This question asks the participants about the problems faced by them for their

exploratory search. It helps to know various problems faced by users for exploratory search.

Question 4: This question asks the participants whether they are aware exploratory search

systems (ESSs). It helps to know user’s awareness of various ESSs.

Question 5: This question asks the participants what they use for exploratory search. It helps to

know preference of users for exploratory search whether ESSs or search engines.

Question 6 and 7: These questions ask the participants about their ability to use search engine

and ESSs. It helps to know how far they can use search engine and ESSs.

Question 8: This question asks the participants which search engine they prefer for exploratory

search if they have option of choosing only search engines. It helps to know which search engine

users prefer most for exploratory search.



46

Question 9: This question asks the participants preference of approach for their exploratory

search. It helps to know what type of approach users prefer for exploratory search and will

support their exploratory search

Question 10: This question asks the participants how far existing search engine or exploratory

search systems (ESSs) support their exploratory search. It helps to know how far user’s

exploratory search is supported by search engines and ESSs.

Question 11: This question asks the participants whether they are aware various techniques that

will support exploratory search. It helps to know user’s awareness of various techniques that will

support ESSs and exploratory search.

Question 12: This question asks the participants who aware of various techniques that will

support exploratory search which techniques they feel will support better exploratory search. It

helps to know the technique that support better exploratory search.

3.13. The Experiment

An experiment was conducted to evaluate how far the existing search engines and

exploratory search systems (ESSs) support user’s exploratory search. Also the experiment aimed

to evaluate the role of domain expertise for exploratory search. The Google (www.google.com)

search engine shown in Figure 3.2 and Delicious (www.delicious.com) social tagging system

shown in Figure 3.3 which is ESSs was selected for the experiment. The Google and Delicious

are chosen based on analysis of answers by survey from respondents in which respondents

answered that most of them use Google if they have option of using only search engine for

exploratory search and respondents who are aware of ESSs and use them for their exploratory

search mostly prefer social tagging system such as Delicious for their exploratory search. Also

Google is mostly used search engine and almost every internet users are familiar with Google’s

interface, search options and additional features provided by Google and Delicious is mostly

used social tagging system that provides tag-based search along with keyword-based search.

Thus the experiment was conducted to evaluate search engine Google and Social tagging system



47

Delicious for exploratory search task and to evaluate the role of domain knowledge for

exploratory search to find how the people with different level of domain knowledge perform

exploratory search and how the domain knowledge affect their exploratory search. Same sixteen

participants responded for survey questionnaire was used for experiment search task which was

followed by short interview session.

Figure 3.2 – Google



48

Figure 3.3 – Delicious

3.13.1. The Exploratory Search Task

The topic taken for exploratory search task was ‘software engineering life-cycle’. This

topic was chosen based on knowledge of participants about topic as this research aimed to

evaluate role of domain expertise for exploratory search. Eleven participants reported to have

good domain knowledge about software engineering while remaining five participants lack

knowledge about software engineering or related fields. The knowledge of participants was

tested using knowledge questionnaire about search topic. The knowledge questionnaire consists

of five questions about software engineering, software engineering life-cycle and related

concepts. All the participants were asked to complete the questionnaire to test their knowledge.

Then all the participants were asked to perform search about ‘software engineering life-cycle’

using Google and Delicious.



49

3.13.2. Evaluation Questionnaire

After the completion of search task by participants, they were asked to complete usability

evaluation and satisfaction questionnaire that helps to analyze the usability of systems and

satisfaction of participants with the systems. Closed questions based on five point Likert scale

measure was used. Closed questions helps to evaluate participant’s satisfaction and usability of

system based on quantitative data.

3.13.3. Evaluation Questionnaire Questions

The evaluation questionnaire was framed to evaluate Google and Delicious by collecting

participants’ response for following aspects based on their exploratory search task.

Question 1: This question asks the participants whether it was easy to use the system. It helps to

evaluate the ease of use of system.

Question 2: This question asks the participants whether it was easy to learn the use of system. It

helps to evaluate the ease of learn of system.

Question 3: This question asks the participants how effectively they can complete task using

system. It helps to evaluate the effectiveness of system.

Question 4: This question asks the participants about the efficiency of system during search. It

helps to evaluate the efficiency of system.

Question 5: This question asks the participants whether they are satisfied with search results

provided by system. It helps to evaluate the satisfaction with search results provided by the

system.

Question 6: This question asks the participants about the organization of results. It helps to

evaluate the presentation and organization of results by the system.

Question 7: This question asks the participants whether they are satisfied with information

acquired. It helps to evaluate the quality of information provided by the system.

Question 8: This question asks the participants whether they are satisfied with overall system. It

helps to evaluate the overall performance of the system.



50

3.13.4. Interview

Interview helps to collect qualitative data from participants using open questions.

Interview session consists of questions to find from users the problems faced by participants

while using the systems during experiment, the experience of participants while using the

systems and qualitative comments of the participants regarding systems merits and demerits,

improvements needed in the current systems and enhancements needed in those systems to

support better exploratory search.

3.13.5. Interview Questions

Question 1: This question asks the participants about the problems and difficulties faced by them

when using the search systems for exploratory search task during experiment. It helps to know

the problems and difficulties faced by users while using search systems for their exploratory

search tasks.

Question 2: This question asks the participants about the merits and limitations of the search

systems. It helps to know the various advantages and limitations of search systems.

Question 3: This question asks the participants about the improvements needed for the current

search systems. It helps to identify the necessary improvements needed for current search

systems to support and satisfy search needs of different types of users.

Question 4: This question asks the participants about the enhancements and features needed for

the future development of search systems. It helps to know the enhancements and features

needed in the future development of current search systems to support better and easier

exploratory search of users.

3.14. Alternative Interface for Exploratory Search

After the analysis of results from survey questionnaire and experiment some results can be

drawn. Also by reviewing and analyzing the existing literatures regarding exploratory search and

ESSs some results can be drawn. All of these can act as a base for alternative interface design to

support better exploratory search.



51

In this research, JavaScript scripting language was used to design HTML based alternative

interface. This research aims to propose new interface by identifying drawbacks in existing

search engines and ESSs to support better exploratory search. This new interface does not

employ any real-time retrieval algorithm or database and ways to enhance those were not

covered in this research study.

3.14.1. Evaluation of Alternative Interface

Same sixteen participants who evaluated Google and Delicious in experiment were used to

evaluate new interface design. It was followed by short interview in which participants were

asked to comment on new interface design about usability of system and satisfaction of

participants with the system and how they feel new interface will support better exploratory

search compared to existing search engines and ESSs such as Google and Delicious evaluated

during previous experiment.

3.15. Data analysis

This research study collects data were collected through three methods such as survey

questionnaire, experiment evaluation questionnaire and interview. The data collected through

survey questionnaire and evaluation questionnaire were analyzed using MS Excel which helps to

perform various statistical calculations and provide diagrammatic representation of results

through various graphs and charts. The data collected through interview and observation were

reviewed manually and discussed in findings.

3.16. Ethical Aspects

This research study was carried out under University of Sheffield ethics policy which was

ethically reviewed under the University of Sheffield’s ethics policy governing research involving

human participants, personal data and human tissue: version 2 (The University of Sheffield,

2011). This research study is classified as ‘low risk’ by research ethics committee. Information

sheet and consent form was provided to the participants. The information sheet contains purpose

of the research along with other details such as method of data collection, processing data and

publication of results. The participants were also informed that participating in research is based

on their own interest and they can withdraw from participation any time they wish. Personal



52

details of the participants were not collected and participants were assured that any data collected

from them will be kept confidential, will not be disclosed to any other person under any

circumstances without their consent and it will be used only for data analysis of this research

study. After informing about the details of research study and ethics policy, the participants were

asked to sign the consent form by accepting terms and conditions.

3.17. Practicalities

The main problem associated with this research study is the selection of participants

based on domain knowledge. Also every participant had experience of exploratory search, most

of the participants were not aware of the term ‘exploratory search’. So it took more time for them

to understand the purpose of this research study by reading the provided information sheet. The

practical problems faced during quantitative survey research were framing questionnaires,

sending questionnaires to participants and receiving response from them. Also involving

participants to evaluate search systems and collecting answers from them for evaluation

questionnaire and interview questions following experimental evaluation was found difficult and

time consuming because there was need to meet participants in their free time and explain them

about the usage of social tagging system (ESS) Delicious as most of the users were not aware of

ESSs and its usage. Designing alternative interface by analyzing the results and existing

literature, explaining the participants about alternative interface, involving participants to

evaluate alternative interface and collecting feedback from them were time consuming. The other

practical problems associated with dissertation format of dissertation and duration for completing

dissertation.



53

CHAPTER 4

RESULTS AND

FINDINGS



54

4.1. Survey Questionnaires

The survey questionnaire focused on following aspects of research:

Participants’ usage of internet and experience of exploratory search

Participants’ experience with search engines, exploratory search systems

and usage of them for exploratory search

Participants’ preference of various approaches and techniques to support

exploratory search

Usage of Internet

1. How often do you use the internet?

Obligations Response Count Response Percentage

Daily 16 100%

Several times a week 0 0%

Weekly 0 0%

Several times a month 0 0%

Monthly 0 0%

Never 0 0%

Table 4.1



55

0

20

40

60

80

100

120

Daily Several times a

week

Weekly Several times a

month

Monthly Never

Usage of Internet

Daily

Several times a week

Weekly

Several times a month

Monthly

Never

Figure 4.1

This question helps to know the usage of the internet by people. From the above graph

shown in Figure 4.1 we could find that every participant uses the internet daily. This states that

almost all people use the internet daily for their various day-to-day activities and information

needs.

Experience of Exploratory Search

2. Do you have any experience of exploratory search?


Yes 16 100%

No 0 0%

Preferred not to say 0 0%

Table 4.2



56

Figure 4.2

This question helps to know the user’s experience of exploratory search. From the above

graph shown in Figure 4.2 we could find that every participant has an experience of exploratory

search. This states that every user have some experience of undefined search goal or unaware of

their search goal which leads to exploratory search. As discussed in literature review most of the

search activity users will have undefined goal, users are unaware and unsure about how and

where to find the information which leads to exploratory search.



57

Problems faced by users for their Exploratory Search

3. What is the main problem faced by you for exploratory search?


Unclear search goal 7 43.75%

Unaware of how and where to

find the information

5 31.25%

Search system does not

support better exploratory

search

4 25%

Table 4.3

Problems faced by users for their Exploratory Search

Unclear search goal

Unaware of how and where

to find the information

Search system does not

support better exploratory

search

Figure 4.3



58

This question helps to know the various problems faced by users for their exploratory

search. From the above graph shown in Figure 4.3 we could find that most of the participants

which count to 43.75% of the participants were found to have unclear search goal. Next to that

31.25% of participants were unaware of how and where to find the information needed to them.

It means users are unaware of which system to use for their search, which path to follow and

how to proceed with their search to attain the information. 25% of participants found the system

they used for their exploratory search does not support better search. This is due to inefficiency

of system to support their search or users find to difficult to use the system and unsatisfied with

the acquired information.

As described in literature review in which Marchionini (2006) explained exploratory

search as which involves both learn and investigate activity and users requires to perform

multiple search session of querying and browsing to acquire required information which cannot

be satisfied by focalized search as explained by JcsCholtes (2010) and search engines along with

advanced search option does not support better exploratory search.

User’s awareness of Exploratory Search System (ESSs)

4. Are you aware of various exploratory search systems?


Yes 7 43.75%

No 9 56.25%


Table 4.4



59

Figure 4.4

This question helps to know the users awareness of various existing exploratory search

systems. From the above graph shown in Figure 4.4 we could find that maximum numbers of

participants are unaware of various exploratory search systems (ESSs) that will support

exploratory search. 43.75% of participants responded that they are aware of some ESSs where

maximum number of participants which counts to 56.25% of participants responded that they are

unaware of any ESSs.

As discussed in literature review, there are many ESSs are available such as information

visualization systems, document clustering systems and intelligent content summarization

systems as described by White et al. (2008) that will support exploratory search but most of the

users are unaware of these ESSs. Some users are aware of some ESSs such as Social tagging

systems and faceted interface systems.



60

Preference of systems by users for Exploratory Search

5. What do you use for exploratory search?


Search Engines 9 56.25%

Exploratory Search Systems 3 18.75%

Both 4 25%

Table 4.5

Figure 4.5

This question helps to know the preference of users between traditional search engines and

exploratory search systems for their exploratory search. From the above graph shown in Figure

6.5 we could find that most of the participants’ counts to 56.25% prefer only Search Engines for

their exploratory search which can be inferred from previous question as participants who are

unaware of ESSs prefer only Search Engines for their exploratory search. But some participants



61

who are aware of ESSs prefer both search engines and ESSs for their exploratory search as 25%

of participants prefer both while 18.75% of respondents prefer only ESSs for their exploratory

search.

As discussed in literature review, exploratory search requires multiple search sessions,

requires users to refine and use multiple queries during iterative search and requires them to

obtain cues about their next step to navigate search results. Thus some users who are aware of

social tagging system such as Delicious use them for their exploratory search in which tags from

other users acts as navigation cues that facilitate exploratory search of users as explained by

Millen et al. (2007). Some users felt that faceted interface system with category search option as

used by them in most of the systems such as library system support their exploratory search

which proved the study of Shneiderman and Kules (2008) to determine better information access

and retrieval provided by faceted search interface. Users who are unaware of ESSs preferred

only traditional Search engines for their exploratory search but most of them face the problems

as discussed during second question as keyword search and query formulation provided by

search engines are complex and difficult as discussed in literature review.

Ability to use Search Engines

6. If you use search engine for exploratory search, how would you describe your ability to

use them?


Beginner 0 0%

Intermediate 0 0%

Experienced 3 23.08%

Expert 10 76.92%

Table 4.6



62

0

10

20

30

40

50

60

70

80

90

Beginner Intermediate Experienced Expert

Ability to use Search Engines

Beginner

Intermediate

Experienced

Expert

Figure 4.6

This question helps to know the ability of participants to use search engine. From the

above graph shown in Figure 4.6 we could find that most of the participants as a percentage of

76.92% of participants are expert in using search engines. Some participants counts to 23.08% of

participants preferred they are only experienced as they are not aware using some options

provided by search engines. As discussed in literature review, even the expert users of search

engines told that search engines does not support their search needs.

Ability to use Exploratory Search Systems (ESSs)

7. If you use exploratory search systems for exploratory search, how would you describe

your ability to use them?


Beginner 0 0%

Intermediate 4 57.14%

Experienced 3 42.86%

Expert 0 0%

Table 4.7



63

0

10

20

30

40

50

60


Ability to use Exploratory Search Systems (ESSs)

Beginner

Intermediate

Experienced

Expert

Figure 4.7

This question helps to know the ability of participants to use exploratory search systems

(ESSs). From the above graph shown in Figure 6.7 we could find that 42.86% of participants

preferred that they are experienced in using ESSs. Remaining 57.14% of participants preferred

that they are only intermediate in using ESS they use. As discussed in literature review, even the

experienced users of ESSs told that ESS does not support better exploratory search and most of

the ESSs are tough to handle for their search needs.

Preference of Search Engines by users for Exploratory Search

8. If you use Search engine, which one do you use most for exploratory search?


Google 10 62.5%

Bing 1 6.25%

Yahoo 3 18.75%

Live Search 1 6.25%

Others 1 6.25%

Table 4.8



64

Figure 4.8

This question was asked to find the users preference of search engine if they have option

of using only Search Engines for their exploratory search. From the above graph shown in Figure

4.8 we could find that most participants as count of 62.5% of participants prefer Google for their

exploratory search. They felt that search option along with advanced search option and search

results provided by Google was better than any other search engines. Next to Google, 18.75% of

participants preferred Yahoo which is also most used along with Google. The users those

preferred Google and Yahoo felt that they were most used because of the additional features

provided by them such Mail, News, Videos and images provided by them. Each of Bing and

Live Search which was formerly known as MSN search was used by one respondent which is

6.25% of total participants. One participant preferred other search engine option as the

participant was found using search engines such About.com, Ask and Answers.com.



65

Approach to support user’s exploratory search

9. Among the following which type of approach do you prefer most for exploratory search?


Content-based search 3 18.75%

View-based search 5 31.25%

Visualization of search results 2 12.5%

Query suggestions 6 37.5%

Table 4.9

Approach to support user's exploratory search

Content-based search

View-based search

Visualization of search results

Query suggestions

Figure 4.9

This question helps to know the users preference of approach provided by various systems

that support and made their exploratory search better and easy. From the above graph shown in

Figure 4.9 we could find that 18.75% of participants’ preferred content-based approach, 31.25%



66

of participants preferred view-based search, 12.5% of participants preferred visualization of

search results and most of the participants’ counts to 37.5% of participants preferred query

suggestions provided by some systems make their exploratory search easy.

The participants who preferred content-based search approach for exploratory search felt

that options to construct and refine their query based on content-based approach will make their

exploratory search easier. As discussed in literature review, content-based approach is supported

by systems that provide query by example search method such as visualization system developed

by Aurnhammer et al. (2006) called TagSphere that helps in content-based image retrieval by

selecting example images from suggestion display and clustering system developed by Wang et

al. (2007) called IGroup that helps users to refine their search results by choosing suitable image

from cluster corresponding to their search query. The participants who preferred view-based

search felt that view-based search will help in easier exploratory search because of the graphical

interaction provided by them. Also some participants felt that view-based search provided by

faceted-interface with multiple category search option that make more easier even for users

without proper domain knowledge. The participants who preferred visualization of search results

approach explained that visualization of search results will help them easy navigation of search

results in information space to explore through their desired path which is used in most of digital

library systems such as the one developed by Seifert and Kruppa (2010). The participants who

preferred query suggestion approach for exploratory search were found to be users of social

tagging system as they provide query suggestions in the form of tags suggested by other users the

provide information cues to users. This proved the study of Krause et al. (2008) as discussed in

literature review that social tagging systems provide better information seeking environment and

effective exploratory search.



67

Support by Search Engines or Exploratory Search Systems for Exploratory Search

10. How far existing Search engines or exploratory search systems support exploratory

search?


1 – Bad 3 18.75%

2 – Fair 5 31.25%

3 – Good 6 37.5%

4 – Very Good 2 12.5%

5 – Excellent 0 0%

Table 4.10

Figure 4.10

0

5

10

15

20

25

30

35

40

1-Bad 2-Fair 3-Good 4-Very Good 5-Excellent

Support by Search Engines or Exploratory Search System

for Exploratory Search

1-Bad

2-Fair

3-Good

4-Very Good

5-Excellent



68

This question helps to know how far the existing Search engines or Exploratory Search

Systems support user’s exploratory search. From the above graph shown in Figure 4.10 we could

find that 18.75% of participants preferred option 1 which corresponds to bad support, 31.25% of

participants preferred option 2 which correspond to fair support, 37.5% of participants preferred

option 3 which correspond to good support, 12.5% of participants preferred option 4 which

correspond to very good support and 0% of participants preferred option 5 which correspond to

excellent support provided by Search Engines or ESSs for exploratory search.

The participants who preferred options 1 and 2 were found to be mostly the users of

Search engines which provide less support compared to ESSs as they provide information to

satisfy only less number of searchers. The participants who preferred options 3 and 4 were found

to be mostly the users of ESSs. The two participants who preferred option 4 of very good support

were found to user’s social tagging system for their exploratory search.

User’s awareness of various techniques to support exploratory search

11. Are you aware of various techniques that will support exploratory search?


Yes 8 50%

No 8 50%


Table 4.11



69

Figure 4.11

This question helps to know the users awareness of various techniques that will support

exploratory search. From the above graph shown in Figure 4.11 we could find that half of

participants are aware while remaining half of participants are unaware of techniques that will

support exploratory search.

As discussed in literature review there are many techniques provided by various

exploratory search systems and search engines that will support exploratory search but only half

of the participants are aware of those techniques.



70

Technique to support better exploratory search

12. If answered yes to above question, which technique you think will support better

exploratory search?


Faceted Interface 2 25%

Visualization 1 12.5%

Clustering 1 12.5%

Semantic web 1 12.5%

Tags 3 37.5%

Table 4.12

Figure 4.12



71

This question helps to know the preference of users to support exploratory search among

respondents who are aware of various techniques to support exploratory search. From the graph

shown in Figure 4.12 we could find that 25% of participants preferred interface technique that

will support exploratory search. Each of visualization technique, clustering technique and

semantic web technology is preferred by one participant. Most of the respondents which count to

37.5% of participants among the participants who are aware of techniques to support exploratory

search preferred tags to support exploratory search. So it is clear that most users are using

systems that contains tags and faceted interface to support exploratory search which the

participants felt that systems with tags and faceted interface are easy to use and make their

exploratory search better and easier. As discussed in literature reviews both tags in the form of

information cues from other users and faceted interface with multiple category search options to

select information based on preferences and constraints helps users in easy query reformulation

for effective information seeking in information space that both support users even without

proper domain knowledge to perform better and easier exploratory search.

4.2. Evaluation Questionnaires

The experimental evaluation of Google and Delicious for exploratory search task focused

on following aspect of research

Participants’ satisfaction with existing search engines, exploratory search

systems, their usage and support for exploratory search

Ease of use of system

1 It was easy to use

the system

1-Strongly

Disagree

2-Disagree 3-Neither

Disagree or

Agree

4-Agree 5-Strongly

Agree

The average ease of use of Google rating was 4.75 with 12 participants rated 5 and

remaining 4 participants rated 4. The average ease of use of Delicious rating was 4.38 with 9

participants rated 5, 4 participants rated 4 and remaining 3 participants rated 3. The participants



72

found Google was easy to use compared to Delicious as Google was mostly used search engine

but some participants felt that advanced search options and some features provided by Google

are difficult to use. Most of the participants found difficult to use Delicious as they claim that

they are first time users of Delicious.

Ease of learn of system

2 It was easy to learn

the use of system

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

The average ease of learning of Google rating was 4.63 with 10 participants rated 5 and

remaining 6 participants rated 4. The average ease of learning of Delicious rating was 4.25 with

8 participants rated 5, 4 participants rated 4 and remaining 4 participants rated 3. The participants

felt that Google was easy to learn compared to Delicious similar to ease of use of Google. Some

participants felt that some features provided by Google needs more practice to learn and most

participants felt that who were first time users of Delicious felt it was difficult to learn most of

the functions.

Effectiveness of system

3 I can effectively

complete task

using system

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

The average effectiveness of Google score was 3.88 with 6 participants rated 5, 4

participants rated 4, 4 participants rated 3 and remaining 2 participants rated 2. The average

effectiveness of Delicious score was 4.13 with 7 participants rated 5, 5 participants rated 4, 3

participants rated 3 and remaining 1 participant rated 2. Participants felt that completion of task

was better and can be done effectively using Delicious compared to Google because they felt that

tags acting as navigation cues are more useful than search results which are links provided by



73

Google. As discussed in literature review tags associated with every object helps to access and

retrieve them effectively.

Efficiency of system

4 Efficiency of

system is good

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

The average efficiency of Google score was 4.06 with 6 participants rated 5, 6 participants

rated 4, 3 participants rated 3 and remaining 1 participants rated 2. The average efficiency of

Delicious score was 3.94 with 6 participants rated 5, 5 participants rated 4, 3 participants rated 3

and remaining 2 participants rated 2. Participants felt that efficiency of system was same with

both the systems with some participants felt Google was slightly efficient than Delicious in some

cases.

Satisfaction with search results

5 Satisfied with

search results

provided by

system

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

The average satisfaction with search results provided by Google rating was 3.81 with 5

participants rated 5, 6 participants rated 4, 2 participants rated 3 and remaining 3 participants

rated 2. The average satisfaction with search results provided by Delicious rating was 4.0 with 6

participants rated 5, 5 participants rated 4, 4 participants rated 3 and remaining 1 participant

rated 2. Participants felt that search results provided by Delicious are better than Google because

Delicious provides tags from other users that help to refine their search query while Google

provides only links to sites. As discussed in literature review, tags from other users acts as



74

navigation cues that helps the users to browse through bookmarks according to time and popular

bookmarks.

Presentation and organization of results

6 Organization of

results is good and

useful

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

The average presentation and organization of results by Google rating was 3.69 with 4


rated 2. The average presentation and organization of results by Delicious rating was 3.62 with 4


rated 2. Participants found that presentation and organization of results are better in Google

compared to Delicious because Google provides results ranked in some order and some

participants felt difficult to access the results provided by Delicious.

Quality of information

7 Satisfied with

information

acquired during

search

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

The average quality of information provided by Google score was 3.19 with 3 participants

rated 5, 3 participants rated 4, 5 participants rated 3, 4 participants rated 2 and remaining 1

participants rated 1. The average quality of information provided by Delicious score was 3.5 with

4 participants rated 5, 4 participants rated 4, 5 participants rated 3, 2 participants rated 2 and

remaining 1 participants rated 2. Most participants are satisfied with information acquired using

Delicious than Google as they are able to find exact information using Delicious but they felt that



75

they need to analyze the results and information provided by Google to acquire information

needed to them. As discussed in literature review, tags interpreted as folksonomies acts as

navigation cues that help to learn and acquire better knowledge that facilitate better information

acquisition to find exact and correct information.

Overall performance of system

8 I am overall

satisfied with

system

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

The average overall performance of Google score was 3.93 with 5 participants rated 5, 6

participants rated 4, 4 participants rated 3 and remaining 1 participants rated 2. The average

overall performance of Delicious score was 3.81 with 4 participants rated 5, 7 participants rated

4, 3 participants rated 3 and remaining 2 participants rated 2. In case of overall performance,

participants felt that Google was better compared to Delicious because they felt that Google was

used often that satisfy most of their information needs and more features are provide by Google.

4.3. Interview

Interview helped to acquire qualitative information from participants regarding the

problems faced by participants during exploratory search task, experience with Google and

Delicious during exploratory search, advantages and disadvantages of Google and Delicious,

improvements needed in those systems and enhancements and features needed in the future

development of those systems to support their exploratory search better. The interview was

structured using following four questions to collect qualitative information from participants.

1. What are the problems and difficulties faced by you when you are using the search systems for

exploratory search task?

2. What are the merits and limitations of the search systems?

3. What are the improvements needed for the current search systems?

4. What are the enhancements and features needed for the future development of search systems?



76

Participants are satisfied with simplicity and ease of Google but 12 out of 16 participants

told that the met with difficult of forming search queries and to analyze the results to navigate

their next step. Also they felt that search results provided by Google are just ranked in some

order and they felt it would be better if the search results are classified. 6 out of 16 participants

told that they felt difficult in analyzing and navigating the search results. Among them five

participants told that they felt very difficult in navigating the search results for their next step due

to lack of domain knowledge of search task. 10 out of 16 participants were not able to complete

the task as they felt Google does not provide effective search results and information.

Participants also felt that advanced search option provided by Google was quite easy to use but it

does not provide better search results and support better information acquisition. A participant

told that “Advanced search option provided by Google does not make any difference in

information acquisition”. Participants told that many irrelevant and unwanted search results are

provided by Google so there is need to navigate number of result pages to find information

suitable to them. Two participants told that need of navigation for number of pages for

acquisition of desired information makes lose their interest to search. 75% of participants

suggested improvements for Google that it would support better search if they are provided with

better search options, provide more relevant and accurate results, classification of search results

and query suggestions by semantic interpretation of user search query.

In case of Delicious, 9 out of 16 participants told that met with many problems while using

the system as they beginners in using the system. 13 out of 16 participants told that use of system

and learning of system is difficult. Participants told that tags helps to refine query and support

better browsing but they told tags should be accompanied with better description. A participant

told “query suggestions in the form of tags helped me to complete my search soon”. Participants

also told that a tag with different meaning and use of tags in different languages provides

irrelevant results. Five participants told they are very confused with use of tags as same keyword

with different meaning provides different results. 81.25% of participants suggested some

improvements for Delicious such as it should be provided with more search options, standard

and good description of tags and classification of search results.



77

4.4. Role of Domain Knowledge for Exploratory Search

The role of domain knowledge for exploratory search was analyzed using the number of

URL visited during their exploratory search task, way of search behavior and based on

completion of task. Eleven participants who are domain experts with good domain knowledge

about search topic were found to be better in terms of search efficiency to find the relevant

information than five participants who are domain novices with least knowledge about search

topic. The role of domain knowledge for exploratory search task was also affected by the system

used to search. In case of Delicious systems, experts were found to use keyword based search

queries visited more URLS but novices were found to use tag based search queries thus they

need to visit only less URLs. 5 out of 11 domain experts found to use only keyword search rather

than tag based search. Thus experts were found to use their own search terms rather than search

tags of others as used by novices. In case of Google where the participants had the option of

performing only keyword based search, novices were found to visit more URLs to collect the

information than experts as novices found difficulty to frame correct search query for keyword

search. The average number of URLs visited by domain novices was found to be 21 whereas the

average number of URLs visited by domain experts was found to be 7. Thus experts were found

to depend more on their domain knowledge but novices found to depend on the knowledge of

others. Experts were found to better in terms of efficiently completing the task and in better

acquisition. 81.82% of domain experts are found to be effective in terms of search task

completion as opposed to 40% of domain novices. Experts were found to acquire more relevant

and accurate information but novices were found to acquire only general information. Thus the

domain knowledge plays a significant role for exploratory search but it also depends on the

system used for search task.

4.5. Summary of the Findings

The data collected through quantitative methods such as survey interview, experiment

evaluation questionnaire and qualitative method interview were analyzed and results were

summarized as follows that acts as base for alternative interface design.

People use internet daily for their various information needs and everyone

have some experience of exploratory search because at least some of the



78

search activity of people will have unclear search goal and they are unaware

of where to find the information and unsure about how to find the

information.

For exploratory search activity, most of the users face the problem of unclear

search goal while remaining users face the problem of unaware of how and

where to find the information or found their search systems does not support

better exploratory search.

Most of the users were not aware of exploratory search systems (ESSs) and

they use only search engines for their exploratory search found to be expert

and experienced users of search engines. Some users are aware of some

ESSs were found to intermediate and experienced users of ESSs. Among the

search engines, Google was used by most of the users for their exploratory

search. But both the users of search engines and ESSs found their search

systems does not support better exploratory search to satisfy their

information needs.

Most of the users prefer approaches such as query suggestions as provided

by social tagging systems and view-based search as provided by faceted

interface systems for their exploratory search.

Most of the users are not aware of techniques to support exploratory search

and among the users who were aware of techniques to support exploratory

search, most of them preferred tags and faceted interface.

The search engine Google and social tagging system (ESS) Delicious got

highest rating from users for their easy to use but Google was found to be

more easier to use than Delicious by users. Also both systems were found

easy to learn by users but Google was found to be much easier to learn than

Delicious.



79

Delicious helps users to effectively complete their exploratory search task

than Google. Similarly Delicious was found to be better than Google in

terms of providing results for user search query and in providing quality of

information to satisfy users search needs.

Users found the efficiency of system and presentation and organization of

results are at equal level with both Google and Delicious but Google was

found slightly better than Delicious by some factors. In case of overall

performance users are more satisfied with Google than Delicious.

Users felt difficulty in forming query words for their Google search and they

found query suggestions provided by Delicious in terms of tags helps to

refine their search query better and helps in easy navigation of search results.

Google was found to be ineffective by users in terms of exploratory search

task completion and problems faced by users with Google are ranked order

of search results provided for users search query, unwanted results and

results irrelevant to users search query.

Most of the users are found to be beginner or intermediate users of Delicious

so they found difficult to use and learn Delicious and they face the problem

with description, meaning of tags and use of tags in different languages.

Users felt that more search options and classification of search results by

both systems will support their exploratory search better.

The domain knowledge of users play an important role for their exploratory

search but it also depends on search systems they use for their exploratory

search that will reduce the role of domain knowledge for exploratory search

task.



80

4.6. Alternative Interface for Exploratory search

Based on analysis of statistical results and experiment results and evaluation of existing

exploratory search systems plus the review of existing literature of exploratory search,

exploratory search systems (ESSs) and techniques to support exploratory search and ESSs, an

alternative interface was designed to support better exploratory search. The new interface was

designed using JavaScript scripting language and it does not employ any real time database or

real time algorithm to retrieve information. The interface was designed based on Google and

social tagging systems because the analysis of statistical results from survey and experimental

evaluation results shows that search engines was used by most of the participants for their

exploratory search in particular Google search engine was used by most of the participants which

they found to be easy to use but participants suggested some approaches provided by social

tagging systems will support their exploratory search better. The general layout, search options

and features provided by new interface was discussed below.

4.6.1. General Layout

The main page of the interface shown in Figure 4.13 is similar to that Google search

results page with options for filtering results by preferences such as images, videos, news,

shopping, blogs, books, places, discussions and patents at the left side of the interface, link to

advanced search option near search box and search results below the search box.



81

Figure 4.13 – Alternative interface: General Layout

4.6.2. Search Options

Three types of search options shown in Figure 4.14 are provided in the interface design

such as normal search option, search option for searching all results or current results provided

near search box and advanced search options.

4.6.3. Keyword Suggestions

The search results page is provided with keywords of user search query along with some

related keyword suggestions that helps users to refine their query to more specific using keyword

suggestions to get more specific desired results.

4.6.4. Search within Results

Search option provided at the bottom of the interface is provided with two options such as

to search all results or to search within current results that help the user to refine their query

using keyword suggestions and submit their own different query to search all results or to search

current results.



82

Figure 4.14 – Alternative interface: Search Options

4.6.5. Advanced Search Options

Advanced search option shown in Figure 4.15 consists of two parts such as options to

place constraints on search and facet category search. The first part of search helps the users to

place constraints on the search query similar to Google advanced search option using different

keyword or more than one keyword, exact phrase or sentence, file type and language. The second

part consists of facet category search that helps the users to refine their search using different

facets such as person, locations, period and themes.



83

Figure 4.15 – Alternative interface: Advanced Search Options

4.7. Evaluation and User Feedback on Alternative Interface

The same sixteen participants who participated in survey and experiment evaluation of

search systems were asked to evaluate the alternative interface for exploratory search. After the

evaluation they were interviewed to collect feedback from them on the new interface design.

Every participant found the general layout which was designed similar to Google’s search result

page easy to use and learn as of Google. Participants felt that different types of search options

provided in the interface made their searching simplified and easier. 75% of participants



84

commented that keyword suggestions along with search keywords helped them to choose the

desired keywords to refine their search queries that made their browsing easier and faster. But

four participants explained that it would be better if the interface provides more accurate and

related keywords corresponding to their search query keywords. Almost all participants are

satisfied with search within current results option and said that the search option for searching

current results helped in easy navigation of search results and refine the search results by

searching using different keywords.

Every participant is satisfied with the advanced search and found it very useful

particularly the faceted search option. 14 out of 16 participants told that the faceted category

search option was very simple to use which is also more useful to navigate the search by

selecting the desired facet category. 6 out of 16 participants felt that the advanced search option

to place constraints on search query could have more options to refine the search query to get

more desired results. A participant told that “Faceted category was better than advanced search

option of placing constraints on search query that makes searching easier”. Thus the new

interface design was found more useful and better than current existing search systems

particularly Google and Delicious taken for exploratory search task experiment by all 16

participants. Thus the summary of participants feedback showed that all participants regarded

that the search option, keyword suggestion, advanced search and the overall alternative interface

design would support and make their exploratory search better and easier.

4.8. Limitations of the study

The main limitation of this research study was the small sample of population used for

data collection. If more time and resources are available, more participants with different internet

user level, different age and with different profession based on domain knowledge could have

been recruited to involve in this research study. The overcome of this limitation would have let

to the development of better alternative interface to satisfy all types and level of users. Most of

the users were not aware of any exploratory search systems. During the evaluation of Delicious

during experiment, most of the users who were not aware of ESSs were not aware how to use the

Delicious for their exploratory search. Even though they were taught how to use it, they were not



85

given more time to learn better due to time limitation. The overcome of this limitation would

have let to more determined evaluation results. Also due to time and resources limitation only

one of the search engine Google and one ESS Delicious was taken for experimental evaluation to

satisfy the evaluation of existing search systems. The overcome of this limitation would have let

to more findings that would have acted as a base for better alternative interface design. The

design of alternative interface was also restricted from adding more additional features due to

time limitation.



86

CHAPTER 5

CONCLUSION



87

With rapid growth of World Wide Web, the number of users using internet for most of

their daily activities and search for information for most has grown due to which there is need for

search system to assist all types of users in effective information retrieval. This research study

started with review of existing literature regarding exploratory search, search engines and

exploratory search systems (ESSs), various techniques and approaches to support exploratory

search. These all acted as a basic for design of survey questionnaire, evaluation questionnaire

following experimental evaluation of search engine and ESS and interview. After analyzing the

results collected through above data collection methods, some important results and findings

were summarized. These along with review of existing literature acted as a base for HTML based

alternative interface designed using JavaScript scripting language. The alternative interface was

designed with aim to make users’ exploratory search better and easier. Finally users were asked

to evaluate the alternative interface and provide feedback on the alternative interface for the

support provided by it for their exploratory search.

5.1. Conclusions

The main objective of this research study was to evaluate the existing search systems for

the support provided by it for users exploratory search task and to develop an alternative

interface that support better users exploratory search task, make exploratory search easier with

better search options and efficient information retrieval. Number of quantitative and qualitative

methods was carried out to attain the objectives. First review of existing literature on exploratory

search and existing search systems was carried out. Also approaches and techniques to support

exploratory search was also reviewed.

The survey was conducted to identify the users exploratory search behavior, their

experience of exploratory search, their experience with existing search systems, their awareness,

preference and ability to use existing search systems for their exploratory search tasks and

preference of techniques and approaches to support their exploratory search task. The survey was

carried from participants recruited based on domain knowledge related to objective of this

research study. The analysis of results from survey questionnaire proved that every user use

internet daily and all have some experience of exploratory search. Most of the users (43.75%)

found to have the problem of undefined search goal for their exploratory search. Most of the



88

users (56.25%) are not aware of ESSs and they use only search engine for their exploratory

search among which Google was used by most of the users (62.5%). Users who use ESSs for

their exploratory search are found to be intermediate or experienced users as opposite to search

engine users are found to be expert or experienced users. Both the users of search engine and

ESSs found that neither supports their exploratory search better. Most of the users preferred

approaches such as query suggestions (37.5%) and view-based search (31.25%) for exploratory

search and only half of the users (50%) are aware of various techniques to support exploratory

search among them most of them preferred tags (37.5%) and faceted interface (25%). The

analysis and summary of above results and finding helped to find solution for the first research

question ‘What will be the searching behavior of users for exploratory search task?’

The same participants responded for survey were asked to evaluate the existing search

engine Google and social tagging Delicious which was exploratory search system as identified

by past researchers. Following the evaluation, the participants were asked to answer the

evaluation questionnaire to identify the users’ satisfaction with existing search systems and the

support provided by those systems for users exploratory search task. The analysis of results from

evaluation questionnaire showed that users found Google was easy to use and easy to learn with

average user ratings of 4.75 and 4.73 compared to Delicious and any other systems. Also users

are most satisfied with overall performance of Google with average user rating of 3.93. But in

terms of effectiveness of system in helping users to complete their tasks, satisfaction of search

results provided for search query and in providing quality of information to satisfy users’

information needs with average user ratings of 4.13, 4.0 and 3.5, users found Delicious were

better than Google. Users found Google and Delicious are at same level in terms of efficiency

with average user ratings of 4.06 and 3.94 and in organization and presentation of results with

average user ratings of 3.69 and 3.62. The participants were also interviewed to collect

qualitative information from them based on evaluation to know their needs of system for

exploratory search task. The analysis of answers from interview helped to find the users

experience with Google and Delicious, problem faced by participants using those systems for

their exploratory search task, merits and demerits of those systems and enhancements needed in

those systems to support their exploratory search better. The main advantage found by users with

Google was easy to use and easy to learn and with Delicious users found the advantage of query



89

suggestions in terms of tags. The problems faced by users with Google were difficulty in query

formation, inefficient for exploratory search task completion, ranked order of results, unwanted

and results irrelevant to users search query and the problems faced by users with Delicious were

description, meaning of tags and use of tags in different languages. The users suggested for

enhancements in Google such as query suggestions, providing more accurate and relevant results

for user search query and in Delicious they suggested it should provide standard and good

description of tags. Also users suggested that both Google and Delicious should provide better

search options and classification of search results. The analysis and summary of above results

and finding helped to find solution for the second research question ‘To what extent the existing

search tools support exploratory search?’

The searching behavior of participants was analyzed to identify the role of domain

knowledge for exploratory search. The analysis of searching behavior of users for exploratory

search task showed that domain knowledge is important for users for their exploratory search but

the role of domain knowledge may be weakened depending on the system used by users for their

exploratory search. The above analysis helped to find solution for the third research question

‘What is the role of domain knowledge in exploratory search, how did it influence the search and

the information collected?’

Based on the results and findings of survey and evaluation questionnaire, an alternative

interface was designed with aim to satisfy users’ need of system to support their exploratory

search better. Same participants used for previous research methods were used to evaluate the

alternative interface and provide feedback on the new interface for the support it could provide

for their exploratory search. The analysis of answers showed that users found the alternative

interface could support their exploratory search better and make it easier. The users are satisfied

with search options, query suggestions and advanced search options such as placing constraints

on search query and faceted category search option and found the above features are more useful

compared to search systems evaluated during experiment. But users also suggested some

improvements such as suggestion of more accurate and relevant keywords by the system. The

analysis of above results helped to find solution for the fourth research question ‘Would the

proposed interface technology make change and support exploratory search behavior?’



90

5.2. Implications of the Study

The implications of this research study are

This research study creates awareness about exploratory searching behavior,

search engines, exploratory search systems and search techniques

This study can act as base for development of better real time search system to

support and satisfy information needs of exploratory search

Every people from professionals to common people and from different

geographical area will be interested about this research

5.3. Suggestions for Future Work

This research study could acts as literature for future research related to this study. The

alternative interface along with some suggestions for improvement could act best literature and

can act as base for design of better system. As participants suggested, the interface should

provide accurate and related keywords corresponding to user search query using semantic

interpretation of users search query. Also as suggested by participants, the advanced search

option for placing constraints on user should be designed with more options to help users t find

definite query to filter results to get more accurate and desired results. It would be better for view

based search of facet search category in advanced search option to generate dynamic facets

during runtime based on users search query. For result exploration, it would be better to have

visualization of search results such as textual visualization, graph visualization and visualization

through semantic clusters. Content based search such as query by example for image search will

also provide better search option to query based on examples provided and find similar

documents. The alternative interface was designed only to enhance existing systems by

identifying enhancements in those systems so it does not employ any real time algorithm and

database. So it will be better to develop a real time information retrieval system using real time

algorithm and database to meet real time users’ information needs.

(Word Count: 20493)



91

BIBLIOGRAPHY

Aurnhammer Melanie, Hanappe Peter and Steels Luc (2006) Augmenting Navigation for

Collaborative Tagging with Emergent Semantics, Cruz I., Decker S., Allemang D., Preist C.,

Schwabe D., Mika P., (Ed.), ISWC 2006: Proceedings of the 5th International Semantic Web

Conference, LNCS 4273, pp. 58-71, Athens, GA, USA: Springer-Verlag, Berlin Heidelberg.

Bates, M. J. (1989) The design of browsing and berrypicking techniques for the online search

interface. Online Review, 13 (5), 407–424.

Berners-Lee, T., Chen, Y., Chilton, L., Connolly, D., Dhanaraj, R., and Hollenbach, J. (2006)

Tabulator: Exploring and analyzing linked data on the semantic web, Proceedings of the 3rd

International Semantic Web User Interaction Workshop, SWUI06, 2006

Blaikie Norman (2004) Analyzing Quantitative Data, From Description to Explanation, SAGE

Publications, London, 2004

Bryman Alan and Bell Emma (2003) Business Research Methods, Oxford University Press, 2003

Bryman Alan (2004) Social Research Methods, Second edition, Oxford University Press, 2004

Bryman Alan (2008) Social Research Methods, Third edition, Oxford University Press, 2008

Capra Robert G. and Marchionini Gary (2008) The Relational Browser Tool for Faceted

Exploratory Search, JCDL ’08, June 16-20, 2008, Pittsburgh, Pennsylvania, USA.

Chi Ed.H. and Nairn Rowan (2009) Information Seeking with Social Signals: Anatomy of a

Social Tag-based Exploratory Search Browser, CHI 2009, Boston, USA.



92

Christel Michael G. (2008) Supporting Video Library Exploratory Search: When Storyboards are

not Enough, Proceedings of the 2008 International Conference on Image and Video Retrieval

CIVR ’08, July 7-9 2008, Niagara Falls, Canada.

Cove J. F. and Walsh B. C. (1988) Online Text Retrieval via Browsing, Information Processing

& Management, 24(1), pp 31-37

Ding Wei, Marchionini Gary and Soergel Dagobert (1999) Multimodal Surrogates for Video

Browsing, DL ’99, Proceedings of the Fourth ACM conference on Digital libraries, New York,

USA.

Dork Marian, Carpendale Sheelagh, Collins Christopher and Williamson Carey (2008) VisGets:

Coordinated Visualizations for Web-based Information Exploration and Discovery, IEEE

Transactions on Visualizations and Computer Graphics, 14(6), 1205-1212.

Duggan, G.B., and Payne, S.J. (2008) Knowledge in the head and on the web: using topic

expertise to aid search. Paper presented at the SIGCHI conference 2008, Florence, Italy.

Flick Uwe, (2002), An Introduction to Qualitative Research, Second edition, SAGE Publications,

London.

Fu, W.T. (2008) The Microstructures of Social tagging: a rational model, Paper presented at the

ACM 2008 conference on Computer Supported Cooperative Work, San Diego, CA, USA.

Hearst Marti A., Yee K.P. and Swearingen K. Li. (2003) Faceted metadata for image search and

browsing, Proceedings of the ACM Conference on Human factors in Computing Systems, pp.

401–408, Fort Lauderdale, FL, USA: ACM Press, 2003.

Hemel Sven Van, Paepen Bert and Engelen Jan (2003) Smart Search in Newspaper Archives

Using Topic Maps, ELPUB 2003, From Information to Knowledge: Proceedings of the 7th



93

ICCC/IFIP International Conference on Electronic Publishing, University of Minho, Portugal,

25-28 June, 2003.

Hearst Mart A. (2006) Design Recommendations for Hierarchical Faceted Search Interfaces,

ACM SIGIR Workshop, Seattle, WA, USA, August 2006.

Huynh D.F., Miller R., and Karger D. (2007) Exhibit: Lightweight structured data publishing, in

Proceedings of the World Wide Web Conference, pp. 737–746, Banff, Alberta, Canada: ACM

Press, 2007.

Hwang Inbeorn, Kahng Minsuk and Lee Sang-goo (2011) Exploiting User Feedback to Improve

Quality of Search Results Clustering, ICUIMC ’11, February 21-23, 2011, Seoul, Korea.

Hyvonen Eero, Junnila Mikka, Kettula Suvi, Makela Eetu, Saarela Samppa, Salminen Mirva,

Syreeni Ahti, Valo Arttu and Vilijanen Kim (2004) Publishing Museum Collections on the

Semantic Web- The MuseumFinland Portal, WWW2004, May 17-22, 2004, New York, USA.

Indratmo (2010) Supporting Exploratory Browsing with Visualization of Social Interaction

History, University of Saskatchewan Library, Saskatoon, pp. 19-23.

Internet World Stats (2010) Usage and Population Statistics, June 30, 2010, Miniwatts Marketing

Group, Available at: www.internetworldstats.com/stats.htm (Last Accessed on 13/04/2011).

JcsCholtes (2010) Understand the two different faces of search: exploratory search and focalized

search, eDiscovery and information Management, Cae Assessment, Use Enterprise Information

Management Principles for eDiscovery, Litigation Readiness and Early, Available at:

http://zylab.wordpress.com/2010/03/26/understand-the-two-different-faces-of-search-

exploratory-search-and-focalized-search/ (Last Accessed on 21/06/2011).



94

Kammerer, Y., Nairn, R., Pirolli, P., and Chi, E.H. (2009) Signpost from the masses: learning

effects in an exploratory social tag search browser, Paper presented at SIGCHI conference 2009,

Boston, USA.

Kerr Bernard, Millen David R. and Feinberg Jonathan (2006) Dogear: Social Bookmarking in

the Enterprise, CHI 2006 Proceedings, Social Computing 1, April 22-27, 2006, Montreal,

Quebec, Canada.

Krause, B., Hotho, A., and Stumme, G. (2008) A Comparison of Social Bookmarking with

Traditional Search, Advances in Information Retrieval, ECIR 2008, LNCS 4956, pp. 101-113,

2008

Kules Bill, Capra Robert, Banta Matthew and Sierra Tito (2009) What do Exploratory Searchers

Look at in a Faceted Search Interface?, Joint International Conference on Digital Libraries

(JCDL), 2009, Austin, Texas.

Marchionini, G. (1997) Information Seeking in Electronic Environments, Cambridge University

Press, June 1997.

Marchionini, G., Brunk, B., Komlodi, A., Conrad, F., and Bosley, J. (2000) Look before you

click: A relation browser for Federal statistics websites, Proceedings of the ASIS 2000 Annual

meeting, Chicago, IL, November 15-18, 2000.

Marchionini Gary (2006) Exploratory search: From finding to understanding, Communication of

the ACM April 2006/ Vol. 49, No. 4.

Mercun Tanja and Zumer Maja (2009) Visualizing for Exploration and Discovery, Libraries in

Digital Age, LIDA 2009, Zadar, Croatia



95

Millen, D. R., Yang, M., Whittaker, S., and Feinberg, J. (2007) Social bookmarking and

exploratory search, Paper presented at the ECSCW 2007, Limerick, Ireland.

Mirizzi Roberto and Noia Tommaso Di (2010) From Exploratory Search to Web Search and

back, PIKM ’10, October 30,2010, Toronto, Ontario, Canada.

Moussa Maher Ben, Pasch Marco, Hiemstra, Der Vet Paul Van and Huibers Theo (2006) The

Potential of User Feedback through the Iterative Refining of Queries in an Image Retrieval

System, AMR ’06 Proceedings of the 4th

International conference on Adaptive Multimedia

Retrieval: user, context and feedback, Geneva, Switzerland, July 27-28, 2006.

Mulholland P., Zdrahal Z. and Collins T. (2008) Investigating the Effects of Exploratory

Semantic Search on the Use of a Museum Archive, IEEE 2008 International Conference on

Distributed Human-Machine Systems, 10-12 March 2008, Athens, Greece.

Park Hogun, Myaeng Sung Hyon, Jang Gwan, Choi Jong-wook, Jo Sooran, Roh Hyung-chul

(2008) SketchBrain: An Interactive Information Seeking Interface for Exploratory Search, HCIR

2008, Redmond, WA.

Punch Keith F. (1998) Introduction to Social Research, Quantitative and Qualitative Approaches,

SAGE Publications, London, 1998.

Rasmussen Edie (2008) Characterizing, Supporting and Evaluating Exploratory Search, NSF IS3

Workshop, Chapel Hill, NC, USA

Rudestam (2007) The Research Process, February 2007, Available at: www.sagepub.com/upm-

data/14647_Chapter1.pdf



96

Schraefel M.C., Wilson Max, Russell Alistair and Smith Daniel A. (2006) MSPACE: Improving

Information Access to Multimedia Domains with Multimodal Exploratory Search,

Communications of the ACM, Apr2006/ Vol. 49, No. 4.

Seifert Inessa and Kruppa Michael (2010) A Pool of Topics: Interactive Relational Topic

Visualization for Information Discovery, ECDL'10 Proceedings of the 14th European conference

on Research and advanced technology for digital libraries, Springer-Verlag Berlin, Heidelberg.

Shneiderman, B., and Kules, B. (2008) Users can change their web search tactics: Design

guidelines for categorized overviews, Information Processing and Management Journal, Vol. 44,

no. 2, pp. 463-484.

Spink, A. (2002) A user-centered approach to evaluating human interaction with Web search

engines: an exploratory study, Information Processing and Management, 38, 401-426.

Stake, R. E. (1994) Handbook of qualitative research, pp. 236-247, Thousand Oaks, CA: Sage.

Stevens, S.S. (1946) On the theory of scales of measurement, Science, New Series, Vol. 103,

No.2684, Jun 7,1946, pp. 677-680, American Association for the Advancement of Science.

The University of Sheffield (2011) Research Ethics: General Principles and Statements, The

University of Sheffield’s Ethics Policy Governing Research Involving Human Participants,

Personal Data and Human Tissue: Version 2, 2011, Available at:

http://www.shef.ac.uk/content/1/c6/11/15/24/General%20Principles%20and%20Statements%20

vs2.pdf (Last Accessed on 04/05/1011)

Trochim William, M. K. (2006) Research Methods Knowledge Base, Available at:.

www.socialresearch- methods.net/kb/index.php (Last Accessed on 10/05/2011).

Villa Robert, Gildea Nicholas and Jose Joemon J. (2008) FacetBrowser: A User Interface for

Complex Search Tasks, MM ’08, October 26-31, 2008, Vanconver, British Columbia, Canada.



97

Wang Shuo, Jing Feng, He Jibo, Du Qixing and Zhang Lei (2007) IGroup: Presenting Web

Image Search Results in Semantic Clusters, CHI 2007, April 28-May 3, 2007, San Jose,

California, USA.

White Ryen, W., Muresan Gheorghe, Marchionini Gary (2006) Evaluating Exploratory Search

systems, SIGIR 2006 Workshop, Seattle, USA.

White Ryen, W., Marchionini Gary, Muresan Gheorghe (2008) Evaluating Exploratory Search

Systems: Introduction to Special Topic Issue of Information Processing and Management,

Journal of Information Processing and Management- IPM, vol.44, no.2.

White Ryen, W., Dumais, S.T. and Teevan, J. (2009) Characterizing the influence of domain

expertise on web search behavior, Paper presented at the Second ACM International Conference

on Web Search and Data Mining, Barcelona, Spain.

Wilson, M. L. and Schraefel, M.C. (2008) Evaluating Collaborative Search Interfaces with

Information Seeking Theory, 1st International Collaborative Search Workshop, 20th June 2008,

Pittsburgh, PA, USA.

Wilson Max L., Kules Bill, Schraefel, M.C. and Shneiderman Ben (2010) From Keyword Search

to Exploration: Designing Future Search Interfaces for the Web, Foundations and Trends in Web

Science, Vol. 2, No. 1 (2010).

Zhang J. and Marchionini G. (2005) Evaluation and evolution of a browse and search interface:

Relation browser, Proceedings of the National Conference on Digital Government Research, pp.

179–188, Atlanta, GA, USA: Digital Government Research Center, 2005.



98

APPENDICES

APPENDIX 1

Survey Questionnaire

1. How often do you use the internet?

Daily Several times a week Weekly

Several times a month Monthly Never

2. Do you have any experience of exploratory search?

Yes No Preferred not to say

3. What is the main problem faced by you for exploratory search?

Unclear search goal

Unaware of how and where to find the information

Search system does not support better exploratory search

4. Are you aware of various exploratory search systems?


5. What do you use for exploratory search?

Search Engines Exploratory Search Systems Both

6. If you use search engine for exploratory search, how would you describe your ability to

use them?




99

7. If you use exploratory search systems for exploratory search, how would you describe

your ability to use them?


8. If you use Search engine, which one do you use most for exploratory search?

Google Bing Yahoo Live Search

Others

9. Among the following which type of approach do you prefer most for exploratory search?

Content-based search

Interface system

Visualization of search results

Query suggestions

10. How far existing Search engines or exploratory search systems support exploratory

search?

Bad ……….. 1

Fair ……….. 2

Good……… 3

Very Good... 4

Excellent….. 5

11. Are you aware of various techniques that will support exploratory search?




100

12. If answered yes to above question, which technique you think will support better

exploratory search?

Faceted Interface

Visualization

Clustering

Semantic web

Tags

APPENDIX 2

Evaluation Questionnaire


the system

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree


the use of system

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

3 I can effectively

complete task

using system

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

4 Efficiency of

system is good

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree



101

5 Satisfied with

search results

provided by

system

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

6 Organization of

results is good and

useful

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

7 Satisfied with

information

acquired during

search

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

8 I am overall

satisfied with

system

1-Strongly

Disagree


Disagree or

Agree

4-Agree 5-Strongly

Agree

COMPARISON OF GOOGLE AND DELICIOUS

Google Delicious


the system

4.75 4.38


the use of system

4.63 4.25

3 I can effectively

complete task

using system

3.88 4.13



102

4 Efficiency of

system is good

4.06 3.94

5 Satisfied with

search results

provided by

system

3.81 4.0

6 Organization of

results is good and

useful

3.69 3.62

7 Satisfied with

information

acquired during

search

3.19 3.5

8 I am overall

satisfied with

system

3.93 3.81

APPENDIX 3

Interview Questions

1. What are the problems and difficulties faced by you when you are using the search systems for

exploratory search task?

2. What are the merits and limitations of the search systems?

3. What are the improvements needed for the current search systems?

4. What are the enhancements and features needed for the future development of search systems?

Documents

EVALUATING USERS EXPLORATORY INFORMATION SEEKING …dagda.shef.ac.uk/dispub/dissertations/2010-11/External/... · 2012. 3. 27. · i ABSTRACT Background The increase in number of