Taking the Library Back from Google Abe Lederman, President and CTO iGroup, October 18-20, 2007

Taking the Library Back from Google

Abe Lederman, President and CTOiGroup, October 18-20, 2007

Abe Lederman Background Earned B.S. and M.S. Computer Science

degrees, MIT Began work in information retrieval in 1988

• Co-founded Verity Developed one of the earliest web-based

search & retrieval applications, 1994 Pioneered “Deep Web” searching in 1999

• Founded Deep Web Technologies, 2002

The “Google Myth”

If you can’t find it on Google,the information doesn’t exist.

Why Do Students Like Google and Google Scholar?

Quickly searches

accessible informationEasy to searchMulti-Lingual supportFREE

What Students Don’t know

Links for many articles on Google Scholar require subscriber login or an access fee.

Many libraries have already paid for subscriptions.

Other databases may be more relevant to the topic.

Google Scholar advanced search is not very powerful.

So Why NOT Google Scholar?

• “Scholarly content” intermixed with “non-scholarly” content

• Spotty coverage, requires publisher cooperation

• Poor relevance (citation counts don’t seem to help)

• Unable to limit searching to specific sources

• Difficult to find the needles in the haystack

Bringing Relevance Back to the Library

Checking the Library Website…• Reference Libraries

• Deep Web Databases

• Specialized Search Directories

• Specialized Search Engines

• Subscription Databases

• Library Catalogs

“Ask A Librarian!”

Federated Search…

Allows you to search, retrieve, and merge de-duped and relevantly ranked results from multiple high-quality information sources.

In Other Words…One Search, Many Sources

LibraryCatalog

Wikis

SubscriptionSources

Public Web Sources

Blogs

SubscriptionSources

Federated search by any other name….

Metasearch 6,690,000

Meta search 1,840,000

Federated Search 339,000

Distributed search 237,000

Broadcast search 106,000

Distributed information retrieval

70,200

Deep Web Search 24,600

* Number of occurences on Google (Sept. 21, 2007)

Tennant’s Tenets

Roy Tennant: Internationally known speaker and writer on library and information technology issues.

• Only librarians like to search, everyone else likes to find.

• All things being equal, one place to search is better than two or more.

• Services should be placed as close to the user as possible.

Tennant’s Tenets (cont.)

“Good enough” is often just that. Our ability to create effective one-stop

searching is dependent on our ability to appropriately target user needs.

The size of the result set doesn’t matter as much as how the results are presented. (‘the Google Lesson’)

Increase efficiency ofIncrease efficiency of information searchinformation search

Gather the most relevant,Gather the most relevant, important informationimportant information

Make better decisions Make better decisions & fewer mistakes& fewer mistakes

More Users, More Users, Find More Find More

InformationInformation

How can Federated Search help your Patrons?

Benefits of Federated Search

One-Stop AccessSupport of More Efficient ResearchConsolidated, Formatted ResultsAuthentication and Access Control

One-Stop Access Single query search across multiple

disparate sources in real-time:• Subscription services (journals, bibliographic

databases, etc.)

• In-house databases

• Unstructured data (white papers, theses, reports, etc.)

• Catalogs (web-based OPAC)

• Publicly available websites

Information Discovery

Support of More Efficient Research

Results across all selected sources are:• Merged

• Presented in relevance-ranked order

• Sorted

• Clustering

Consolidated, Formatted Results

Filtering and removal of duplicates Ranking by relevance Displaying results by source Permitting results to be

• Downloaded

• Printed

• Automatically emailed using Alerts

Flagging of results from subscribed sources

Authentication and Access Control

Authorization based on:• IP address

• User ID and password

Implementation using:• Proxy server

• Management of session cookies

Different access levels for different user groups• Virtual server support

#1 Benefit to Libraries

Improve your ROI on Licensed Content!

Integrating Trusted Science + Technology Research

What does Federated Search look like?

• 15 leaders in science and technology research• 3 million documents• 150 years of knowledge

Bringing Federated Search to Your Patrons

Clearly Define Requirements

Compile your list of sources to federate

• Access to Internal Sources Licensed Product vs. Managed Solution Staff Resources Timelines Determine features important to users

Will the Real Federated Search Please Step Forward…

Easy-to-Use, Customizable Interface Incremental display of real-time and indexed

results Clustering or Visualization Relevance ranking of results on-the-fly Support Boolean and fielded searching Access sources via multiple protocols

(XML gateway, HTTP, SR/U, SR/W, Z39.50)

Sample Features Checklist

• Integration with URL Resolver

• Integration with EndNote or other export capabilities

• Spell Checker

• Alerts Options

• Export to an RSS Feed

• Advanced Customizability

Vendor Evaluation

Demonstrations and Pilots• User reaction

• Ease of search

• Fulfillment of requirements and features

Relationship• Future Relationship

• Flexibility

The Future of Federated Search

Multi-Lingual Searching Personal Libraries Automated Source Selection

Conclusion

Help Patrons to…• Improve their search

• Find what they need

• Access scholarly sources

• Return to your library

for information

Thank You!

Abe Ledermanabe@deepwebtech.com