Digital Library Issues and Trends

William H. Mischow-mischo@uiuc.edu

Grainger Engineering Library Information Center

University of Illinois at Urbana-Champaign

Technology, Scalability, and Metadata (or Not)

NSDL Panel, October 13, 2003

Digital Libraries• ‘Digital’, ‘Virtual’, ‘Electronic’ Library as

network-based library without regard to place and time.

• Tendency to apply term to digital collections and resources.

• Digital Collections vs. Digital Libraries.• Emphasis on the integration of collections

and services via standards and Web protocols and practices.

• Distributed, heterogeneous repositories require federation and linking.

Scholarly Communication Overview

• E-Resources are Web-based and publisher-centric.• Growth of Heterogeneous Distributed Repositories.• ARL libraries spent $132 million on e-resources in 2001.• 30K e-journals (TDNet), >33% available via multiple

aggregators and/or publishers.• Publisher’s provide value-added services and ‘branding’ of

journals. Prestige of journals and publishers in Academia.• Cooperation on reference linking standards (DOI,

OpenURL, CrossRef).• Alternative scholarly communication models - Academia

institutional repositories, preprint servers.

Distributed Information Environment• We live in a world of multiple, heterogeneous information

repositories, resources, portals, and IR systems. – OPACs: local, regional, national shared bibliographic databases.– Locally mounted and remote A & I Services.– Discrete publisher and vendor full-text repositories.– Open OAI services (OAIster at Michigan) and preprint servers.– Web search engines.– Vertical and custom publisher and vendor portals (ARL Portal,

DOE Information Bridge, Elsevier Scirus, EI Village, PubMed and BioMed Central, Public Library of Science, AIP, NSDL). Surface Web and Hidden Web.

– Local metadata, digital objects, GIS, finding aids.– Institutional Repositories (D-Space).– Instructional (course) management systems (WebCT, Blackboard).– Harvestable (OAI) sites and services.

Distributed Repository Issues • Metasearch (broadcast and federated) over distributed,

heterogeneous repositories.• Publishers and vendors efforts to become one-stop-shopping

portals: BigChalk, Elsevier, Ingenta, EbscoHost, AIP.• Ironic need to ‘unbundle’ DLs for optimum search (IEEE IEL,

ACM, EI Village, Elsevier, Wiley, Springer, Kluwer, AIP, etc.)• Role of A & I Services. Tenuous? But important in Local Link

Resolving.• Link Resolver Servers (vendor: SFX, Encompass) via DOI,

OpenURL (metadata), CrossRef.• Integration of collections with reference, instructional and

navigation services -TOC, collaborative mechanisms, remote reference assistance.

• Search and discovery navigation: Best-Match and Quorum Searching.

Digital Library Tools• We have at our disposal the tools to create

integrated digital libraries from the distributed digital resources environment in which we operate:– Standard retrieval environment (Web) and

interface/client (Web Browser);– Standard transport mechanisms to connect

heterogeneous content (HTTP, OAI, SOAP);– Standard metalanguages and tools for describing and

transforming content and metadata (XML, DTDs & Schemas, XSLT, DC/DCQ, RDF, METS);

– Standardized search/retrieval mechanisms (HTTP Post/Get, SQL, Z39.50, Object Oriented Databases);

– Standard linking tools and infrastructure (DOI, OpenURL, CrossRef).

• Candidate set of ‘best practices’ for IR.

Metadata Issues• Metadata vs. full-text search--not either-or situation.• Robust metadata industry: publishers (ISI, CA), libraries,

archives, museums, A & I Services.• Automatic extraction and generation of metadata. Role of XML.• Problems with reverse engineering metadata into large

unstructured chunks of text.• Image retrieval. David Forsyth and Jennifer Trant image

database.• Google (3 billion—4.2 billion). Open Web with no metadata vs.

Hidden Web (500 times larger?) with metadata.• Problem areas in full-text searching: authors, subject/author

browse, mathematics, precision, Google PageRank.• Users trust of Web searches (41%/51%) vs. library search

(99%/97%). DLF and Outsell study.• Metadata for presentation (MathML).• OpenURL and CrossRef and Link Resolvers.• Studies of OAI metadata show limitations in implementation, but

many have rich metadata (ArX, IOP) for S & D.

Ongoing Investigations• Relationship between interoperability models for

search and discovery: federated searching (OAI harvested) and broadcast, simultaneous searching of distributed repositories. Not mutually exclusive.

• OAI Provider and Harvesting software. Rights Management. Static Repositories.

• Mathematics -- MathML.• Reference Linking integration built on OpenURL

and DOI.• Search Assistant software with simultaneous

search, point-of-contact assistance, and remote reference capability.

MetaSearch Implementations• Ex Libris MetaLib.• Endeavor EnCompass (MuseGlobal).• Innovative Interfaces MetaFind.• Ovid Multiple Search and reference De-Duping.• ISI Web of Knowledge.• Gale Corporation InfoTrac Total Access.• WebFeat.• California Digital Library SearchLight system.• Fretwell-Downing.• Locals (Grainger Library, Los Alamos).

OpenURL-Based Services

• Standard for expressing and transmitting metadata.

• Promise of standardized, normalized search results.

• Use in Link Resolvers – commercial and local..

• Use of CrossRef metadata database to look up DOIs.

Continuing Issues• Role of Authors, Academic Institutions, Libraries,

Publishers, Abstracting & Indexing Services.• Rapid growth of remote access.• Disintermediation may affect both Libraries and

Publishers.• Information as Function not Place.• Provide ‘Digital Libraries’ out of digital collections and

services.• Web Services• Evolution of service mechanisms: processing &

archiving, search, remote reference, navigation enhancements, link resolving.

4th Generation Information Systems• Integration of heterogeneous information resources.• Broadcast and Simultaneous Searching of Multiple

Resources tailored to user needs.• Remote Reference and Instruction (Collaboration

software, multimedia, suggestion-based systems).• Integration with Learning Management Systems.• Software-Aided Search Navigation (Best-Match

searching) and Strategy Modification.• Dynamic Links to Full-Text. Appropriate Copy

problem.• One-Stop-Shopping.