Upload
mervin-briggs
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
JISC/NSF PI Meeting, June 24-25
Archon - A Digital Library that Federates Physics Collections with Varying Degrees of Metadata Richness
Department of Computer Science
Old Dominion University, Norfolk, VA 23529K. Maly, M. Zubair, M. Nelson
In Collaboration With
Los Alamos National Laboratory (R. Luce)&
American Physical Society (M. Doyle)
Motivation
Lack of a federation service that provides an unified interface to diverse collections in the physics domain having metadata that differ in richness, syntax, and semantics
Motivation• Dissemination and discovery of Physics resources
• Contributors
LANL, APS, AIP, CERN
researchers, teachers
• Users
Students, teachers, researchers
Arc: The Basic Federation Engine
User Interface
Search Engine (Servlet)
JDBC
Data Normalization
History Harvest
Daily Harvest
Data Provider
Data Provider Cache Oracle MySQL
Harvester
Grouper Local Query Cache and Session Related Date
Displayer
Database (Metadata &
Index)
Searcher
Session Manager
Arc: The Basic Federation Engine
Challenges• Resource Discovery
– Diversity in metadata richness– Lack of controlled vocabulary– Ease of discovering (formula based discovery)– Cross linking support– Classification
• Creation and Maintenance– Freshness of metadata– Dynamic nature of collections– Filtering
• Economic Sustainability– Rights management– Who pays? For what?
Issues – No controlled vocabulary
• Different subject classifications• Same authors but different rendering• Same affiliation but different form
Interactive resource discovery approach components
Ha rve st e r
I nd e xG e ne ra to r f o rUn io n o f Ke y
M e t ad a ta F ield s
Ha rve st e dM e ta d a ta
I n de x edfie ld
co nt e nt s
Us er In te rfa c e
S ea rch En g in e
122
1 Use r in t era ct t o ide n tif y a ll th e c olle ct ion s to b ese a rch e d a nd w it h wh a t a ll o p tio n s.
Use r ex ec u te s ea rch ba se d o n t he se lec te do pt ion s
Issues - Equation based search
• Representing search query• Rendering of equations and embedding them into
the HTML display• Integrating into search interface• Identifying equations inside the metadata• Filtering equations• Equation storage
DisplayEqn
EqnFilter EqnRecorder
Img2Gif
EqnExtractor Acme.JPM.Encoders.GifEncoder
Eqn2Gif
EqnSearch oai.search.Search
cHotEqn MathEqn
EqnCleaner
Eqn Data DC Metadata
Servlet
Image Converter
Formula Filter
Formula Extractor
Filtering Equations
• Errors in equation encoding, some examples:– missing "$" in LaTeX representation– illegal LaTeX symbols
• Simple equations like "n=3"
Filtering/categorizing Equations
Approach:
Use of "Stop Equation File" similar to "Stop Word File" used for indexing.
In equation filtering context, the stop equation file consists of rules in form of regular expressions, which describe the LaTeX string to be dropped. The regular expression approach gives us the flexibility to describe easily variety of strings to be filtered.
How to search for records using equations?
Three search alternatives (or any combination of these) for the user:
•Search for docs containing all formulae found in a) abstracts b) subject fields of documents containing user input ‘keywords’•Search for docs containing formulae defined by category (e.g. integrals, moments, limits)• Browse formulae by various categorizations and search for docs containing selected formulae
Issues - Cross Linking References
• Obtaining references from full-text documents or parallel metadata sets
• Bad format of such references when obtained from full text
• Needed standard way to represent across collections
Issues – Name similarity
• Authors use different names for themselves and their affiliation
• Could use authority files, difficult to create and maintain across different collections
Similarity approach
Clustering
Iterative refinement approach:
•Coarse level clusters based on approximate string matching (edit-distance, soundex, n-gram)
•Refining clusters based on affiliation where available
Presentation
Allow user to follow search by clicking authors and then selecting appropriate, i.e., no authority files
Homogenizing User Space
• Enabling Web users to discover information in OAI collections (DP-9 Service)– http://arc.cs.odu.edu:8080/dp9/
• Enabling OAI users to discover information in Web enabled non-OAI compliant collections/databases/web sites
DP-9 Service for Exposing OAI Collections to Web
Web EnabledNon-OAI Compliant
Collections/Databases/Web Sites
Web EnabledNon-OAI Compliant
Collections/Databases/Web Sites
Web EnabledNon-OAI Compliant
Collections/Databases/Web Sites
OAI Service Provider
Gateway to Non-OAICollections
WIDL Description(XML based language)
WIDL Description(XML based language)
WIDL Description(XML based language)
Vac: Gateway Service for Harvesting Non-OAI Collections
Sample Description in WIDL of a Web enabled Non-OAI Collection
<WIDL NAME=‘’NonOAIGateway" Template=‘’TRcollector" BASEURL="http://www.princeton.edu" VERSION="2.0">
<SERVICE NAME=‘’getURL" METHOD="GET" URL="" INPUT=‘’" OUTPUT=‘’urlOutput" />
</BINDING> <BINDING NAME="urlOutput" TYPE="OUTPUT"> <VARIABLE NAME=‘’link" TYPE="String"
REFERENCE="doc.p[1].text" /><VARIABLE NAME=‘’title" TYPE="String" REFERENCE=‘’title" /> <VARIABLE NAME=‘’author" TYPE="String"
REFERENCE=‘’author" /><VARIABLE NAME=‘’descriptionr" TYPE="String"
REFERENCE=‘’abstract" /></BINDING> </WIDL>
Federation/archives Consistency
User Interface
Search Engine (Servlet)
JDBC
Data Normalization
History Harvest
Daily Harvest
Data Provider
Data Provider Cache Oracle MySQL
Harvester
Future Tasks
• Post processing of search results for easier navigation• Exploiting richer metadata and handling diversity in metadata
across all participating collections• Concentrate on interactive search interface for resource
discovery• Data normalization, authority files, filtering• Investigating different schemes for maintaining
federation/archives consistency• More high level services beyond formula based search and
cross-linking• User testing!!!!
Links
• ODU DL research group:– http://dlib.cs.odu.edu/
• Main federation engine:– http://arc.cs.odu.edu/
• NSDL research:– http://archon.cs.odu.edu/
• ITR/IM research– http://kepler.cs.odu.edu/
Not used
Automated metadata mapping approach
Los A lam osC ollection
Am ericanPhysica lSoc ie ty
C ollection
ArcService P rovider
O AI Layer O AI Layer O AI Layer
H arvester
M etadataProcessor
H arvestedM etadata
U nified andN orm alizedM etadata
U ser In terface
Search Engine
Registration Server(XML m apping for
each DP)
Nam eauthority
file
T R IService P rovider
O AI Layer