WP-ESIP Federation Interoperability

JAMES FREW

University of California, Santa Barbarahttp://www.bren.ucsb.edu/~frew

WP-ESIP Federation Interoperability

• Background– CAN interoperability requirements– Federation Interoperability Working Group

(FIG)

• Interoperability criteria– System-wide Interface Layer (SWIL)– Catalog interoperability

• Proposed technologies

CAN Interoperability Requirements

• Specific– Make public-domain products available on

Internet– Announce products and services through

GCMD– Comply with Federal standards (e.g. FGDC)

• Philosophy– Interoperability best resolved experimentally– Federation must decide how much

CAN Interoperability Requirements

• Process– Each ESIP proposes one of

{V0, ECS, CIP, FGDC GEO, custom}as System-Wide Interface Layer (SWIL)– custom: “permits the ESIP to be searched and

queried as if it is part of a larger whole”

– Federation determines and evolves these standards and interfaces– SWIL-specific funding available

cluster

What is the SWIL?

ESIP

ESIP ESIP

ESIP ESIP

SWIL

customer

customer

A common viewof the Federationthat all its participantsagree to support

SWIL Elements

• Online services– How you can reach us

• Vocabularies and models– What language(s) we speak

• User interfaces– What we look like

Federation Interoperability Working Group (FIG)

• May 1998 (1st Federation meeting)– FIG established; charged with coordinating

development of SWIL– Kenn Gardels elected chair

• Summer/Fall 1998– Inventory relevant systems, protocols, and

standards, and ESIP activities

FIG Timeline (cont’d)

• Dec 1998 (2nd Federation meeting)– Endorse layered implementation strategy

• metadata data functions

– Endorse clusters of ESIPs as “bottom-up” interoperability mechanism

• Winter/Spring 1999– FIG “tiger team” prepares catalog

interoperability (CI) evaluation criteria– April 1999: loss of Kenn Gardels;

Yonsook Enloe acting chair

FIG Timeline (cont’d)

• May 1999 (3rd Federation meeting)– CI evaluation criteria presented and approved– James Frew elected chair

• Summer 1999– CI-level SWIL candidate systems solicited

• 4 proposals as of 13 Jul 1999• Evaluation team forming

– 30 Aug - 02 Sep 1999• FIG at UCLA: synthesize CI SWIL

• Light touch– Just metadata, not data

• Satisfies basic requirements– GCMD– FGDC

• Satisfies “query larger whole” sub-requirement

• max(!/$): best chance to do something quickly– Many existing or pending alternatives

Catalog Interoperability Rationale

Overall Criteria

• Allow single, multiple, or composite solutions– Multiple: must be equivalent– Composite: should be seamless

• Security and access control– Expose subsets of catalog information

• Comply with relevant standards

• Discover and describe services as well as data

Overall Criteria: Risks

• Maturity• Acceptance

– By users– By providers

• Support• Technological change

– Continue to support “obsolete” technologies– Migrate to newer technologies

Catalog Interoperability Criteria

• Discovery / search• Browse• Logical data

model• User interface

• Local extensibility• Technology• Scalability /

Bottlenecks• Costs• Compatibility

Discovery

• Specificity– Collection– Granule

• Retrieval capabilities– Ranking– Relevance– extent of search

compliance

• Search capabilities– Geospatial

• “bounding-box”– including Z

– “Fielded search”– Free text– Temporal– Common vs. local

attributes

Search

Browse

• Specificity– By collection

• E.g. coverage summaries

– By granule

• Options– Static

• Fixed parameters

– On-demand• User-specified

parameters

• Vocabularies– Valids / Domains– Use applicable

standards

• Inter-attribute relationships– Parent-child– Thesauri– Other TBD

Data Model

User Interface

• Implementation– Web browser– Other clients

• Java app• Z39.50• Internet search

engines

• Extensibility– APIs

• Open & complete

– Encodings• XML

• Attributes• Vocabularies• Search capabilities• Retrieval

capabilities• Data access

• Provide access to local extensions

Local Extensibility

Technology

• Portability– Platform

dependencies

• Implementation– Language– communication

• Persistent connections

• Non-standard ports and/or protocols

• Firewalls

• Number of providers

• Number of users• Volume of data• Performance

– Rates– Latencies

• Asymmetric degradation

• Fault tolerance

Scalability and Bottlenecks

Costs

• “plug-in”– Purchase– Construction– Configuration– Administration

• Distribution– Providers– Federation

• Existing systems, clusters, and protocols– GCMD– V0– Z39.50

Compatibility

Proposed Interoperability Technologies

• GCMD• Mercury• EOSDIS Version 0• Big Sur• DIAL• MOCHA

Global Change Master Directory (GCMD)

• Purpose– Search and discovery tool for Earth science

data set descriptions– Metadata services: search– Data services: subscription– Direct links to data through alternative

interfaces– Z39.50 access to FGDC Clearinghouse

Mercury – Metadata Search and Data Retrieval

• Purpose– XML Web-based system providing common

view of disparate metadata and data files– Metadata services: directory-level search– Data services: search and access– FGDC and Z39.50 compliant

EOSDIS Version 0 – Metadata Publication and

Data Ordering• Purpose

– Automated search, order, and access for online and nearline archives

– Metadata publication: search and access– V0 protocol (PRODUCT_REQUEST message):

order and access– Access to local data services

Big Sur

• Purpose– Integrated, distributed data and metadata

for search, browse, and access– Metadata services: input data attributes;

data history– Data services: functional processing; links to

visualization and access tools– Accessible from platforms connected to a

Big Sur database

Data and Information Access Link (DIAL)

• Purpose– Web-based software tools for organizing

and distributing metadata– Metadata services: search, query, browse,

and access– Data services: access and order in multiple

formats– Dynamic visualization, X-Y plotting,

animation, subsetting, etc.– Integrated with EOSDIS V0 and GCMD

MOCHA - Middleware for Integrating Distributed

Data• Purpose

– Java architecture for integrating distributed heterogeneous data

– Metadata services: distributed queries using XML & RDF

– Data services: executes shipped code at data sources for filtering and aggregation

– Java middleware deploys plug & play code to data sources

– Use XML to exchange and interpret metadata and code

Conclusion

• “Bottom-up” interoperability is already happening– Web, clusters, etc.

• Federation-wide challenge:synthesize a common view– Cook up a nourishing batch of SWIL without

losing the flavors of each ingredient