NextGen Network-Enabled Weather (NNEW) Registry/Repository

DAWG-RegRep-1OJN/KC 8/5/2008

MIT Lincoln Laboratory

NextGen Network-Enabled Weather (NNEW)

Registry/Repository

Oliver Newell, Kajal Claypool

5 August 2008

MIT Lincoln LaboratoryDAWG-RegRep2OJN/KC 8/5/2008

Overview

• Registry/repository background

• NNEW registry/repository use case overview

• Registration of 4-D cube data domain taxonomy

• Registration of a dataset

• Basic dataset and service provider discovery

• Enhanced discovery using Ontologies


Key Roles of a NNEW Registy/Repositry

• Build-time– Storage/discovery of service interface descriptions (WSDL and

associated schemas)– Storage of dictionary/vocabulary information (e.g. coordinate reference

system dictionary, upper-level weather ontology)

• Run-time– Discovery of datasets and their

associated data access services using high-level metadata

– Dataset metadata within registry includes weather cube domain ‘membership’ (e.g. SAS)

– Dataset can be a member of more than one domain


NNEW Registry/Repository

• Universal Description, Discovery, and Integration (UDDI) Registry

– Underpowered with respect to NNEW use cases Limited query capabilities (especially geospatial queries)

– Industry support on the wane… IBM, Microsoft (co-creators of UDDI spec) closed public registries

in 2006 IBM now offers more capable (but proprietary) registry/repository

solution OASIS UDDI Technical Committee now inactive

• ebXML Registry/Repository– More flexible, extensible solution– Freely-available version (freebXML) used in variety of domains

DOD Metadata Repository uses freebXML ‘under the hood’

• NNEW Approach– Adopt ebXML registry/repository – extend as needed– Partner with ebXML registry/repository vendor (Wellfleet)– Formalize extensions with OASIS ebXML technical committee


ebXMLRegRep

Standard

Metadata

Fede

rate

d

Info

rmat

ion

Man

age-

men

t

SO

A

Gov

erna

nce

Events

Secure

Architecture

Cataloging, Validation, Version Control,Lifecycle Support,Extensible Info Model

Taxonomies, Classifications, Relationships

Federated Queries, Inter-registry links

Digital Signatures,Audit Trail,Access Control,SAML SSO

SOA Registry

Repository

Registration, Discovery, Queries

Content-Based Event Notification

Key features of ebXML RegRep


DOD Metadata Repository

Source: <https://metadata.dod.mil/mdr/about.htm>

Web GUI

• Based upon freebXML Registry OS project

• Customized web based GUI client

• Manages schemas, data elements, attributes, document type definitions, style-sheets, data structures

• Not currently used for registration of services for run-time discovery purposes– ebXML reg/rep does support the run-time use case


NNEW Registry-Repository/Catalog

• For 2007/2008 demos, using early release of ebXML RegRep 4.0 – compliant regrep (OASIS draft standard expected ~4th Qtr 2009)


Data Discovery and Access using OGC Services and ebXML Registry (Dec ‘07)

OGC service-enableddisplay Client

FAA Technical CenterNational Center for Atmospheric Research (NCAR)

Web CoverageService

(OGC WCS)

IcingTurbulence

Data

NOAA

SurfaceObservations

DataWeb Coverage

Service(OGC WCS)

MIT Lincoln Laboratory

CIWSForecast

Data

Catalog(ebxml)

Web CoverageService

(OGC WCS)

Virtual 4-D Weather Cube

“Discover Available Data Sets”

“Get data from time T1 to T2in spatial region A and return in NetCDF formatusing a Lambert ConformalProjection”

MIT/LL

NCAR

NOAA

RequestedData


Overview








Creation of 4-D Cube Single Authoritative Source Domain

“The designated administrator for the SAS domain accesses a 4-D Wx Data Cube registry and determines the identifiers of all datasets categorized as the SAS for their respective data type. A list of all the datasets is passed to the Enterprise service manager responsible for the top-level information registry. The service manager verifies that all the selected datasets have the appropriate access rights and services for the SAS domain, and registers the domain in the registry for subsequent use by all consumers of SAS information.”

Source: NextGen Network-Enabled Weather Use Cases, Version 3.1 https://wiki.ucar.edu/download/attachments/17760853/NNEW-UseCases-v3.doc?version=3


Creation of 4-D Cube Experimental SAS Domain

“NCAR, NASA, NOAA, and MIT Lincoln Laboratory are collaborating on a research project to produce a new weather data product. The 4-D Wx Data Cube data access mechanisms are used by the project due to its distributed nature. The majority of inputs that are used to create the new product are those provided by the SAS, with the exception of a higher-quality surface winds product. A new ‘Experimental SAS’ is generated for the duration of the research project, and made available in a registry. Following the completion of the project, the ‘Experimental SAS’ designation is removed from the product’s metadata registry .”



Build-Time Dataset/Service Discovery

“A small flight services company serving the state of Alaska wishes to incorporate a summary of winds information into an existing ‘quick-look’ Web page that they provide to general aviation users. The company browses the NextGen registry to discover what types of wind data are available, and what services exist to access them. An appropriate data access service is located, and members of the company’s software team download the service and data format schema needed to construct a data access client. A thin layer of additional logic is used to convert the raw data to the desired ‘synopsis’ form, and the updated Web application is made available to pilots over the open Internet for flight planning purposes. The data access client software, which conforms to a service standard used to disseminate many types of weather data, provides the company with easy access to numerous other types of weather data in the future.”



Run-Time Dataset/Service Discovery

“A client application contacts a registry containing dataset information for the weather cube, and requests information about ‘air temperature’ datasets available in the SAS domain. Metadata for the single, primary, ‘air temperature’ data set are returned, along with metadata for zero or more backup datasets, each with a priority indication describing the ordering in which the backups should be used. For each dataset, metadata describing one or more data access services, along with the data formats supported by each service, are returned. The client application selects the most appropriate service for the primary dataset, and begins accessing data. ”



Overview








4D Weather Data Cube Domains

1. Single Authoritative Source (SAS)

2. Regulatory2a – Government2b – Commercial

3. Both 1 & 2a4. Intermediate 4a – Government 4b – Commercial

4D Weather Cube Domains

• SAS data (supports ATM decisions) to be available on open & unrestricted conditions• Opportunities for commercial entities to provide weather information still exist• Flexible support for different domains needs to be incorporated into architecture

• SAS data (supports ATM decisions) to be available on open & unrestricted conditions• Opportunities for commercial entities to provide weather information still exist• Flexible support for different domains needs to be incorporated into architecture


Importing the Cube Domain Classification Scheme


Weather Cube Domain TaxonomyebRIM Encoding (some details ommitted)<RegistryObjectList xmlns="urn:oasis:names:tc:ebxml-regrep:xsd:rim:4.0" xmlns:rim="urn:oasis:names:tc:ebxml-regrep:xsd:rim:4.0">

<rim:RegistryObject xsi:type="rim:ClassificationSchemeType" id="urn:ogc:def:ebRIM-ClassificationScheme:FAA:DataCubeDomains" isInternal="true" nodeType="urn:oasis:names:tc:ebxml-regrep:NodeType:UniqueCode"> <rim:Name> <rim:LocalizedString xml:lang="en" value="FAA/NOAA/DOD Data Cube Domain Taxonomy" /> </rim:Name>

<rim:ClassificationNode code="DataCube" parent="urn:ogc:def:ebRIM-ClassificationScheme:FAA:DataCubeDomains" id="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube"> <rim:Name> <rim:LocalizedString xml:lang="en" value="DataCube" /> </rim:Name>

<rim:ClassificationNode code="Restricted" parent="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube:" id="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube:Restricted"> <rim:Name> <rim:LocalizedString xml:lang="en" value="Restricted" /> </rim:Name> <rim:ClassificationNode code="Government"> ... </rim:ClassificationNode> <rim:ClassificationNode code="Commercial"> ... </rim:ClassificationNode> </rim:ClassificationNode>

<rim:ClassificationNode code="Unrestricted" parent="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube" id="urn:ogc:def:ebRIM-ClassificationNode:FAA:DataCube:Unrestricted"> <rim:Name> <rim:LocalizedString xml:lang="en" value="Unrestricted" /> </rim:Name> <rim:ClassificationNode code="SAS"> ... </rim:ClassificationNode> <rim:ClassificationNode code="Regulatory"> ... </rim:ClassificationNode> </rim:ClassificationNode> </rim:ClassificationNode> </rim:RegistryObject></RegistryObjectList


Viewing the Cube Domain Taxonomy via the RegRep UI


Overview








Datasets and Metadata

Metadata

Metadata

DatasetMetadata

DatasetSeries

Metadata

Dataset Series

e.g. Precip Data for most recent one year period

Registry/Repository

ServiceMetadata

Data Access Service(e.g. JMBL, WCS, WFS)

Operates On

Data AccessService

OperatesOn dataset association(s) can be added at RegistrationTime via registry UI if not included in Service Metadata


ISO 19115/19139 Metadata


ISO 19119 Service Metadata Class Diagram (from specification)

• ISO 19119 (Conceptual Model) does not yet have a standardized schema (ala the ISO 19115/19139 pair)• ISO 19119 (Conceptual Model) does not yet have a standardized schema (ala the ISO 19115/19139 pair)


OGC Cat-ebRIM Dataset/Service Registry Information Model Mapping

• Using ebRIM mapping for Service Metadata temporarily until standard 19119 schema is available• Using ebRIM mapping for Service Metadata temporarily until standard 19119 schema is available


Registry/Repository Basic Profile Supports ISO 19119 Service Taxonomy

• JMBL, WFS, WCS services can all be classified under this scheme• Flexibility to declare datasets available via JMBL and/or OGC WFS/WCS

• JMBL, WFS, WCS services can all be classified under this scheme• Flexibility to declare datasets available via JMBL and/or OGC WFS/WCS


INSPIRE Geoportal Dataset/Service Discovery (ISO 19115/ 19139)


INSPIRE Geoportal Metadata EditorISO 19115/19139

• INSPIRE portal does not (apparently) support a service interface• ebXML regrep UI being designed to support the functionality of the

INSPIRE portal, plus the ebXML registry service interface (ebRS)

• INSPIRE portal does not (apparently) support a service interface• ebXML regrep UI being designed to support the functionality of the

INSPIRE portal, plus the ebXML registry service interface (ebRS)


Overview








Dataset Discovery: All Datasets in Cube


Dataset Discovery: All Restricted Datasets in Cube

• Lightning is only dataset series classified under DataCube/Restricted – all other datasets not returned by query

• Lightning is only dataset series classified under DataCube/Restricted – all other datasets not returned by query


Dataset Discovery: All Unrestricted Datasets in Cube

• All datasets with the exception of lightning returned by query• Follow-up queries allow determination of services that operate on each

returned data set (may be zero or more)

• All datasets with the exception of lightning returned by query• Follow-up queries allow determination of services that operate on each

returned data set (may be zero or more)


Overview








Intelligent Discovery of Datasets

• Intelligent discovery– Consults knowledge base/ontology to find alternative

meanings Clustered by: synonyms, parent, children

– Enables discovery of resources without exact keyword match


Intelligent Discovery of Datasets

SPARQLQueryEngine

SPARQLQueryEngine

OntologyEditor

OntologyEditor

OntologyMapper

OntologyMapper

Query EditorQuery Editor

Service Discovery UI/Client


WCSWCS

air_temperatureair_temperature

ebXML Registry/Repository

Architect

Domain Expert

Service Consumer

Service Provider

Service Provider

Service Provider

WCSDataset: air_temperature

JMBLDataset: temperatureAir

WCSDataset: surface_air_temperature

JMBLJMBL

temperatureAirtemperatureAir

WCSWCS

surface_air_temperaturesurface_air_temperature

• Supports discovery of datasets and associated datasets using CF standard names, JMBL parameter names, or potentially other terms

• Flexibility to support JMBL and/or OGC service access to the same dataset

• Supports discovery of datasets and associated datasets using CF standard names, JMBL parameter names, or potentially other terms

• Flexibility to support JMBL and/or OGC service access to the same dataset


Registry/Repository and Infrastructure Support

DomainOntologyDomain

Ontology

MappingOntologyMappingOntology

SPARQLQueryEngine

SPARQLQueryEngine

OntologyEditor

OntologyEditor

OntologyAlignmentOntologyAlignment

Query Editor

Query Editor

OWL-SService Description

Generator

OWL-SService Description

Generator



ServiceAnnotator UI

ServiceAnnotator UI

Service DescriptionService Description

Service AnnotationService Annotation

ebXML Registry/Repository

Architect

Domain Expert

Service Consumer

Service Producer


Discussion

• Is an ebXML registry/repository potentially useful in the Joint METOC context?

• Would it be complementary to JMCAT ?

• Would it largely replace JMCAT functionality?

• What is DOD’s view of ISO 19115/19139 metadata vs. DDMS?

Documents

NextGen Network-Enabled Weather (NNEW) Registry/Repository