OPeNDAP infrastructure in European Operational Oceanography T Loubrieu (IFREMER) T Jolibois (CLS)

opendap dev - meeting, Boulder, Feb 2007

OPeNDAP infrastructure in European Operational Oceanography

T Loubrieu (IFREMER)

T Jolibois (CLS)


The Mersea project

• The objective of MERSEA is to provide an integrated service of global and regional ocean monitoring and forecasting to intermediate users and policy makers in support of safe and efficient offshore activities, environmental management, security, and sustainable use of marine resources.

• From a technical point of view it aims at setting up :– From :

• 5 ocean forecasting centres

• 1 in-situ data centre

• 4 remote sensing data centres

– Into one european ocean monitoring and forecasting system.


The main overall services

• Discovery– Catalogue of products (ISO19115)– Dataset inventory based on THREDDS

• Visualize– Images browser (maps, sections, time series…)– Centralized image generator based on OPeNDAP resources

• Download– Restricted or free OPeNDAP servers– Single Sign On system.

• Monitor– Check timeliness of datasets delivery thanks to THREDDS

inventory


Global architecture

Mersea Portal

SSO

(catalogue/inventory)

Viewing V1(quicklooks)

Viewing V2(Google Earth)

Download

OpenDap (http/https)

ftpWMS (http/https)

User Management

Discovery

Autres http(s)

(DATA PROVIDERS)

(SERVICES USERS)

MI

MT

EP

s


Data access infrastructure

What ? Who ? Country ? Opendap server Vers

In-situ

SST

IFREMER France TDS 3.X

2.X

In-situ IFREMER France Dap4cor 1.0

Global forecast

Altimetry

CLS France TDS (AS) 2.X

Sea Ice MET.NO Norway server3 3

NE Atl forecast NERC United Kingdom TDS (AS) 2.X

Mediterranean forecast

INGV Italy TDS 3.X

Arctic forecast NERSC Norway TDS 3.X

SST CNR Italy TDS 3.X

Baltic forecast DMI Danemark TDS 3.X


Dataset Inventory : requirements

• The Mersea Dataset Inventory lists the references to the actual datasets the project disseminates through http and opendap services.

• The inventory access is free.

• It is useful for : – Downloading the datasets

– Monitoring the availability of the datasets.


Dataset Inventory : TDS usage

• The TDS servers in version 3.X are used to make inventory available (THREDDS catalog V1).

• Hierarchical organisation : one top THREDDS inventory federates remote THREDDS inventories with ‘catalogRef’ tags.

• So the distributed catalogs are directly maintained and updated by the data providers, but the user sees only one catalogue.


Dataset Inventory : Metadata

• The set of mandatory metadata has been defined at the project level so that the catalogs are homogeneous.

• Then a searching or browsing tool work the same way for all the distributed catalogs.

• The mandatory metadata are :– Documentation (link to the ISO19115 Mersea product

catalog web page).

– GeospatialCoverage

– TimeCoverage

– Variables (using CF convention).


Dataset Inventory : browsing interface

• The development of a browsing web interface is planned.

• Technically : it is planned to provide html pages with signed javascript code (AJAX) embedded for accessing the distributed THREDDS XML catalogs.


Dataset inventory : architecture


Integrated Viewing Service

• Centralized daily images generator (using FERRET) and accessing datasets through OPeNDAP servers of the project (TDS, server3, dap4cor).

• The images can be browsed through a php web interface :– Raster maps

– Scattered maps (in-situ measurement positions + values)

– Time series

– Vertical section are processed.


Integrated viewing service


Integrated download service : functions

• OPeNDAP servers (mainly TDS) are used for download.

• A Single Sign On (SSO) is needed because :– Some datasets download accesses are restricted.

– In order to make users feel as if they were requesting one only European data centre : they must log in only once (even if they are requesting distributed services).

• Central Authentification Service (CAS) from Yale University has been chosen for SSO.


Integrated download service : SSO

• CAS provides – an authentification server (login/password form or POST

request).

– A java tomcat filter (CASClient) added to TDS configuration so that any data provider can trust the users authentification (login).

• The CASClient filter is going to be patched so that a data provider can easily configure the authorizations/restrictions of access depending on the login of the users and the currently requested URL.


Integrated download service :demonstration client

• A lightweight download demonstration tool dedicated to MERSEA was required.

• Its specifications are :– Graphically builds opendap request.

– Saves opendap responses as netCDf files.

– Lightweight and highly portable desktop software.

– Manage CAS authentification (need for cookies management and automatic URL redirection -http status 302- management, process the login form).

– To Be Properly Define : Integrates a THREDDS metadata searching tool.


Download Scenario (1/5)










Seadatanet (G Maudire, IFREMER)• Seadatanet aims at setting up a pan-european infrastructure for

ocean observation data management.• In the version 2 of the system (next year), Seadatanet will use

distributed OPeNDAP servers (dapper, TDS…) in order to :– Provide overall and cross checked observation inventory (with double

identification, quality assesment information).– Set up an integrated download service.

Naiad (JF Piollé, IFREMER/CERSAT)• Processing server :

• for satellite swath data retrieval (time and geographical indexation of datasets)

• Data-mining. • Uses DAP data model for internal data exchanges.• DAP Request syntax not yet implemented


What we would need ?

• Security management with SSO (or identity federation) over distributed OPeNDAP servers.

• Easy configuration of the presentation layer for THREDDS, so that the output graphical charter can be adapted to the project it is used for.

• Simple semantic thing (upper layer) to really interoperate servers AIS ?

Documents

OPeNDAP infrastructure in European Operational Oceanography T Loubrieu (IFREMER) T Jolibois (CLS)