ODIP: integrating regional marine data infrastructures for global ocean science

Helen Glaves Senior Data Scientist

British Geological Survey

Today’s presentation

Where it all started

Ocean and marine data:

Management on the regional scale

Moving to a global framework: the ODIP approach

Re-use of marine – why it’s matters

First ocean explorers

Phoenicians and Greeks:

First known ocean explorers (2000 – 400BC)

Extensive knowledge of marine science

Early example of the importance of data/ information preservation!

The Vikings: early marine scientists

Explored North Atlantic:• Iceland – 700AD• Greenland - 995AD• North America -

1000AD

Developed detailed knowledge of:

• Currents• Tides• Winds

Age of Discovery: 1400s- 1900s:A time of exploration and adventurous men!

• Christopher Columbus• Ferdinand Magellan• Vasco De Gama• Captain Cook

Captain Cook: first ocean scientist?• 3 voyages (1768 – 1780)

• Produced maps, charts &scientific samples

• Charted Australia & New Zealand

• Explored Hawaii

• First to include a full-time naturalist (Joseph Banks)

Oceanographic data

Wide range of measurements and variables

Derived from broad spectrum of multidisciplinary projects/programmes

Collected by multitude of research institutes, governmental organisations and private companies

Using various sensors to measure physical, chemical, biological, geological and geophysical parameters

Data acquistion Sensors installed on various platforms:

Research vessels

Satellites

Buoys/floats/gliders

Aircraft

Submersibles

Fixed moorings

Fauna

Barriers to re-using data

Use of different

Formats

Standards

Best practice

Co-ordinate systems

Technologies

National and organisational data access policies

9

Regional data infrastructures A number of regional initiatives have

developed marine data management

infrastructures

Promoted and supported by international

organizations - UNESCO‘s

Intergovernmental Oceanographic

Commission (IOC), GEO etc.

BUTImplemented according to regional

requirements and priorities

USA

Europe

Australia

Ocean Data Interoperability Platform

EU-US-Australia collaborative project

Grant Number: 312492

Call: FP7-INFRASTRUCTURES-2012-1-INFSOActivity: INFRA-2012-3.2: International co-operation with the USA

on common e-infrastructure for scientific data

Start date: 1 October 2012

Duration: 36 months

Funded in parallel by European Commission, National Science Foundation (NSF) and Australian Government

How ODIP is achieving its objectives? Developing a collaborative approach and promoting organised

dialogue between partners

Establishing a European - USA - Australia co-ordination platform to support development of interoperability between existing marine data management infrastructures

Creating and publishing inventories of existing standards and policies

Regular joint workshops to develop interoperability solutions and/or agree on common standards

Development of prototype for testing and evaluating potential interoperability solutions

Europe: 10 EU funded partners: 6 countriesNERC-BGS/BODC, MARIS, OGS, IFREMER, HCMR, ENEA, ULG,

CNR, RBINS-MUMM, TNO

ODIP partners

USA: NSF funded partners (supplement to existing R2R project)

San Diego Supercomputer Center (SDSC)

Scripps Institution of Oceanography (SIO)

Woods Hole Oceanographic Institute (WHOI)

Lamont-Doherty Earth Observatory (LDEO)

Florida State University: Center for Ocean-Atmospheric Prediction Studies (FSU)

Australia

Integrated Marine Observing System (IMOS)

International

UNESCO IOC-IODE

Associate partners

Europe

Alfred Wegener Institute for Polar Research (AWI)

MARUM

USA

NOAA US-IOOS, NOAA US-NODC, NOAA NGDC

UNIDATA

Australia

Australian National Data Service (ANDS)

Geoscience Australia (GA)

CSIRO

How ODIP is achieving its objectives? Developing a collaborative approach and promoting organised

dialogue between partners

Establishing a European - USA - Australia co-ordination platform to support development of interoperability between existing marine data management infrastructures

Creating and publishing inventories of existing standards and policies

Regular joint workshops to develop interoperability solutions and/or agree on common standards

Development of prototypes for testing and evaluating potential interoperability solutions

How ODIP is achieving its objectives? Developing a collaborative approach and promoting organised

dialogue between partners

Establishing a European - USA - Australia co-ordination platform to support development of interoperability between existing marine data management infrastructures

Creating and publishing inventories of existing standards and policies

Regular joint workshops to develop interoperability solutions and/or agree on common standards

Development of prototypes for testing and evaluating potential interoperability solutions

NERC Vocabulary Server (NVS)

• lists of standard terms for populating fields in oceanographic metadata

• Used by SeaDataNetfor population of CDI metadata records

• Accessed via RESTful URIs or SOAP

• SPARQL endpoint available http://vocab.nerc.ac.uk/sparql

How ODIP is achieving its objectives?

Developing collaboration platform for organised dialogue between partners

Creating and publishing inventories of existing standards and policies

Regular joint workshops to develop interoperability solutions and/or agree on common standards

Development of prototypes for testing and evaluating potential interoperability solutions

3rd ODIP workshopTownsville, Australia

5 – 8 August 2014

How ODIP is achieving its objectives?

Developing collaboration platform for organised dialogue between partners

Creating and publishing inventories of existing standards and policies

Regular joint workshops to develop interoperability solutions and/or agree on common standards

Development of prototypes for testing and evaluating potential interoperability solutions

ODIP 1: objective

Establishing interoperability between the SeaDataNet, IMOS and NODC data discovery and access services using brokering services

Lead by European partners via SeaDataNet

Initially addressing use of brokers at the metadata level

Progress to data access services (possibly including authentication, authorisation and accounting (AAA) systems)

ODIP 1: the plan make use of the (Euro)GEOSS (GEO-DAB) broker service to

harmonise 3 regional services to a common level

SeaDataNet (Europe)

IMOS (Australia)

NODC (USA)

start at metadata level, but progress to data access, including providing solutions for possible AAA systems

use the broker to facilitate access to data from the regional services by the GEOSS portal and Ocean Data Portal (ODP)

Interoperability established between SeaDataNet metadata discovery services and IODE-ODP and GEOSS portals

Creation of SeaDataNet web service for collections of metadata records

Using GEO-Discovery and Access Broker (DAB)developed for GEOSS

SeaDataNet collections now exposed on GEOSS and ODP portals

ODIP 1: progress

ODIP 1: SeaDataNet web service

Established web service for collections of metadata entries (ISO19115 – 19139 schema)

Collections made by aggregation on Discipline (SDN vocab P08), Data centre (SDN EDMO-code), and geometric type (point / track / surface)

Around 1.5 million CDI granules resulted in approx. in 400CDI collections (includes URL to CDI service for details)

REST web service deliver collections in XML format:

http://seadatanet.maris2.nl/gi-cat-seadatanet/sdn-cdi-aggr-seadatanet_v3.xml

25

Interoperability established between SeaDataNetmetadata discovery services and IODE-ODP and GEOSS portals

Creation of SeaDataNet web service for collections of metadata records

Using GEO-Discovery and Access Broker (DAB)developed for GEOSS

SeaDataNet collections now exposed on GEOSS and ODP portals

ODIP 1: progress

GEOSS Brokerage service Developed and maintained by CNR (Italy)

Middleware for connecting heterogeneous/distributed resources contributing to the GEOSS portal

3 main functionalities:

Discovery of brokered resources

Semantics-enriched discovery

Access of resources

Used to harvest the SeaDataNet collections and convert to Generic Brokerage Reference Schema, adopting SeaDataNet vocabs

27

ODIP 1 – SeaDataNet webservices

SeaDataNet collections available as 2 public web services provided by CNR via the GEO-DAB broker:

OGC Catalogue Service for the Web (CSW) Version 2.0.2Service – HTTP POST method:http://seadatanet.essi-lab.eu/gi-cat/services/cswiso

OAI-PMH interface, at:http://seadatanet.essi-lab.eu/gi-cat/services/oaipmh

Update of SeaDataNet metadata catalogue triggers GEO-DAB harvesting of XML for new records

28

Interoperability established between SeaDataNetmetadata discovery services and IODE-ODP and GEOSS portals

Creation of SeaDataNet web service for collections of metadata records

Using GEO-Discovery and Access Broker (DAB) developed for GEOSS

SeaDataNet collections now exposed on GEOSS and ODP portals

ODIP 1: progress

ODIP 1 – SeaDataNet in GEOSS GEOSS portal harvests SeaDataNet metadata from the

OGC-CSW

Test Client at CNR (ESSI lab) :

http://seadatanet.essi-lab.eu/gi-cat/gi-portal/

30

ODIP 1: SeaDataNet in IODE-ODP

IODE ODP portal harvests SeaDataNet collections from OIA-PMH service

31

ODIP 2: objective ODIP 2: Establishing interoperability between cruise

summary reporting systems in Europe, the USA and

Australia

Lead by Rolling Deck to Repository (R2R) partners

(USA)

Improvement of delivery and exchange of cruise

summary information through the use of common

formats and vocabularies

Use GeoNetWorks for routine harvesting of cruise data

for delivery via the Partnership for Observation of

Global Oceans (POGO) portal

ODIP 2: the plan Publish ISO Cruise Summary Reports at regional

nodes:

Marine National Facility (Australia)

SeaDataNet (Europe)

R2R (USA)

Deploy GeoNetwork catalogues at regional nodes

providing both a GUI (web portal) and API (CSW

service)

Harvest GeoNetwork nodes into POGO global

catalogue

SeaDataNet CSR (Cruise Summary Report) schema adopted by:

R2R consortium partners (USA)

Marine National Facility (Australia)

ISO Cruise Summary Reports published at regional nodes

Deployment of GeoNetwork catalogues at regional nodes underway

ODIP 2: progress

European Directory of Marine Organisations

(EDMO)

SeaDataNet CSR (Cruise Summary Report) schema adopted by:

R2R consortium partners (USA)

Marine National Facility (Australia)

ISO Cruise Summary Reports published at regional nodes

Deployment of GeoNetwork catalogues at regional nodes underway

ODIP 2: progress

SeaDataNet CSR (Cruise Summary Report) schema adopted by:

R2R consortium partners (USA)

Marine National Facility (Australia)

ISO Cruise Summary Reports published at regional nodes

Deployment of GeoNetwork catalogues at regional nodes underway

ODIP 2: progress

EU GeoNetwork deploymentEurope: http://www.ifremer.fr/geonetwork-sdn/

R2R GeoNetwork CSR instance

http://catalog.rvdata.us/geonetwork

GeoNetwork deployed:

http://www.cmar.csiro.au/geonetwork

Published a test set of CSR for R/V Southern Surveyor

Marine National Facility GeoNetworkinstance

ODIP 3: objective

Establishment of a prototype for a Sensor Observation Service (SOS) for selected sensors installed on vessels and in real-time monitoring systems using sensor web enablement (SWE)

Lead by AODN (Australia)

regional initiatives progress towards the adoption of SWE allowing direct standardised access to the data from operational sensor systems

ODIP 3: the plan

establish a collaboration tool (Github)

compile inventory of SOS services and their endpoints

compile inventory of instrument SensorML records & O&M structures

compile inventory of vocabulary and registry services

Working groups to:

assess SOS performance

propose templates for SensorML/StarFL and O&M profiles

examine vocabulary services and potential mappings

Set-up a test bed

ODIP 3: progress GitHub collaborative tool set-up

Test bed established

2 public SOS servers running V4.0 and V3.6 of 52oNorth SOS server

Fully open allowing full set of SOS requests including registration of sensors and adding data

Working groups established:

• assess SOS performance

• propose templates for SensorML/StarFl and O&M profiles

• examine vocab services and potential mappings

ODIP 3: progress GitHub collaborative tool set-up

Test bed established

2 public SOS servers running V4.0 and V3.6 of 52oNorth SOS server

Fully open allowing full set of SOS requests including registration of sensors and adding data

Working groups established:

• assess SOS performance

• propose templates for SensorML/StarFl and O&M profiles

• examine vocab services and potential mappings

ODIP3: SOS services: 52o North v4 SOS V4.0

Base Address: http://115.146.93.169:8080/IMOS-SOS/

XML Address: http://115.146.93.169:8080/IMOS-SOS/sos/pox/

Two Features of Interest

IMOS/DAVIES/SF1 = Sensor Float 1 at Davies Reef

IMOS/HERON/RP8 = Relay Pole 8 at Heron Island

Two parameters

Water temperature (Deg. C.)

Depth (m)

SOS v3.6

Base Address: http://130.220.209.177:8080/IMOS-SOS-36/

XML Address: http://130.220.209.177:8080/IMOS-SOS-36/sos/

To be set up with same Features of Interest and parameters

Supports SOS v1 and v2

ODIP3: SOS services: 52o North v3.6

ODIP 3: progress GitHub collaborative tool set-up

Test bed established

2 public SOS servers running V4.0 and V3.6 of 52oNorth SOS server

Fully open allowing full set of SOS requests including registration of sensors and adding data

Working groups established:

• assess SOS performance

• propose templates for SensorML/StarFl and O&M profiles

• examine vocab services and potential mappings

Underpinning development of a robust common global framework for marine data management by:

Establishing interoperability between existing regional data management infrastructures

Creating an approach that can be adopted by agencies and organisations in other geographical area

Supporting and promoting international collaboration across the marine data management community

Facilitating re-use of marine data

ODIP: what is it achieving?

Data is fundamental to a range of activities in the marine domain:

Research

Monitoring

Forecasting

Management

Many of these activities are now highly multidisciplinary/ ecosystem level requiring easy access to large volumes of data

Re-use: why is it important?

Marine data is precious

expensive to collect

Inherently unique

Sparse spatial and temporal coverage

Needed to provide answers to local and global issues

Capture once – use it many times

Find out more

Project website

Join the ODIP community

Contact us

Social media

International conferences

Other related initiatives

BCube (NSF)

Ocean Data Portal (ODP)

Research Data Alliance

Belmont Forum

www.odip.eu

Thank you to: ODIP partners

Dick Schaap (SeaDataNet)

Roger Proctor/ Scott Bainbridge (IMOS)

Bob Arko (R2R)

Sergey Belov (IODE)

Chris Oosthuizen (IMOS)

Genny Anderson (

Thank you!