Ocean data dissemination

Jon Blower, University of Reading, UK

Steve Hankin, Bob Keeley, Sylvie Pouliquen, Jeff de la Beaujardière, Edward Vanden Berghe, Margarita Conkright Gregg, Janet Fredericks, Derrick Snowden

… and many others

Context

• Sylvie Pouliquen (Day 1):– Growth of online ocean databases in last 10 years

• Bob Keeley (Day 4):– Good data management practices

• Themes of this talk:– Integrating data– Exchanging data between communities

• Steve Hankin (Day 5):– The way forward

Drivers

•Modern science demands the ability to integrate different data streams

•We need to communicate data outside our community

Theme 1:Integrating data

The Internet Global TelecommunicationsSystem (GTS)

Paper records

How do we disseminate data?

Internet: where we are now (mostly)

We’re pretty good at providing data direct to scientists

But to integrate data we need to be able to share data between machines

Automation,automation

andautomation!

Achieving automation 1:Standardize data formats

• Ocean modelling community has standardized around CF-NetCDF file format

• Obs community currently more heterogeneous

• But CF-NetCDF being adopted in many obs projects:– In situ, satellite, underway, radar

• Biology community has rather different data

• And ASCII file formats still have practical uses

Hankin, Pouliquen CWPs

2: Improve metadata handling• Everyone means different things by the word

“metadata”!

• Aspects include:– Spatial and temporal referencing– Using standard terms for measured quantities / species– Describing measuring instruments– Describing context– Describing quality

• Needs to be machine-understandable

• If you ask for too much metadata, you may get none at all! Gregg, Fredericks, Snowden

CWPs

Example: Ocean BiogeographicInformation System (OBIS)

http://www.iobis.org

100,000 records viewed or downloaded per day18.5 million records633 distinct datasets105,000 species

Simple data format attracted lots of input!

Species richness

Potentialspread ofinvasiveSpecies(lionfish)

Vanden Berghe CWP

3. Direct communicationbetween machines

Web Services Web Services Web Servicese.g. OPeNDAP

ECCO-JPL minus World Ocean Atlas

Example: intercomparison ofdistributed data

4. Catalogues

• Discovery of data is currently a problem

• Solution is to create machine-readable (and human-searchable) catalogues

• These can be aggregated

5. Access control

• Access control can be a barrier to automation

• But often exists for good reasons:– Data privacy (esp. in biological domains)– IT security– Audit trails

• Strong trend towards open access to data

• Single sign-on technologies can help

• But problem will never completely go away

Example: SeaDataNet• Amazon-like discovery and

delivery of data

• Integrates different data sources

• Harmonizes file formats and vocabularies

• Single sign-on

• Links to Ocean Data View

Theme 2:Exchanging data withother communities

Key issues

• Our key technologies are not widely-used in many other communities

• We need technologies that are common across communities

• Geographic Information Systems provide a promising approach– Commonly used by policymakers, decision-makers and

terrestrial science groups

• (Side note: Many other communities do not need to see the full complexity of ocean data)

Linking ocean data with GIS

• Visualize ocean data using Web Map Service standard

• Capability now built into THREDDS data server

www.reading.ac.uk/godiva2

Ocean Data Portal

• Developed through International Oceanographic Data and Information Exchange programme

• Integrates data from National Ocean Data Centres

• Uses and promotes standards

http://www.oceandataportal.org/

Reed CWP

Google Ocean: reaching the public

OpenGIS technologies key points

• Strong potential for integrating ocean data with other geospatial data

• Visual integration of ocean data is popular and powerful– Google Earth and Web-GIS have lowered the barrier to

entry

• But integration of actual data is a key problem!– GIS concepts don’t map neatly to 4D datade la Beaujardière

CWP

Combining all the above:Some large integrating efforts

• WMO Information System (WIS)– Combines GTS and internet-based delivery

• MyOcean (European Marine Core Service)– Integrated catalogues– Data delivery in CF-NetCDF via FTP and OPeNDAP– Visualization based on Web Map Services

• INSPIRE– European Spatial Data Infrastructure– Heavy use of GIS standards

• GEOSS– Global Earth Observation System of Systems

Tools: The missing link weneed to forge

argos = getArgos(‘Atlantic’, 2008, ‘delayed’)

plot(argos)

profiles = getProfiles( … )

Recommendations

• To enable data integration and communicate with other communities:

1. Converge on small number of file formats2. Pursue GIS integration, but be aware of costs and

benefits- recognize the limitations

3. Promote our own proven technologies in GIS community

4. Set up cross-community pilot projects5. Invest in linking data systems with end user tools

• Sylvie Pouliquen (Day 1):– Growth of online ocean databases in last 10 years

• Bob Keeley (Day 4):– Good data management practices

• Themes of this talk:– Integrating data– Exchanging data between communities

• Steve Hankin (Day 5):– The way forward