Open Data Access Protocol

Webb Pinner

May 2, 2005

CSC 536

Acknowledgements

Peter Cornillan - Graduate School of Oceanography, DODS

James Gallager - OpenDAP.org

Tom Sgouros - OpenDAP.org

John Howland - Deep Submergence Lab, Woods Hole Oceanographic Institute

Outline of Presentation

Background Information

The OpenDAP protocol

Oceanographic Dive Reply System

Summary

Questions?

Background

• DODS was created in 1993 by URI, GSO, MIT, NASA and NOAA for use in oceanographic datasets.

• It's purpose was to "define the structure of a client-server based distribution system for access to oceanographic data over the Internet”

Background (cont.)

• DODS was quickly adapted for use by the atmospheric data community.

• Because DODS no longer dealt with strictly with ocean data the name was changed to the Data Access Protocol

• 2002 the project was made open source and thus OpenDAP was created

The OpenDAP Architecture

• Provides uniform interface to data

• Allows SQL-like data querying

• Read-only access to data

• Platform independent

• Client/server model

• For use over Internet (TCP/IP)

OpenDAP Implementation

• Uses http as transport protocol– Server implemented as CGI extension in

Apache webserver– Requests for data are sent to server in the

form of a URL

• URL sent to server describes exactly what data is requested– Includes which columns and/or data

constraints.

Example URL

• http://dap_server/cgi-bin/nph-dods/data/dataset1– http - protocol– dap_server - servername– cgi-bin/nph-dods - dap server software location– data - director where dataset is located– dataset1 - name of dataset

• Extension determines which service is called

• Constraints may be added to request specific datasets

OpenDAP Implementation(Services)

• Three basic services:– Data Descriptor Service (DDS)– Data Attribute Service (DAS)– Data (DODS)

• Additional services:– HTML form (HTML)– ASCII representation (ASCII)

OpenDAP Implementation(Constraints)

• Constraints specify which fields to display and in what order to display them.– http://…/data.dods?date&time&temp

• Constraints specify search criteria.– http://…/data.dods?temp>100&temp<175

Current Users

• NASA - Global Hydrology Resource Center• NOAA - National Data Buoy Center (NDBC)• NOAA - National Oceanographic Data Center

(NODC)• University of Hawaii/SOEST - Sea Level

Center (SLC)• Woods Hole - Upper Ocean Mooring Data

Archive (UOP)

OpenDAP DEMO

OpenDAP = Middleware/Dist. System?

• Kinda, sorta, not quite

• Uniform interface to data (Middleware)

• User is unaware of OS, storage format or access protocol (Middleware)

• Data distributed across multiple servers (Dist. System)

OpenDAP = Middleware/Dist. System?

• Read-only Access• Each Dataset is independent of other

datasets• No such thing as replication, commit

protocols, concurrency control, etc.

Oceanographic Dive Replay System

Oceanographic Dive Replay System

• Replay a dive complete with textual data and images

• Retrieve data from remote servers

• Provide a graphical means to interact with large ascii datasets

• Provide means to search dataset

WHOI - Virtual Vans

WHOI - Virtual Vans

• Replays a dive complete with textual data and images

• Retrieves data from WHOI server

• Provide a HTML based GUI

• Provide means to search event data

Virtual Vans - Shortcomings

• Unable to search by any other dataset

• Unable to create custom datasets for downloading

• GUI can not dynamically accommodate new sensors.

• All datasets stored at one location

OpenDAP Replay System

• Use DAS, DDS information to dynamically generate interface

• Use URL constraints to search based on any data type

• Use URL constraints to customize downloaded datasets

• Multiple servers may provide datasets

OpenDAP Replay System

Summary

• Introduced the OpenDAP protocol, it’s history, architecture and current uses

• Quick OpenDAP demo• Case Study: Dive Replay System• Looked at existing system and its faults• Discussed a new OpenDAP-based

systsm and its advantages

Questions?