Upload
abraham-park
View
223
Download
1
Tags:
Embed Size (px)
Citation preview
IPY Data management
A brief summaryØystein Godøy
IPY Organisation
ICSU/WMOJoint Committee
Data Policy & Management
Subcommittee
ObservationsSubcommittee
Education, Outreach & Communication Subcommittee
ProgrammeOffice
Space TaskGroup
Twelfth WMO Congress, Resolution 40 (1995)
Thirteenth WMO Congress, Resolution 25 (1999)
ICSU General Assembly Resolution (1996)
ICSU Assessment on Scientific Data and Information (2004)
Article III-1c from the Antarctic Treaty (1959)
the Intergovernmental Oceanographic Commission Data Exchange Policy (2003)
International consensus on free and open access
The primary purpose should be to serve the objectives of IPYInternational exchange (>60 countries)Interdisciplinary science (>50,000
participants representing physical, life, and social scientists)
Building a legacyIncrease the value of observations
Overarching purpose
Data used by IPY
Data generated by IPY
Special Cases:• Human subjects• Intellectual property of LTK• Where data release may cause harm
“…the IPY Joint Committee requires that IPY data, including operational data delivered in real time, are made available fully, freely and on the shortest feasible timescale.”
http://www.ipy.org/Subcommittees/final_ipy_data_policy.pdf
IPY Data Policy—Defining IPY Data
Implications
Publish and share your dataIncreased data valuePromote interdisciplinary useHave someone measured something you
need?
Preserve and archive your dataGenerate a legacyThe basis for the next IPY
What has been done?
The Data committee has defined a data policy
Workshop on data management http://classic.ipy.org/international/joint-committee/data-management.h
tm
IPY metadata profile http://ipydis.org/metadata.html
Data coordinators in US, Canada, UK and Norway
Data released for IPY use by ECMWF, ESA and others
Some data management projects have been funded
But not enough...IPY Layer in Google Earth available at
http://ipy.org
IPY data policy - metadata
“All IPY data must be accompanied by a full set of metadata that completely document and describe the data.”
“Regardless of any data access restrictions or delays in delivery of data itself, all IPY projects must promptly provide basic descriptive metadata in an internationally recognized, standard format to an appropriate catalog or registry.”ISO19115/ISO19139
“All data registries and repositories collecting data and metadata from IPY projects are required to collect and share sufficient information to adhere to the IPY Metadata Profile”
Basic who, what, where, when in either FGDC, DIF, THREDDS (ISO coming, but could use some help), plus some information on metadata provenance.
Controlled vocabulary from GCMD for some fields.
The “bare minimum of information necessary to allow simple discovery across disciplines and to ensure we can track the heritage of the metadata in a broadly distributed data management environment.”
Details are available at http://ipydis.org/
IPY Metadata Profile (and crosswalk)
• Entry ID (controlled)• Data set title• Data set progress• Data set summary• Data set citation
information including Online Resource
• Parameters• Locations• ISO topic categories• Temporal coverage• Spatial coverage
• Data center contact information
• Access restrictions• Use constraints• Data Set Language• Metadata contact
information• Metadata authority• Metadata version• Last revision• IPY flag• IPY Project ID
IPY Metadata profile
Formats:• Negotiate common formats and conventions
• ASCII is useful but not really a precise format• avoid proprietary formats • some suggestions: netCDF is popular for some, OGC (WMS/WCS/WFS)
compatibility is nice• Archives and users may have different needs
Access:• Integrate with many systems to allow increased user discovery (register
with the IPYDIS)• Use open source software when possible, use open standards everywhere.
Preservation• Open Archive Information System Reference Model• Attribute and provide info for attribution readily through all gateways
The Data
What is out there?
http://www.ipy-ice-portal.org
ELOKA The Exchange for Local
Observations and Knowledge of the Arctic
works to provide data management and user support to facilitate the collection,
preservation, exchange, and use of local observations and knowledge of the
Arctic.
http://nsidc.org/eloka PI: Shari Gearheard
18 May 2006
http://arcticportal.org
• Assist on compliance with standards, identification of archives, development of the union catalogue, and other data management requirements for IPY.
• Visibly track the data flow for IPY.
• In collaboration with the IPO, develop a data registry that will continue throughout the IPY.
• Survey the planned projects and the data they intend to collect and identify existing archives, portals, experts, and significant gaps in the IPY data infrastructure.
Mark Parsons—Overall, US
Øystein Godøy—Operational Data, Norway
Canadian Coordinator—Overall, Canada
National coordinators in Netherlands, China, UK
Data coordinators
International Operational Data Coordinator (1)
Fist version based upon DAMOCLES systemPostGRES metadatabaseOpeNDAP ⇒ THREDDS (OpeNDAP/WCS
support) or early 2008Adding WMS support for selected
fields/datasetsExpected late 2007
Second version based upon SIMDAT and first versionExpected 2008
International Operational Data Coordinator (2)
Final version should be WIS DCPCExpected 2009
Have focused on ECMWF data so farDirect access at ECMWF disks if using WIS
DCPC software
Interface for metadata submissionAs well as notifications on interesting
datasets
Import/export of metadataMethod and format open issue
International Operational Data Coordinator (3)
How can this service help promoting remote sensing data during IPY?Access to metadata?
Provide metadata information on available products (??)
Users get forwarded to data catalogues at the space centres(??)
OGC access to selected datasets(??)
National recommendations (1)
NorwayNational IPY-projects should relate to the
relevant standards forData collectionProcessingArchivalDescriptionTransferMetadata (ISO19115, ISO19139)OpenGIS
National recommendations (2)
NorwayData and metadata shall be freely
available with minimal time delay
Metadata shall be transmitted to relevant national and international data centres
Data shall be transmitted to relevant national and international data centres for long term archival
Delivery of data and metadata is the responsibility of the project coordinator
National recommendations (3)
NorwayA minimum of standardised file formats
should be used
Existing national and international infrastructure shall be utilised for long term archival
Data used in publications shall be correctly cited
The use of licences will be evaluated if this can improve data access
National requirementsNorway
IPY projects with NFR-contractsshall submit a data management plan within
3 months of contract signingDescribing who is responsible for overall data
managementWho is responsible for datasetsWhere datasets will be stored and finally archived
shall submit yearly updates of the data management planWhat have been collectedWhere is it stored
National IPY-data serviceNorway
DokIPYFunded by NFRShall connect existing metadata- and
databases in NorwayExtract and exchange metadataUtilise existing infrastructure and national
mandates atInstitute of Marine ResearchNorwegian Polar InstituteNorwegian Meteorological Institute
Robust/simple exchange of metadata
METNO version
Relies on the same software as DAMOCLES and International Operational Data Coordination
New feature – metadata exchangeExamining international
requirements/specifications, not much found yet
Funding situation require simple and robust solutions
Special issue
Remote Sensing of EnvironmentSpecial issue on Remote Sensing of the
CryosphereRemote sensing of Environment, Vol. 111,
Issues 2-3, 30 November 2007.Available online
How can we interface?