Ensuring Credibility of NASA’s Earth Science Data

Session Title: IN11D. Data Curation, Credibility, Preservation Implementation, and Data Rescue to Enable Multi-source Science I

AGU Fall Meeting 2013

M. Maiden, H. K. Ramapriyan, A. Mitchell, S. Berrick, J. Walter, K. MurphyNASA Headquarters, and

NASA Goddard Space Flight CenterDecember 9, 2013

Presentation Overview

NASA’s Earth Science Data Systems Program has been working on transparency, completeness, permanence, and ease of access and use over many years.

The presentation covers:• NASA Earth Science Division Missions Current and Planned• Earth Observing System (EOS) Data and Information System

(EOSDIS)• NASA Data Policy• NASA Earth Science Data Processing and Management • NASA Earth Science Standards Process, Metadata

Architecture Enhancements & ISO 19115 Metadata Support• Data Quality, Fitness-for-Purpose and Provenance of NASA

Data• NASA readiness to work with others on interoperability and

useful Earth science data availability

Landsat-8 (USGS)

NASA Earth Science Operating Missions (2013)

3

•

••

•

•

•

•

•

•

•

•

•

••

Calendar Year 2005 2006 2007 2008 2009 2010 2011 2012+

•

•

•

•

•

Sal Island, Cape Verde

San Jose, Costa Rica

Vera Cruz, Mexico

Honolulu, HI

Anchorage, AK

Fairbanks, AK

Thule, Greenland

Portsmouth, NH

Warner Robins, GA

Houston, TX

Edwards, CA

Ponca City, OK

Mildenhall, England

Punta Arenas, Chile

Antarctica

Thailand

Svalbard, Norway

Iceland

Cedar Key, FL

Barrow, AK

•

Saskatoon Goose Bay

Tobago

Wallops, VA

Papeete, French Polynesia

YellowknifeCold Lake

•

••

Ft. Lauderdale, FL

•

•

•

•

•

••

•

•

•

•u

•

•Kiruna, Sweden

Surprise Valley, NV

Palmdale, CA

Darwin, Australia

•

Kona, HI•

Guam

2005-2012 Airborne Campaigns

SMAP2014

ICESat-22016

SWOT2020

PACE2020

(NOTIONAL)

L-Band SARNET 2021

(NOTIONAL)

CLARREONET 2022

OCO-22015

SAGE-III(on ISS) 2015

GRACE-FO2017

OCO-3(on ISS) 2017

GPM2014

CYGNSSEVM-1, 2017

TEMPOEVI-1, 2019

EVI-22020

EVM-22021

EVI-32022

NASA Earth Science Planned Missions (2014-2023)

5

EOSDIS Mission

The Earth Science Data and Information System (ESDIS) project provides access to data through the development and operation of the science systems of the Earth Observing System (EOS) Data and Information System (EOSDIS).

Data Access Policy – enabling transparency

NASA Earth Science Data Policy• No period of exclusive access• Except where agreed upon with international partners, data and derived

scientific products are available at no cost to all users • Any variation in access will result solely from user capability, equipment, and

connectivity• All NASA-generated standard products are made available (upon request)

along with the source code for algorithm software, coefficients, and ancillary data used to generate these products.

• See (http://science.nasa.gov/earth-science/earth-science-data/) for full text of the policy

Data are made available to all users promptly• After an initial checkout period• Appropriate caveats about data quality are provided in product documentation

NASA ESD fully supports the GEOSS Data Collection of Open Resources for Everyone (GEOSS Data-CORE) which is a distributed pool of documented datasets with full and open, unrestricted access at no more than the cost of reproduction and distribution.

Earth Science Mission Standard Products: Data Processing and Management

NASA’s Earth Science Division (ESD) requires each of the Earth science mission to develop and maintain a Data Management Plan (DMP) to address the management of data from Earth science missions, from the time of their data collection/observation, to their entry into permanent archives.

Instrument Principal Investigators (PIs) are responsible for science algorithms, science data processing software, product quality assessment and validationMost EOS standard products are generated at Science Investigator-led Processing Systems (SIPSs) under supervision of PIsProcessed data are delivered to EOSDIS Data Centers for archival and distributionIn a few cases standard products are generated at EOSDIS Data Centers• Science software is delivered by PIs to Data Centers• Priorities for processing and reprocessing are set by PIs

NASA Earth Science Division Approved Standards – enabling completeness

NASA has developed a process for both identification of appropriate standards and subsequent adoption for use in NASA earth science data systems.

NASA’s ESDIS Standards Office (ESO) provides standards guidance and vision:

• Facilitate the identification of standards being used across ESD missions and projects

• Provide coordination for and facilitate the use of ESDIS endorsed standards and ESDIS developed standards.

• Identify future candidate standards and facilitate the production for review.• Solicit and/or encourage submission of candidate standards, technical notes

and standards-related documents from the ESD community

Metadata standards - http://earthdata.nasa.gov/data/standards-and-references/metadata-standards

Data format standards - http://earthdata.nasa.gov/data/standards-and-references/data-format-standards

Metadata Architecture Enhancements

ESDIS initiated a study of EOSDIS metadata capabilities/processes in the spring of 2012 to evaluate opportunities for improved data access and use, reduce efforts by metadata providers, and improve metadata integrity.

The Common Metadata Repository (CMR) will manage the evolution of NASA Earth Science metadata in a unified and consistent way by providing a central storage and access capability that streamlines current workflows while increasing overall data quality and anticipating future capabilities.

Developing a Unified Metadata Model (UMM) that unifies the EOSDIS metadata assets into one cohesive model.  The UMM represents the unification of the GCMD DIF, EMS and ECHO 10 Metadata formats and provides a framework to map existing metadata into broadly accepted formats such as ISO19115.

NASA ISO 19115 Metadata Support

NASA established the Metadata Evolution of NASA Data Systems (MENDS) working group to achieve consensus on implementation of ISO 19115.

Requirement for future missions: NASA’s Satellite Mission Data Systems Requirements for Science Data Management states the <<mission/instrument name(s)>> science data products metadata shall conform to ISO 19115 Geographic Information - Metadata standards

• NASA ISO 19115 Best Practices – With inputs from the ECHO and DIF metadata model mapping, we have developed the NASA ISO 19115  Best Practices convention with supporting schema and style sheets that focus on including ISO-compliant content in NASA science formats.

NASA’s Soil Moisture Active Passive (SMAP) mission is the first in a new series of missions, to employ metadata based on the ISO 19115 standard.

• Developed a toolkit of ISO reading, writing and parsing tools for initial use by SMAP mission but reusable for other missions and data systems.

NASA’s SAGE (Stratospheric Aerosol and Gas Experiment) III is the next mission slated to support ISO 19115 (also 2014 on ISS).

Preservation of NASA Data and Information Contents – enabling permanence

NASA started in 2010 a dedicated programme “Earth System Data Records Uncertainty Analysis”: estimating, validating, and conveying quantitative measurement differences between sensors, validation measurements and/or models, systematic errors in long-term Earth system data records. • Number of investigations supported - 21

Definition of Provenance/Context Content specification:• Developed and published NASA Earth Science Data

Preservation Content Specification (PSC). PSC is used by NASA Earth science missions for closeout planning and ensuring required contents are preserved

Data Quality & Fitness for Purpose – enabling ease of access and use today

NASA fully embraces the QA4EO key principle Data quality is the key solution to support the increasing number of sensors and end users, and the generation of data used for sensible applications.

ISO 19115 provides mechanisms for including comprehensive data quality in metadata which is critical for ensuring understanding and appropriate use of data.

ISO 19157 extends the Data Quality section of 19115. This technical standard defines a set of measures for the data quality sub elements identified in the ISO Geographic Information series.

The measures will be applicable when evaluating the quality of geophysical datasets and assessing their fitness for purpose. Multiple measures can be defined for each data quality sub element and the choice of which to use will depend on the type of data and its intended purpose.

Data Quality & Fitness for Purpose – enabling ease of access and use tomorrow

• Committee on Earth Observation Satellites (CEOS) Working Group on Information Systems and Services (WGISS) – NASA is providing system engineering support to the development of the CEOS WGISS Integrated Catalog (CWIC), an international community catalog of satellite data that currently provides search and access to over 1800 data products (~50 million granules) from NASA, NOAA, USGS, INPE-Brazil and CCMEO-Canada with more data providers on the way.

• US Group on Earth Observations (USGEO) Data Management Working Group – NASA is supporting the work of USGEO to facilitate earth science data set and data system interoperability across the US federal agencies and implement the data management practices described in the National Strategy for Civil Earth Observations.

• The President’s Climate Action Plan – NASA plans to support the implementation of Climate Data Initiative and the Toolkit for Climate Resilience described in the plan.

The NASA Earth Science Data Program is enabling more and more tools and connections to Earth Observation data from other sources, working as part of:

CWIC uses “Connectors” to provide translation from the CWIC (CSW) API to the agency system.

Data Quality & Fitness for Purpose – enabling ease of access and use tomorrow

NASA’s Worldview (https://earthdata.nasa.gov/labs/worldview/) is but one entry into images of NASA datasets in a GIS-viewable way. Apps have been built by third parties to the Global Imagery Browse Services (GIBS) API.

.

Opportunities exist to partner with researchers, applied scientists, information specialists, and the public to create user-friendly tools and portals.

• As an example, in the 2014-2015 timeframe, NASA will be opening up a Sea Level Rise portal for sea level change research to be accessible and useful to scientists, the general public, policy-makers, businesses, and other stakeholders.

Conclusions

NASA has been developing new tools and ways to improve ease of access and use of the data for the Earth System Science research community. NASA was in the forefront of free and open sharing of data and associated information for Earth observations. NASA utilizes standards for data format, metadata content, and required documentation for any data that are ingested into our distributed Earth Observing System Data and Information System, or EOSDIS. NASA has been improving its data management practices for over twenty years to assure permanence of data utility through reliable preservation of bits, readability, understandability, usability and reproducibility of results.NASA is increasingly focused on using the new technologies available to work with partners across the government, with the education and commercial sectors, and internationally, to crack open the power of Earth science data and information for all.

Related AGU Sessions

Poster• IN13A-1545. Ramapriyan, H. - Data Preservation – Progress in NASA’s Earth

Observing System Data and Information System (EOSDIS)

• IN31C-1509. Pilone, D. et al. - The Common Metadata Repository: A High Performance, High quality Metadata Engine for Next Generation EOSDIS Applications

• IN31C-1512. Boller, R. et al. - See it first: Interactively and Visually Discovering Interesting Satellite Data with NASA Worldview

• PA31A-1818. Behnke, J. et al. - Public Access to NASA's Earth Science Data

Oral• IN31D-02. Murphy, K. - Learning from LANCE: Developing a Web Portal

Infrastructure for NASA Earth Science Data

• IN34A-04. Mitchell, A. et al. - NASA’s Earth Observing Data and Information System – Supporting Interoperability through a Scalable Architecture

• IN43B-08. Walter, J. et al. - Evolving NASA’s Earth Science Data Systems