Support for NPP/NPOESS by The HDF Group

Mike Folk, Elena PourmalThe HDF Group

Annual HDF Briefing to ESDISMarch 31, 2009

March 31 2009 Annual HDF Briefing to ESDIS 1

Goal

March 31 2009 Annual HDF Briefing to ESDIS 2

To develop and adapt HDF5 software and provide HDF5 support for NPP/NPOESS.

NPOESS Priorities and HDF5

• Data calibration/validation• Data accessibility and usability• Tool development• System maintenance• User support

March 31 2009 3Annual HDF Briefing to ESDIS

Areas of immediate need

1. Ensure easy and intuitive access and use by diverse communities

2. Enable producers and consumers to view content, manage metadata, and convert data to other formats.

3. Provide high-quality, rapid-response support of HDF5 for NPOESS users at all levels.

March 31 2009 4Annual HDF Briefing to ESDIS

1. Ensure easy and intuitive data access and use by

diverse communities

March 31 2009 Annual HDF Briefing to ESDIS 5

Region references

March 31 2009 Annual HDF Briefing to ESDIS 6

Root -/

Root - /

Data Array

Product Group Agg

Gran N

ReferenceObject

ReferenceRegion

ReferenceRegion

New routines to facilitate data access

• Access data pointed to by region references Retrieving data pointed to by region references

requires a series of complex function calls. New API would encapsulate these into a few routines.

• Rewrite region references in datasets When a dataset with region references is moved from

one file to another, the references of the datasets are invalid in the new file.

The process of updating reference values is complex and error prone.

New functions can address both of these concerns.

March 31 2009 7Annual HDF Briefing to ESDIS

• Access individual quality flags. NPOESS quality flags are packed into an integer. In HDF5, integer types are “atomic” – any I/O

operation access an entire integer. To access individual NPOESS quality flags in a

convenient and consistent manner, routines are needed that can unpack and return individual quality flags represented by bits packed in an HDF5 integer.

March 31 2009 8Annual HDF Briefing to ESDIS

2. Enable producers and consumers to view content,

manage metadata, and convert data to other formats.

March 31 2009 Annual HDF Briefing to ESDIS 9

Tools enhancements

• There is interest in tools to view and export packed quality flag values, as well as data pointed to by region references.

• Tools of particular interest: h5dump – a command line utility for dumping

information from an HDF5 file. HDFView – a graphical tool for displaying and

editing the contents of HDF5 files.

March 31 2009 10Annual HDF Briefing to ESDIS

Tool enhancements

• H5dump modifications Extract, display, and export a subset of packed

bits corresponding to NPOESS quality flags. Extract, display, and export data pointed to by

region references.• HDFView modifications

Display a rectangular region of data in a dataset corresponding to set of region references.

Display packed bits corresponding to the NPOESS quality flags.

March 31 2009 11Annual HDF Briefing to ESDIS

3. Provide high-quality, rapid-response support of HDF5 for NPOESS users at all levels.

March 31 2009 Annual HDF Briefing to ESDIS 12

Support enhancements

• Assign high priority to helpdesk requests from the NPOESS user community, and ensure a one business-day response to all queries known to be NPOESS-related.

• Support HDF5 on AIX 5.2.

March 31 2009 13Annual HDF Briefing to ESDIS

Status

• Region reference API

• Initial design defined, implemented• Requirements/design draft doc, RFC

Tools• H5dump, HDFView: gathering requirements

• Quality flags API: requirements/design draft doc, RFC Tools: just starting to think about

March 31 2009 14Annual HDF Briefing to ESDIS

Status

• HDF5 support Small but steady stream of requests to helpdesk

from NPP/NPOESS community Request range from simple questions to requests

for new features• AIX 5.2

AIX 5.3 tested daily on borrowed system Hoping to get access on IDPS system at NASA

March 31 2009 15Annual HDF Briefing to ESDIS

Future plans

• Complete current tasks by August 31, 2009.• Working on long-term agreement to continue the

same three-pronged approach1. Technology development to improve usability

for users of NPOESS/NPP data products2. High quality, rapid response support to

NPOESS/NPP HDF5 users at all levels3. System maintenance and support

March 31 2009 16Annual HDF Briefing to ESDIS

Thank You!

This report is based upon work to be supported by a Cooperative Agreement with NASA under NASA contract NNG08HZ14C. Any opinions, findings, and conclusions or

recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the

National Aeronautics and Space Administration.

March 31 2009 Annual HDF Briefing to ESDIS 17

Questions/comments?

March 31 2009 Annual HDF Briefing to ESDIS 18