Page 1

Dimension Scales Dimension Scales in in

HDF-EOS2 & HDF-EOS5HDF-EOS2 & HDF-EOS5

Abe Taaheri, Raytheon IISHDF & HDF-EOS Workshop XIV

Champaign, ILSep. 29, 2010

Page 2

• What is a Dimension Scale ?

• Dimension Scales & Metadata

• Dimension Scales APIs– he2

– he5

• Code example – Writing (he2, he5)

– Reading (he2, he5)

• Sample he2 and he5 files with Dimension Scales

Page 3

It is a sequence of numbers placed along a dimension to demarcate intervals along it

What is a Dimension Scale?What is a Dimension Scale?

• HDF4HDF4- It is an array with size and name similar to

its assigned dimension

- Stored using a structure similar to the SDS array

- One scale is assigned per dimension

Page 4

* What is a Dimension Scale?

• HDF5HDF5

– A HDF5 dataset.

– With additional metadata that identifies the dataset as a Dimension Scale

– Typically Dimension Scales are logically associated with the dimensions of HDF5 Datasets

– The meaning of the association is left to applications.

* Pedro Vicente talk, HDF/ HDF-EOS Workshop IX* Pedro Vicente talk, HDF/ HDF-EOS Workshop IX

Page 5

Example: 3D dataset

Dataset: 3D Array with 5 x 7 x 10

dimensions

3 Dimension Scales Datasets

5

7

10

Page 6

• A dimension scale is not required to be a 1-D array, or to have a specific datatype

• A dataset dimension can have more than 1 associated dimension scale

• A Dimension Scale can be shared by two or more Dataset dimensions

More on Dimension Scale in HDF5

Page 7

Example: 3D dataset in HDF5

Dataset: 3D Array with 5 x 7 x 10

dimensions

Several Dimension Scales Datasets

Page 8

HDF5 Dimension Scale Metadata

• When the Dimension Scale is associated with a dimension of a Dataset, the association is represented by attributes of the two datasets.

• The following dataset attributes are used to describe dimension scale datasets:

– Attribute named “CLASS” with the value “DIMENSION_SCALE”

– Optional attribute named “NAME”

– Attribute references to any associated Dataset

Page 9

• HDF4

– Since Version 4.0 ?

– SDsetdimscale()

– SDgetdimscale()

– SDsetdimstrs(): label, unit, format

– SDgetdimstrs()

• HDF5

– Since Version 1.8

– H5DSset_scale()

– H5DSattach_scale(), H5DSdetach_scale()

– H5DSset_label(), H5DSget_label()

– A few more APIs

Page 10

HDF Augmentation ToolHDF Augmentation Tool

Augmentation Tool adds HDF5 Dimension Scales inside the file so that netCDF-4 APIs can access the data

Page 11

• HDF-EOS

– Added a few routines to HDF-EOS2 to create dimension scales like HDF4

Version 2.17

– Added a few routines to HDF-EOS5 to create dimension scales like those added by the HDF Augmentation Tool

Version 1.13

Page 12

HDF-EOS2 APIsHDF-EOS2 APIs

Page 13

• Name: SWsetdimscale

• Signature:

– intn SWsetdimscale( int32 swathID,

char *fieldname, char *dimname, int32 dimsize,

int32 numbertype, VOIDP data)

• Purpose:

– Sets dimension scale for a given dimension

Page 14

• Name: SWgetdimscale

• Signature:

– intn SWgetdimscale( int32 swathID,

char *fieldname, char *dimname,

int32 *dimsize, int32 *numbertype,

VOIDP data)

• Purpose:

– Gets dimension scale for a given dimension

Page 15

• Name: SWsetdimstrs

• Signature:

– intn SWsetdimstrs( int32 swathID,

char *fieldname, char *dimname, char *label,

char *unit, char *format)

• Purpose:

– Sets the label, unit, and format strings for a given dimension

Page 16

• Name: SWgetdimstrs

• Signature:

– intn SWgetdimstrs( int32 swathID, char *fieldname, char *dimname, char *label,

char *unit, char *format, intn len)

• Purpose:

– Gets the label, unit, and format strings for a given dimension

Page 17

• Similar APIs for Dimension Scales in

– Grid Fields

Page 18

HDF-EOS5 APIsHDF-EOS5 APIs

Page 19

• Name: HE5_SWsetdimscale

• Signature:

– herr_t HE5_SWsetdimscale( hid_t swathID, char *fieldname, char *dimname,

const hsize_t dimsize, hid_t numbertype,

void * data)

• Purpose:

– Sets dimension scale for a field dimension within the swath

Page 20

• Name: HE5_SWgetdimscale

• Signature: – long HE5_SWgetdimscale( hid_t swathID,

char *fieldname, char *dimname, hsize_t *dimsize, hid_t *numbertype, void * data)

• Purpose: – Gets dimension scale for a field dimension

within the swath

Page 21

• Name: HE5_SWwritedscaleattr

• Signature:

– herr_t HE5_SWwritedscaleattr( hid_t swathID, const char *dimname, const char *attrname, hid_t ntype, hsize_t count[], void *datbuf)

• Purpose:

– Writes/Updates a dimension scale attribute in a specific swath

Page 22

• Name: HE5_SWreaddscaleattr

• Signature:

– herr_t HE5_SWreaddscaleattr( hid_t swathID, const char *dimname, const char *attrname,

void *datbuf)

• Purpose:

– Reads a dimension scale attribute from a specific dimension

Page 23

• Name: HE5_SWinqdscaleattrs

• Signature:

– Long HE5_SWinqdscaleattrs( hid_t swathID, const char *dimname, char *attrnames,

long *strbufsize)

• Purpose:

– Retrieve information about the attributes defined

Page 24

• Name: HE5_SWdscaleattrinfo

• Signature:

– herr_t HE5_SWdscaleattrinfo( hid_t swathID, const char *dimname, const char *attrname,

hid_t *ntype, hsize_t *count)

count : Number of attribute elements

• Purpose:

– Returns information about attribute(s) in a specific dimension scale

Page 25

• Similar APIs for Dimension Scales in

– Grid

– Zonal Average

Page 26

Code ExamplesCode Examples

Page 27

Writing Code Example (HDF-EOS2)

#include "hdf.h"

#include "HdfEosDef.h"

int main()

{

intn status, i;

int32 swfid, SWid;

int32 nbands, nGeoTrack;

int32 bands[15]={3,6,9,12,15,18,23,26,29,32,33,34,36,37,39};

char label[16];

char unit[16];

char format[16];

float dataGeoTrack[20];

swfid = SWopen ( "SwathFile.hdf", DFACC_RDWR );

SWid = SWattach ( swfid, "Swath1“ );

Page 28

nGeoTrack = 20;for (i=0; i<20; i++) { dataGeoTrack[i] = (float) I; }status = SWsetdimscale ( SWid, "Temperature", "GeoTrack",

nGeoTrack, DFNT_FLOAT32, dataGeoTrack );

nbands = 15;status = SWsetdimscale ( SWid, "Spectra", "Bands",

nbands, DFNT_INT32, bands );

strcpy (label, "Bands");strcpy (unit, "none");strcpy (format, "I2");status = SWsetdimstrs ( SWid, "Spectra", "Bands", label, unit, format );

SWdetach ( SWid );SWclose ( swfid );return 0;

}

Page 29

Writing Code Example (HDF-EOS5)

#include <HE5_HdfEosDef.h>

int main()

{

herr_t status = FAIL;

hid_t swfid = FAIL;

hid_t SWid1 = FAIL;

int data[15]={1,2,3,6,9,12,15,18,23,26,29,32, 33,34,35};

char label[16], unit[16], format[16];

int nbands;

hsize_t count[3];

swfid = HE5_SWopen ( "Swath.h5", H5F_ACC_RDWR );

SWid1 = HE5_SWattach ( swfid, "Swath1“ );

Page 30

nbands = 15;

status = HE5_SWsetdimscale ( SWid1, "Spectra", "Bands", nbands, H5T_NATIVE_INT, (void *)data );

strcpy(label, "Bands Dim");

strcpy(unit, "None");

strcpy(format, "I2");

count[0]= 12;

status = HE5_SWwritedscaleattr ( SWid1, "Bands", "label", H5T_NATIVE_CHAR, count, label );

count[0]= 6;

status = HE5_SWwritedscaleattr ( SWid1, "Bands", "unit", H5T_NATIVE_CHAR, count, unit );

count[0]= 4;

status = HE5_SWwritedscaleattr ( SWid1, "Bands", "format", H5T_NATIVE_CHAR, count, format );

Page 31

int datbuf_i1[1] = {-999};

count[0]= 1;

status = HE5_SWwritedscaleattr ( SWid1, "Bands", "MissingValue", H5T_NATIVE_INT, count, datbuf_i1 );

int datbuf_i2[3] = {-999, 0, 999};

count[0]= 3;

status = HE5_SWwritedscaleattr(SWid1, "Bands", "IntValues", H5T_NATIVE_INT, count, datbuf_i2 );

HE5_SWdetach ( SWid1 );

HE5_SWclose ( swfid );

return 0;

}

Page 32

Reading Code Example (HDF-EOS2)

#include "hdf.h"#include "HdfEosDef.h"Int main(){ intn status, i; int32 swfid, SWid; int32 *datbuf_int32; int32 nbands, data_type, dimsize, dimsize1; intn len, buffsize; char label[16], unit[16], format[16];

swfid = SWopen ( "SwathFile.hdf", DFACC_RDWR ); SWid = SWattach ( swfid, "Swath1“ );

/* get Bands */dimsize = SWdiminfo ( SWid, "Bands“ );buffsize = SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize1,

&data_type, NULL );

Page 33

datbuf_int32 = (int32 *) malloc(buffsize);buffsize = SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize1,

&data_type, (void *)datbuf_int32 );for (i=0; i<dimsize; i++) { printf("i = %d datbuf_int32 = %d\n", i, datbuf_int32[i]); }

/* get str attributes of Bands*/len = 15;status = SWgetdimstrs ( SWid, "Spectra", "Bands",

label, unit, format, len );printf(" \t\tlabel = %s unit = %s format = %s \n\n", label, unit, format);

free(datbuf_int32);datbuf_int32 = NULL;

SWdetach ( SWid ); SWclose ( swfid ); return 0;}

Page 34

Reading Code Example (HDF-EOS5)#include <HE5_HdfEosDef.h>int main(){ herr_t status = FAIL; hid_t swfid = FAIL, SWid = FAIL, numtype; int i,j, bufsize; int *databuff; long nattr, strbufsize; char *attrlist, *attrname = (char *)NULL; size_t fldnmlen[HE5_HDFE_NAMBUFSIZE]; char *fldnm[HE5_HDFE_NAMBUFSIZE]; hid_t *ntype; hsize_t count = 0; void *attr; int *attr_int; float *attr_flt; char *attr_char; hsize_t dimsize; swfid = HE5_SWopen ( "Swath.h5", H5F_ACC_RDWR ); SWid = HE5_SWattach ( swfid, "Swath1“ );

Page 35

bufsize = HE5_SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize, &numtype, NULL );

databuff = (int *) malloc(bufsize );bufsize = HE5_SWgetdimscale ( SWid, "Spectra", "Bands", &dimsize,

&numtype, (void *)databuff );

for( i = 0; i < dimsize; i++){ printf ("DscaleVal[%d] = %d\n",i,databuff[i] ); }

nattr = HE5_SWinqdscaleattrs ( SWid, "Bands", NULL, &strbufsize );attrlist = (char *) calloc(strbufsize + 1, sizeof(char));

nattr = HE5_SWinqdscaleattrs ( SWid, "Bands", attrlist, &strbufsize );nattr = HE5_EHparsestr(attrlist, ',', fldnm, fldnmlen);

Page 36

for( i = 0; i < nattr; i++) { attrname = (char *) calloc(fldnmlen[i] + 1, sizeof(char) ); memmove ( attrname,fldnm[i],fldnmlen[i] ); ntype = (hid_t *) calloc(1, sizeof(hid_t)); if ( strcmp ( attrname, "REFERENCE_LIST") == 0 ) continue; status = HE5_SWdscaleattrinfo ( SWid,"Bands", attrname,

ntype, &count ); if( (int)*ntype == 0) {

attr_int = (int *) malloc(count*sizeof(int));attr = (void *) attr_int; }

if( (int)*ntype == 10) {attr_flt = (float *) malloc(count*sizeof(float));attr = (void *) attr_flt; }

if( (int)*ntype == 57) {attr_char = (char *) malloc((count+1)*sizeof(char));attr = (void *) attr_char; }

status = HE5_SWreaddscaleattr ( SWid,"Bands", attrname, attr ); printf("\tAttribute Name: %s \n",attrname); printf("\tNumber of attr elem: %lu \n", (unsigned long)count); printf("\tData type of attribute: %d \n", (int)*ntype);

Page 37

if( (int)*ntype == 0) {attr_int = (int *) attr;for (j = 0; j < count; j++) { printf("\tAttrvalue[%d]: %d \n", j,attr_int[j]); }}

if( (int)*ntype == 10) {attr_flt = (float *) attr;for (j = 0; j < count; j++) { printf("\tAttrvalue[%d]: %f \n", j,attr_flt[j]); } }

if( (int)*ntype == 57) {attr_char = (char *) attr;attr_char[count]='\0';printf("\tAttrvalue: %s \n", attr_char);free ( attr_char );}

}

HE5_SWdetach ( SWid ); HE5_SWclose ( swfid ); return 0;}

Page 38

HDFView 2.4:

uses plug-in with hdf-eos5-1.11, Hdfeos2.15, HDF4.2r3, HDF5-1.6.7

Test File content (Test File content (he2he2))

Page 39

HDFView 2.4: uses plug-in with

hdf-eos5-1.11, Hdfeos2.15, HDF4.2r3, HDF5-1.6.7

Test File content (Test File content (he5he5))

Page 40

HDFView 2.6: Without plug-in

HDF4.2r4, HDF5-1.8.3

Test File content (Test File content (he5he5))

Page 41

Test File content (Test File content (he5he5))HDFView 2.6: uses plug-in with

hdf-eos5-1.12, Hdfeos2.16, HDF4.2r4, HDF5-1.8.3

Page 42

File content (File content (HIRDLS he5HIRDLS he5))

HIRDLS he5 Modified with

HDF Augmentation

Tool

Page 43

Suggestions?

– Do we need new APIs ?Do we need new APIs ?

– Improvements for existing APIs ?Improvements for existing APIs ?

– What else? What else?

Page 44

Thank You!