Metadata/Time-Date Tools
(Toolkit_MTD)
Abe Taaheri , Megan Donovan Spencer , Larry Klein(Space Applications Corp.)
Doug Ilg (RSTX) September 23, 1998
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-2
Overview
• Science Data Production (SDP) TOOLKIT
• Metadata/Time-Date Tools (TOOLKIT_MTD)
• Metadata Tools
• Time/Date Tools
• Example (call sequences)
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-3
SDP TOOLKIT
• SDP TOOLKIT is an interface between EOS instrument data
producing software and the ECS production archival system.
• Besides the tools handling metadata and time/date conversion, it
includes many other tools such as :
- File I/O tools (IO)
- Error/Status Reporting (SMF)
- Process Control Tools (PC)
- Shared Memory Management Tools (MEM)
- Spacecraft Ephemeris and Attitude Data Access Tools (EPH)
- Digital Elevation Model Tools (DEM)
- Ancillary Data Tools (AA)
- Celestial Body Position Tools (CBP)
- Geo-Coordinate Transformation Tools (GCT)
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-4
Science Data Production andArchival Scenario
Science Data Processing
SDP Toolkit
Product Generation
Executable (PGE)
(Generated by ScienceData Server)
Metadata
ConfigurationFile
2
(ASCII File)
InventoryMetadata
(HDF)
Product
Scientific Data Set
Structural Metadata
(HDF-EOS Only)
Inventory Metadata
Archive Metadata
Scientific Data Set
(Generated byProcessing)
PCFFile
Inputs
4
3
1
5
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-5
SDP TOOLKIT (cont.)
• EOSDIS extended data providers will not need the entire TOOLKIT to
handle metadata and data structures (HDF-EOS, HDF)
• This necessitated creating a smaller package to allow these
providers the capability of formatting their products in ECS standard
formats, without requiring the entire SDP toolkit package
• The smaller package could also be used by application developers
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-6
Toolkit_MTD
• Tools relevant to metadata and time/date conversion tools areextracted from Science Data Processing (SDP) Toolkit
• Process Control File, that contains Process Control informationfor I/O, is replaced by a short file mapping logical identifiers tophysical file names
• Errors are reported as in HDF and HDFEOS
• Installation Scripts are modified for quick installation of HDF,HDFEOS, and Toolkit_MTD
• New functionality added to read Metadata from ASCII file
• Tested on SUN and SGI platforms, will be ported to other unixplatforms; HP, DEC, IBM, and also Windows and Mac
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-7
Metadata Tools
• PGS_MET_SetFileId()
Sets logical IDs assigned for the user defined files.
• PGS_MET_GetFileId(char *filename)
Retrieves logical ID assigned for a file.
• PGS_MET_Init(PGSt_PC_Logical fileId, PGSt_MET_all_handles mdHandles)
Initializes MCF file containing metadata.
• PGS_MET_Init_NonMCF(PGSt_PC_Logical fileId, PGSt_MET_all_handles mdHandles)
Initializes an ASCII file containing metadata.
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-8
Metadata Tools (cont.)
• PGS_MET_SetAttr(PGSt_MET_handle mdHandle,char *attrNameStr,void *attrValue)
Assigns values to metadata attributes (after initializing MCF file).
• PGS_MET_GetSetAttr(PGSt_MET_handle mdHandle, char* attrNameStr, void* attrValue)
Retrieves values that are set automatically, or by the user.
• PGS_MET_GetPCAttr(PGSt_PC_Logical fileId, PGSt_integer version, char *hdfAttrName, char* Name, void * parmValue)
Reads metadata parameters held in HDF attributes or in aseparate ASCII file.
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-9
Metadata Tools (cont.)
• PGS_MET_GetConfigData(char* attrName, void* attrValue)
Certain configuration parameters are held in theconfigfile as “10220|REMOTEHOST|sandcrab”.
This tool would retrieve the value "sandcrab" from theconfigfile given the name of the parameter "REMOTEHOST".
• PGS_MET_Write(PGSt_MET_handle mdHandle, char * hdfAttrName, PGSt_integer hdfFileId)
Writes metadata after all the metadata parameters are set inmemory. Tool checks that all the mandatory parameters are set.
• PGS_MET_Remove()Removes ODL representation of all MCF files and someinternal files used by the MET tools.
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-10
Time/Date Tools
• PGS_TD_UTCtoTAI( char asciiUTC[28],
PGSt_double *secTAI93 )
Converts UTC time in CCSDS ASCII Time Code (A or B format)
to Toolkit internal time (real continuous seconds since
12AM UTC 1-1-93).
• PGS_TD_TAItoUTC( PGSt_double secTAI93,
char asciiUTC[28])
Converts Toolkit internal time (real continuous seconds
since 12AM UTC 1-1-93) to UTC time in CCSDS ASCII
Time Code A format.
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-11
Time/Date Tools (cont.)
• PGS_TD_TAItoTAIjd( )
• PGS_TD_TAIjdtoTAI( )
• PGS_TD_TAItoGAST( )
• PGS_TD_UTC_to_Sctime( )
• PGS_TD_SCtime_to_UTC( )
• PGS_TD_ASCIItime_AtoB( )
• PGS_TD_ASCIItime_BtoA( )
• PGS_TD_UTCtoGPS( )
• PGS_TD_GPStoUTC( )
• PGS_TD_UTCtoTDTjed( )
• PGS_TD_UTCtoTDBjed( )
• PGS_TD_TimeInterval( )
• PGS_TD_UTCtoUTCjd( )
• PGS_TD_UTCjdtoUTC( )
• PGS_TD_UTCtoUT1( )
• PGS_TD_UTCtoUT1jd( )
• PGS_TD_LeapSec( )
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-12
Example
/* include files */
#include <PGS_MET.h>
#include <PGS_tk.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <hdf.h>
#include <mfhdf.h>
#include <PGS_SMF.h>
#define INVENTORYMETADATA 1
#define ARCHIVEDMETADATA 2
#define ODL_IN_MEMORY 0
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-13
Example (cont.)
int main()
{
/***** Declarations *****/
PGSt_MET_all_handles mdHandles;
PGSt_SMF_status ret = PGS_S_SUCCESS;
PGSt_integer fileId, sdid, value,rtid;
PGSt_integer i, ival;
PGSt_integer version;
PGSt_double dval;
char *sval[5];
char *start_time = "";
char my_HDF_file[PGSd_MET_FILE_PATH_MAX]="";
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-14
Example (cont.)
/**********************************
Associate logical IDs with physical
filenames
***********************************/
ret = PGS_MET_SetFileId();
/* get values from Config file */
ret = PGS_MET_GetConfigData(“Runtime_ID”,&rtid);
ret = PGS_MET_GetConfigData(“VERSION_NUMBER”,&ival);
ret = PGS_MET_GetConfigData(“STARTTIME”,
&start_time);
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-15
Example (cont.)
/* get value from metadata block of input file */
ret = PGS_MET_GetPCAttr(10265,1,INVENTORYMETADATA,
"EquatorCrossingLongitude",&dval);
/* get value from ASCII metadata file */
ret = PGS_MET_GetPCAttr(5731,1,”coremetadata”,
”TestingAttribute1",&sval);
for(i=0; i<1; i++) printf(“%s”,sval);
printf(“\n”);
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-16
Example (cont.)
/* Initialize MCF file */
fileId = 10250;
ret = PGS_MET_Init(fileId,mdHandles);
if (ret !=PGS_S_SUCCESS)
{
printf("initialization failed\n");
exit(1);
}
/* assign value to attribute in MCF */
ret = PGS_MET_SetAttr(mdHandles[1],
"LocalVersionID",&ival);
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-17
Example (cont.)
ret = PGS_MET_SetAttr(mdHandles[1],
”EquatorCrossingLongitude",&dval);
ret = PGS_MET_SetAttr(mdHandles[1],
”RANGEBEGININGDATETIME",&start_time);
ret = PGS_MET_SetAttr(mdHandles[1],
"ParameterValue.1",57.5)
/* assign value to attribute in MCF
in Archive block */
ret = PGS_MET_SetAttr(mdHandles[2],
"Runtime_ID",&rtid);
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-18
Example (cont.)
/* Read back in value of attribute in memory */
ret = PGS_MET_GetSetAttr(mdHandles[2],
"Runtime_ID",&rtid);
/* Write metadata to HDF and ASCII files */
version =1;
fileId = 5039;
ret = PGS_PC_GetReference(fileId,
&version, my_HDF_file);
if (ret == PGS_S_SUCCESS)
{
sdid = SDstart(my_HDF_file,DFACC_CREATE);
}
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-19
Example (cont.)
/* write INVENTORYMETADATA to HDF file */
ret = PGS_MET_Write(mdHandles[INVENTORYMETADATA],
"coremetadata",sdid);
/* write ARCHIVEDMETADAT to HDF file */
ret = PGS_MET_Write(mdHandles[ARCHIVEDMETADATA],
"archivemetadata",sdid);
/* write to non-HDF file */
fileId = 5804;
ret = PGS_MET_Write(mdHandles[ODL_IN_MEMORY],
NULL,fileId);
EOS
NASA'S MISSION TO PLANET EARTH
EARTH OBSERVING SYSTEM
EA
RT
H P
RO
BE
S
DA
TA
INF
OR
MA
TIO
N S
YS
TE
M
LK-20
Example (cont.)
/* write to default non-HDF file */
ret = PGS_MET_Write(mdHandles[ODL_IN_MEMORY],
NULL, NULL);
SDend(sdid);
/* Remove MCF from memory */
ret = PGS_MET_Remove();
}