CCSDS Meeting data Archive Ingest - June 2006 CNES 1 CCSDS - MOIMS Area Data Archive Ingest WG CNES Report Rome meeting – June 2006 Claude Huc

CCSDS Meeting data Archive Ingest - June 2006 CNES 1

CCSDS - MOIMS AreaCCSDS - MOIMS Area

Data Archive Ingest WGData Archive Ingest WG

CNES Report CNES Report

Rome meeting – June 2006Rome meeting – June 2006

Claude Huc


ContentsContents

CNES standards, evolution of data engineering domain (RNC)

Data exchange standard for archiving, defined in France

Current work on the ‘Producer Archive Interface Specification’


CNES standards (RNC)CNES standards (RNC)current content of current content of

the data engineering domainthe data engineering domain

At the highest level An introductory document justifying its existence The OAIS reference model

On a technical level A document defining Criteria for evaluating data formats The DEDSL abstract standard The DEDSL XML/DTD syntax The EAST specification The PAIMAS

New document recently introduced: Long term archiving of data:Applicable rules and recommendations for projects producing

data


CNES standards (RNC)CNES standards (RNC)applicable rules and recommendations for projectsapplicable rules and recommendations for projects

ContentsContents

Description of participants and their roles3 Basic rules

The basic information to be archived should be precisely defined with close cooperation between the Project team and the Archiving department.

The objective is not to simply keep files but to ensure that the information will always be available in a useable form.

The user communities for this data should be clearly identified

Archiving of information from Space experiments involves at least : Archiving the core data for the experiment:

main data, auxiliary data (orbit, attitude, etc.),

Archiving metadata associated with the data, Archiving the documentation.


CNES standards (RNC)CNES standards (RNC) applicable rules and recommendations for projectsapplicable rules and recommendations for projects

contentscontents

Rules related with the syntax Data structure

Files with records XML files Data bases

Coding Interger and real numbers Character strings in data files

Image files Raster or vector format

Documentary data Structured files (XML, HTML, XHTML, Open Document Format/ISO 26300…) PDF/A files



Rules for content Synthesis of main content

Main data Complementary data for experimental data Metadata Documents and publications Graphic data

Main data Identification Calibration Quality Standardisation: time, date, latitude, longitude, spatio-temporal reference Domain standards



Metadata Metadata for identifying a set of data

Overall description of a set of data Description of granules

Metadata for using data Auxiliary data

Data to help researchers

Documents and publications

Relation between project process and work phases defined in the PAIMAS

Project phases

Project-Archive interactions

Documents delivered by the Project

Comments

Phase A Preliminary definition phase

Archiving plan, draft version The preliminary agreement between the Project team and the Archive department should be described in detail during the Preliminary Requirements Review (REP).

Phase B Formal definition phase

Archiving plan, complete version

The formal agreement between the Project team and the Archive Department should be described in detail during the Preliminary Design Review (RDP).

Phase C and D

Formal definition phase

Final version of archiving plan Definition, production and qualification of ground segment.

Phase E – kick-off

Transfer and validation phaseFormal acceptance prior to beginning transfer to ensure:- compliance of data with what is expected- correct operation of transfer process

Possible revision of archiving plan during phase E.Acceptance report

The archiving plan may be revised depending on events which occur during phase E: total or partial failure of an instrument or on the contrary extending of the mission beyond its nominal duration.

Phase E - routine

Transfer and validation phase

In-depth validation.

The data and documents to be archived are not all delivered by the Project team at the same time. The coherence and completeness of all documents should thus be validated during phase E.


Exchange standard for dataExchange standard for data to be archivedto be archived

Published in March 2006 by the Ministry of Culture and Communication and the Ministry of the Economy, Finances and Industry

Purpose: to exchange information between the Archive departments and their partners

Producers

Archive management

Supervisory bodies

Archive users



This covers: transfer between producer and archive (SIP)

transfer between archive and user community (DIP)

changes to the archive at the producer’s request

deletion of documents



Example of sequence diagram

Producer Archive

SIP transfer

Transfer receipt

Notification of SIP validation

Anomaly warning

Receipt of anomaly warning



UML modellingTaking into account of signatures

XMLDsig

PKCS7Implementation as an XML diagram

Objective to automate transfer and validation while taking national specificities and regulations into account


Ongoing work for Producer Ongoing work for Producer Archive Interface SpecificationArchive Interface Specification

The document is currently being thoroughly revised

Some concepts have been changed

The terminology has also evolved

Much thought has gone into the work while taking into account: the initial work achieved analysis done by NASA descriptions of actual cases the new version of XFDU

This will be covered in a special presentation


The end…The end…