VO Course 06: VO Data-models

VO, Data Models, and VO ArchivesJuan de Dios Santander Vela (IAA-CSIC)

Overview

Data Model Definition

IVOA and Data Modelling

Radio Astronomy DAta-Model for Single-dish

Data Models

Data Model Definition

Complete description of the set of entities needed for information storage in a particular field, which specifies both the data being stored, and the relationships between them.

JUAN DE DIOS SANTANDER [email protected]

INSTITUTO DE ASTROFÍSICA DE ANDALUCÍA (CSIC)BECARIO I3P DE POSTGRADO (01/2006-12/2006)BECARIO PREDOCTORAL IAA (03/2005-12/2005)

Information Structure

mailto:[email protected]






















































DATA MODEL DEPENDENT



Data model elements

Fields (Attributes in OOP, XML)

Data types

Semantics

Relationships



Field Object Number Source RA Dec Epoch

Data Type BigInt String Float Float String Code

Example Data

3123453445OBGGALAX_1

‘CIG 96’96

132.34788h12º50’52’’

-05.3345-0h5º20’42’’

‘J2000’‘Epoch J2000’

ATTRIBUTES & DATA TYPES

Information StructureSEMANTICS & UCDS

Field Object Number Source RA Dec Epoch

Meaning Object identifier Source name Right

Ascension Declination Epoch code

UCD meta.id; meta.code

meta.id; meta.name

pos.eq.ra; meta.main

pos.eq.dec; meta.main

time.epoch; meta.code

Elements of a DMEntities

Data model building blocksGrouping of related attributes

FieldsActual data elements of the model

RelationshipsBetween attributes in different Entities

Cardinalities Data types

Units

Semanticsvia UCDs & UTypes

Data Model Roles

Discovery

Evaluation

Data Access

Transformation

Data Model Roles

Discovery

Evaluation

Data Access

Transformation

IVOA Data Modelling

IVOA Data Modelling

IVOA Data Modelling

IVOA Data Modelling

IVOA Data Modelling

IVOA Data Modelling

Observation Data Model

Observation

ObsData Characterisation Target

Coverage

Location Support Sensitivity

Resolution Precision

Curation Provenance

Observing Configuration

Observing Elements

Processing

Calibration

Independent

Dependent

of observationdata reduction

DRAFT

Observation Data Model

Observation

ObsData Characterisation Target

Coverage

Location Support Sensitivity

Resolution Precision

Curation Provenance

Observing Configuration

Observing Elements

Processing

Calibration

Independent

Dependent

of observationdata reduction

DRAFT

Characterisation DM

Figure 4: The layered structure of Characterisation: This diagram synthe-sises the Property/Axis/Layer approach. The concepts are represented in yel-low. The coarse description is designed by the first level (blue boxes), while thepale blue ones represent the complementary metadata. The Bounds, Supportand Sensitivity classes are nested levels of detail to add knowledge about theCoverage of an Observation. Symmetrically, Resolution and Sampling mayalso have the 4-level structure of description. The complete Characterisa-tion for one observation is obtained by filling the tree for each relevant axis:spatial, spectral, temporal, etc.

17

Characterisation DM: Errors

Figure 5: This class diagram illustrates the CharacterisationAxis class andits relationship with the Accuracy class, which encompasses various types oferrors such as systematic or statistical.

20

Spectral Data Model

Spectrum

Target Data Characterisation CoordSys Curation DataID Derived

Spectral DM: Data

TimeAxis

SpectralAxis

n

Accuracy

value,unit

FluxAxis

StatError SysError

DataAxis

Data

StatErrHighStatErrLow

BinSize

BinHigh

BinLow

Background

Model

ucd

unit

ResolutionQualityValue

1,..SegmentSize

Figure 2: Diagram for Data object

Characterization

CharacterizationAxis

SpatialAxis

TimeAxis

SpectralAxis

n

Accuracy Resolution

value,unit

FluxAxis

Locationvalue

Bounds

Extent Start

Stop

Support

Area Fill

StatError SysError

Calibration

BinSize

Coverage

unit

ucd

name

Figure 3: Diagram for Characterization object8

Spectral Characterisation

TimeAxis

SpectralAxis

n

Accuracy

value,unit

FluxAxis

StatError SysError

DataAxis

Data

StatErrHighStatErrLow

BinSize

BinHigh

BinLow

Background

Model

ucd

unit

ResolutionQualityValue

1,..SegmentSize

Figure 2: Diagram for Data object

Characterization

CharacterizationAxis

SpatialAxis

TimeAxis

SpectralAxis

n

Accuracy Resolution

value,unit

FluxAxis

Locationvalue

Bounds

Extent Start

Stop

Support

Area Fill

StatError SysError

Calibration

BinSize

Coverage

unit

ucd

name

Figure 3: Diagram for Characterization object8

Space-Time Coordinates

Radio Astronomy DAta-Model for Single-dish

RADAMS’ SourcesData Model for Observation

McDowell, Bonnarel et al., IVOA Data Model WG Internal Draft

Data Model for Astronomical Dataset Characterisation

McDowell, Bonnarel et al., IVOA Data Model WG Note

IVOA Spectral Data ModelMcDowell, Tody et al., IVOA Data Model WG Working Draft

IVOA Data Model for Raw Radio Telescope DataLamb and Power, IVOA Radio IG Note

RADAMS StructureObservation

ObsData

Characterization Provenance

Curation

Policy

PackagingTarget

Accuracy

SensitivityPrecisionResolutionCoverage


ObsData


Curation

Policy

PackagingTarget

Accuracy


Data Model for Observation


ObsData


Curation

Policy

PackagingTarget

Accuracy


Data Model for Dataset

Characterisation


ObsData


Curation

Policy

PackagingTarget

Accuracy

SensitivityPrecisionResolutionCoverage Undefined Data Models

RADAMS Detail36

CHAPTER 7.DETAILED DESCRIPTION OF RADAM CLASSES

Table 7.1:AxisF

rame.Spatialmetad

ata.

FITS

Attribute

Keyword

UCD

Descrip

tion

axisName

assign

meta.i

d;

meta.m

ain

Axisname.

calibrati

onStatus

assign

obs.ca

lib;

meta.c

ode

Calibrati

on status from

a con-

trolled

vocabulary

:unc

al-

ibrate

d,cal

ibrate

d,rel

a-

tivea , no

rmaliz

edb .

ucd

assign

meta.u

cd;

meta.m

ain

MainUCD for

the axis.

unit

assign

meta.u

nit;

meta.m

ain

Mainunits for

the axis.

refPos

assign

meta.r

ef;

meta.i

d

Identificati

onof the orig

in po-

sition

withinthe spaceR

ef-

Frame from

a control

ledvoc

ab-

ulary; See Space-

Time Coordi-

nateData

Model [13].

spaceRefFram

e

WCSNAM

E

orRAD

ESYS

pos.fr

ame;

meta.i

d

Identificati

onof the refe

rence

system

froma con

trolled

vo-

cabulary

; seeSpace-

Time Co-

ordinate

DataModel [13]

: FK4,

FK5, EL

LIPTIC

. . .

coordEquinox

assign

pos;

time.e

quinox

Equinox (only if need

ed).

epoch

assign

pos;

time.e

poch

Epoch(on

ly if needed).

independentAxis

assign

pos;

obs.pa

ram;

meta.c

ode

Boolean

flagindicat

ing

whetherthe axis

is indepen-

dentof the rest

or not.

undersamplingStatu

sass

ign

pos;

obs.pa

ram;

meta.c

ode

Boolean

flagindicat

ing

whether the dataare

sampled

in this axisor not.

regularS

tatus

assign

pos;

obs.pa

ram;

meta.c

ode

Boolean

flagused

incase

ofsam

pleddata,

indicating

whether sampling is regu

laror

not.

a relati

verefe

rs to calibrate

d data,exce

pt foran additiv

e or multiplica

tivecon

stant.

b normal

ized refe

rs to dimensionless

quantities

, such as thoseresu

lting from

the division

between

two commensurab

le datasets.

7.5. PACKAGING

61

Table 7.40: Calibration metadataa .

FITS

AttributeKeyword UCD

Description

timestampDATE-RED

obs.param;

time.epoch

Timestamp for the calibration

step being performed.

parameter.nameassign

obs.calib;

obs.param;

meta.id

Keyword defining the parameter

that we will characterisewith the

remaining attributes.

parameter.typeassign

obs.calib;

obs.param;

meta.code

Type of calibration parameter

used, from a controlled vo-

cabulary: additive, factor,

polynomial, exponent

ial,

logarithmic.

parameter.valueassign

obs.calib;

obs.param;

meta.number

Value for the main calibra-

tion parameter, where parame-

ter.type is not polynomial.

parameter.sigma assignobs.cali

b;

obs.param;

meta.number

Value of sigma, for exponential

calibrations.

parameter.calCoe�.[n] assignobs.cali

b;

obs.param;

meta.number

nth degree coe⇥cient for a

polynomial calibration parame-

ter; polynomial degree is derived

from the maximum n.

aIt is mandatory that at least one [parameter.name,

parameter.type,

parameter.value]

triplet appears, with fluxScale as parameter.name, and one of

antennaTemperatu

re, mbBrightnessTemp

erature, or S nu as the parameter.value, with a

parameter.type of string.

Observation

• Target.Name

• Target.Descrip

tion

• Target.Class

• Target.Pos

• Target.spectra

lClass

• Target.redshift

.statError

• Target.redshift

.Confidence

Target

Figure 7.9: Target data model.

order to find a maser, but that Target will be most likely included within an

Astronomical object, such as a planetary nebula, associated to that Target.

7.5 Packaging

The Packaging class has to deal with how di�erent pieces of data will be

presented. For instance, if we wanted to retrieve data belonging to the maser

60CHAPTER 7. DETAILED DESCRIPTION OF RADAM CLASSES

Table 7.39: Processing Step metadata.FITS


Description

timestampDATE-RED obs.param;time.epoch Timestamp for the processing


type

assignobs.param;meta.code Type of processing applied

to source data; comes from a

controlled vocabulary: unproc-

essed, noiseWeightedAverage,

nonWeightedAverage.

softwarePackageassign

meta.software;meta.id Software package used for data

processing; should come from

a controlled vocabulary: CLASS,

AIPS, AIPS++, CASA, MOPSIC,

GILDAS, MIRA, MIR, other. In the

case of other, the actual package

that was used should be added

as a parameter, with param-

eter.name as softwarePackage

and the parameter.value as the

package name.

parameter[n].name assignobs.param;meta.code Additional processing parameter

name, whose value will be in pa-

rameter.value; eventually, we will

have a controlled list of possible

parameter.name values.

parameter[n].type assignobs.param;meta.code From a controlled vocabulary:

integer, float, string. . . At

least all of FITS data types should

be present.

parameter[n].value assignobs.parama

Value for the parameter indicated

by parameter.name.

aThe final UCD to mark parameter[n].value will be calculated when writing the VOTable,

as it depends on parameter.type; it will be obs.param; meta.number most of the time, but

it could be obs.param; meta.name or obs.param; meta.code, depending on the context.

RADAMS Detail36

CHAPTER 7.DETAILED DESCRIPTION OF RADAM CLASSES

Table 7.1:AxisF

rame.Spatialmetad

ata.

FITS

Attribute

Keyword

UCD

Descrip

tion

axisName

assign

meta.i

d;

meta.m

ain

Axisname.

calibrati

onStatus

assign

obs.ca

lib;

meta.c

ode

Calibrati

on status from

a con-

trolled

vocabulary

:unc

al-

ibrate

d,cal

ibrate

d,rel

a-

tivea , no

rmaliz

edb .

ucd

assign

meta.u

cd;

meta.m

ain

MainUCD for

the axis.

unit

assign

meta.u

nit;

meta.m

ain

Mainunits for

the axis.

refPos

assign

meta.r

ef;

meta.i

d

Identificati

onof the orig

in po-

sition

withinthe spaceR

ef-

Frame from

a control

ledvoc

ab-

ulary; See Space-

Time Coordi-

nateData

Model [13].

spaceRefFram

e

WCSNAM

E

orRAD

ESYS

pos.fr

ame;

meta.i

d

Identificati

onof the refe

rence

system

froma con

trolled

vo-

cabulary

; seeSpace-

Time Co-

ordinate

DataModel [13]

: FK4,

FK5, EL

LIPTIC

. . .

coordEquinox

assign

pos;

time.e

quinox

Equinox (only if need

ed).

epoch

assign

pos;

time.e

poch

Epoch(on

ly if needed).

independentAxis

assign

pos;

obs.pa

ram;

meta.c

ode

Boolean

flagindicat

ing

whetherthe axis

is indepen-

dentof the rest

or not.

undersamplingStatu

sass

ign

pos;

obs.pa

ram;

meta.c

ode

Boolean

flagindicat

ing

whether the dataare

sampled

in this axisor not.

regularS

tatus

assign

pos;

obs.pa

ram;

meta.c

ode

Boolean

flagused

incase

ofsam

pleddata,

indicating

whether sampling is regu

laror

not.

a relati

verefe

rs to calibrate

d data,exce

pt foran additiv

e or multiplica

tivecon

stant.

b normal

ized refe

rs to dimensionless

quantities

, such as thoseresu

lting from

the division

between

two commensurab

le datasets.

7.5. PACKAGING

61

Table 7.40: Calibration metadataa .

FITS


Description

timestampDATE-RED

obs.param;

time.epoch

Timestamp for the calibration


parameter.nameassign

obs.calib;

obs.param;

meta.id

Keyword defining the parameter

that we will characterisewith the

remaining attributes.

parameter.typeassign

obs.calib;

obs.param;

meta.code

Type of calibration parameter

used, from a controlled vo-

cabulary: additive, factor,

polynomial, exponent

ial,

logarithmic.

parameter.valueassign

obs.calib;

obs.param;

meta.number

Value for the main calibra-

tion parameter, where parame-

ter.type is not polynomial.

parameter.sigma assignobs.cali

b;

obs.param;

meta.number

Value of sigma, for exponential

calibrations.

parameter.calCoe�.[n] assignobs.cali

b;

obs.param;

meta.number

nth degree coe⇥cient for a

polynomial calibration parame-

ter; polynomial degree is derived

from the maximum n.

aIt is mandatory that at least one [parameter.name,

parameter.type,

parameter.value]

triplet appears, with fluxScale as parameter.name, and one of

antennaTemperatu

re, mbBrightnessTemp

erature, or S nu as the parameter.value, with a

parameter.type of string.

Observation

• Target.Name

• Target.Descrip

tion

• Target.Class

• Target.Pos

• Target.spectra

lClass

• Target.redshift

.statError

• Target.redshift

.Confidence

Target

Figure 7.9: Target data model.

order to find a maser, but that Target will be most likely included within an

Astronomical object, such as a planetary nebula, associated to that Target.

7.5 Packaging

The Packaging class has to deal with how di�erent pieces of data will be

presented. For instance, if we wanted to retrieve data belonging to the maser

60CHAPTER 7. DETAILED DESCRIPTION OF RADAM CLASSES

Table 7.39: Processing Step metadata.FITS


Description

timestampDATE-RED obs.param;time.epoch Timestamp for the processing


type

assignobs.param;meta.code Type of processing applied

to source data; comes from a

controlled vocabulary: unproc-

essed, noiseWeightedAverage,

nonWeightedAverage.

softwarePackageassign

meta.software;meta.id Software package used for data

processing; should come from

a controlled vocabulary: CLASS,

AIPS, AIPS++, CASA, MOPSIC,

GILDAS, MIRA, MIR, other. In the

case of other, the actual package

that was used should be added

as a parameter, with param-

eter.name as softwarePackage

and the parameter.value as the

package name.

parameter[n].name assignobs.param;meta.code Additional processing parameter

name, whose value will be in pa-

rameter.value; eventually, we will

have a controlled list of possible

parameter.name values.

parameter[n].type assignobs.param;meta.code From a controlled vocabulary:

integer, float, string. . . At

least all of FITS data types should

be present.

parameter[n].value assignobs.parama

Value for the parameter indicated

by parameter.name.

aThe final UCD to mark parameter[n].value will be calculated when writing the VOTable,

as it depends on parameter.type; it will be obs.param; meta.number most of the time, but

it could be obs.param; meta.name or obs.param; meta.code, depending on the context.

RADAMS: OverviewObservation

ObsData

Characterization

Coverage Resolution Precision

Target Provenance Curation Policy Packaging

Accuracy

perAxisType

LocationBoundsSupportSensitivity


ObsData

Characterization



Accuracy

perAxisType


Original to RADAMS


ObsData

Characterization



Accuracy

perAxisType


Original to RADAMS

Specified forradio astronomy

RADAMS — ObsDataObsData

AxisFrame.Observable

-location-bounds-support

Coverage.Observable

Accuracy

Resolution.Observable

SamplingPrecision.Observable

AxisFrame.Spectral

Accuracy


Coverage.Spectral

Resolution.Spectral

SamplingPrecision.Spectral

AxisFrame.Temporal


Coverage.Temporal

Accuracy

Resolution.Temporal

AxisFrame.Spatial


Coverage.Spatial

Accuracy

Resolution.Spatial

CharDM — Spatial axis-dataID

ObsData

+ObsData.dataID-axisName: spatial-calibrationStatus-ucd: pos-unit: deg-spaceRefFrame: FK5-coordEquinox: J2000-refPos-independentAxis: true-undersamplingStatus: false-regularStatus: true

AxisFrame.Spatial

+ObsData.dataID+AxisFrame.Spatial-refVal

Resolution.Spatial

+ObsData.dataID+AxisFrame.Spatial-locationName-bounds

Coverage.Spatial

-quality-statError-sysError

Accuracy

+ObsData.dataID+AxisFrame.Spatial+Resolution.Spatial.refVal-value: HPBW

Resolution.Spatial.RefVal

+Coverage.Spatial.locationName-coord.Name: {RA, DEC}-coord.Value

Coverage.Spatial.Location

CharDM — Temporal Axis

-dataID

ObsData

+ObsData.dataID-axisName: spatial-calibrationStatus-ucd: time-unit: s-spaceRefFrame: UTC-refPos: TOPOCENTRIC-independentAxis: true-undersamplingStatus: true-regularStatus: false

AxisFrame.Temporal

+ObsData.dataID-refVal

Resolution.Temporal

+ObsData.dataID+AxisFrame.Temporal-locationName-bounds

Coverage.Temporal

-quality-statError-sysError

Accuracy

+Coverage.Temporal.locationName-coord.Name: time-coord.Value

Coverage.Time.Location

+Coverage.Temporal.bounds-coord.maxValue-coord.minValue

Coverage.Time.Bounds

-startValue[n]-endValue[n]

Coverage.Time.Support

CharDM — Spectral axis

-dataID

ObsData

+ObsData.dataID-axisName: spatial

-calibrationStatus-ucd: em

-unit: Tmb, Tsky, Jy

-refPos:-dopplerDef-independentAxis: true

-undersamplingStatus: false

-numBins: 384

-regularStatus: true

AxisFrame.Spectral

+ObsData.dataID-refVal

Resolution.Spectral

+ObsData.dataID+AxisFrame.Spectral-locationName-bounds

Coverage.Spectral

-quality-statError-statErrorHigh-statErrorLow-sysError-sysErrorHigh-sysErrorLow

Accuracy

+Coverage.Spectral.locationName-coord.Name: CentralFreq

-coord.Value-coord.Name: MaxPeakFreq

-coord.Value-coord.Name: MaxPeakValue

-coord.Value

Coverage.Spectrum.Location

+Coverage.Spectral.bounds-coord.maxValue-coord.minValue

Coverage.Spectral.Bounds


Coverage.Spectrum.Support

+ObsData.dataID-refVal: pixelScale

SamplingPrecision.Spectral

CharDM — Observable axis

-dataID

ObsData

+ObsData.dataID-axisName: spatial

-calibrationStatus-ucd: phot.flux

-unit: Jy/K, Jy/Beam

-independentAxis: true

-undersamplingStatus: false

-numBins: ADC bits/sample?

-regularStatus: true

AxisFrame.Observable

+ObsData.dataID-refVal: flux/SNR

Resolution.Observable

+ObsData.dataID+AxisFrame.Observable-locationName-bounds

Coverage.Observable

-quality-statError-statErrorHigh-statErrorLow-sysError-sysErrorHigh-sysErrorLow

Accuracy

+Coverage.Observable.locationName-coord.Name: MaxPeakFlux

-coord.Value

Coverage.Observable.Location

+Coverage.Observable.bounds-coord.maxValue-coord.minValue

Coverage.Observable.Bounds


Coverage.Observable.Support

+ObsData.dataID-refVal: ADC bits per sample?

SamplingPrecision.Observable

Target

• targetName

Observation

• name

• description

• class

• pos

• spectralClass

• redshift.statError

• redshift.Confidence

Target

ObservationProvenance

InstrumentConf

-name

AntennaConf

-polarisation: L,R,X,Y

Feed

-name

BeamConf

+AntennaConf.name

-mount

-majorAxis

-minorAxis

-effectiveArea

Antenna

-name

-description

-shortName

-locationName

-URL

Instrument

+Instrument.locationName

–latitude

-longitude

-height

Location

+beamConf.name

-beamMajor

-beamMinor

-sensitivity

-meanBeamSolidAngle

-minorBeamSolidAngle

-directivity

-gain

-resolution

Beam

+beamConf.Name

-skyCentreFreq

-ifCentreFreq

-bandwidth

Receiver

+beamConf.name

-chanSeparation

-freqResol

-velRefFrame

-refChanNum

-refChanFreq

-restFreq

-molecule

-transition

Spectrum

+beamConf.name

-chanSeparation

-velResol

-velRefFrame

-refChanNum

-refChanVel

-restFreq

-molecule

-transition

Velocity

{This set is defined

for each instrument

band AND receiver

configuration

Provenance.Instrument

Observation Provenance InstrumentConf

-timeStamp

-opacity

-temperature

-humidity

-wind

AmbientConditions+AmbientConditions.timeStamp

-opacity

-skydipStart

-azimuth

-elevation.[n]

-tsky.[n]

-atmosphericModel

OpacityCurve

Provenance. AmbientConditions

InstrumentConf

-timeStamp

Processing

Observation Provenance AmbientConditions

+Processing.timeStamp

-parameter.name

-parameter.kind

-parameter.value

-parameter.sigma

-parameter.calCoeff[n]

Calibration

+Processing.timeStamp

-kind

-softwarePackage

-parameter.name

-parameter.kind

-parameter.value

ProcessingStep

Provenance.Processing

Curation

Observation

-Publisher-PublisherID-Logo-Version-Contributor-Contact.Name-Contact.Email-Subject–Service.InterfaceURL-Service.BaseURL-Service.StandardURI-Service.StandardURL

Curation

-ProjectID-Title-Description

Project

-Name-Operator

Observer

-projCategory

ProjCategory

-projType

ProjType

-Title-Description-scheduleType

Proposal

-Name-Mail

Author

-startDate-endDate

Term

ObservingProgram

-observationID-title-description-creator-observerID-date-facility-instrument

DataID

-Name-Mail

PrincipalInvestigator

-Name-Mail

coInvestigator

common curation data

belongsto

encom

passes

has data

belongs to

observed by

origin

ate

s fro

m

written bywill perform

is one of

is o

ne o

f

scheduled at

Policy

+projectID

-observationID

-title

-description

-creator

+observerID

+operatorID

-date

-facility

-instrument

DataID

-ProjectID

+principalInvestigator

+coInvestigator[n]

-Title

-Description

Project

-userID

-name

-institution

-contactInfo

Users

+projectID

-roleKind

-permissions

Policy

providesuseridentification

User RoleAssignationAlgorithm

Role and Permissions

PolicyStart

Stop

obsData.PrincipalInvestigator.ID == loggedUser.ID?

loggedUser.role =principalInvestigator

obsData.observer.ID == loggedUser.ID?

loggedUser.role =observer

loggedUser.ID found in obsData.coInvestagators array?

loggedUser.role =coInvestigator

loggedUser.ID found on dbUsers, and observatoryStaff is true?

loggedUser.role =observatoryStaff

loggedUser.role =none

yes

no

yes

yes

yes

no

no

no

VOPack Schema/ schema originatingQuery

voPack@

ID

ID

@ version

description

originatingQuery

packUnit

1..

packUnit@

packUnitType

type

@

anyURI

URI

@

string

informationPath

description

packUnit

0..

characterisation

description

characterisation

References & Links

Integrating Radio Astronomical Archives and Legacy Tools in the VO Framework, Ph.D. Thesis, Chapter 5

Data Model for Astronomical DataSet Characterisation

Spectral Data Model

http://www.iaa.es/~jdsant/tesis.pdf






http://www.ivoa.net/Documents/latest/CharacterisationDM.html




http://www.ivoa.net/Documents/latest/SpectrumDM.html

http://www.ivoa.net/Documents/latest/SpectrumDM.html

Technology

VO Course 06: VO Data-models