Upload
jake-williams
View
218
Download
3
Tags:
Embed Size (px)
Citation preview
NVO Summer School, Aspen Sept 14 2004 1
Data Access Layer Working GroupImage and Spectral Access
Doug TodyNational Radio Astronomy Observatory
National VIRTUAL OBSERVATORY
NVO Summer School, Aspen Sept 14 2004 2
DAL Services
Dataset
Time Series
Table Source Catalog
Event List
Visibility Data
Image NDImage
1D Spectrum
SED
NVO Summer School, Aspen Sept 14 2004 3
Simple Image Access (SIA)
• Provides access to "image" data– (instead of spectrum, catalog, etc.)– regularly sampled (pixelated) data– generally an image of the sky, with a WCS
• Service-oriented data discovery– query service to discover data
• Access to image metadata– can get image metadata without retrieving the actual
image– uniform description based on standard data models
• Access to image datasets– data may be virtual or computed on demand– uniform interface to any type of image data
NVO Summer School, Aspen Sept 14 2004 4
SIA - Basic Usage
• Simplest possible query– HTTP GET, e.g., http://myvo.nvo.org/SIAPServlet?POS=12.0,10.0&SIZE=0.2
– similar to cone search but ROI is a rectangle defining the ideal image coverage on the sky, not merely a search region
• Query response– VOTable describing images, one image per row– for each image there is an access reference, a URL
• Get Data– simply fetch the image at the given URL
• SIA is deceptively simple! It can do a lot more than is apparent, but simple usage should be kept simple.
NVO Summer School, Aspen Sept 14 2004 5
SIA - Interface Concepts
• Service protocol independent– URL (REST), WS, ADQL all implement the same interface
model– document-oriented, pass through
• Relational query model– relational: flat table, relationships inferred from metadata– generally it is up to the client to refine the query
• Uniform access to data– atlas, cutout, etc., images all treated the same– mediation to a standard data model (partial currently)
• Image service types– cutout, mosaic, atlas, pointed
NVO Summer School, Aspen Sept 14 2004 6
SIA - Interface Concepts
• Virtual data– most VO data analysis uses virtual data– virtual data is generated on the fly by the service– may involve subsetting, filtering, transformation,
analysis
• Data Model-based data access– addresses problem of heterogeneous data– allows disparate software to share the same data– same physical data can be viewed via different
models, e.g., image or spectrum
NVO Summer School, Aspen Sept 14 2004 7
SIA - Interface Summary
• Query– Simple positional query
• POS, SIZE– Image FORMAT
• FITS, graphic, HTML, metadata
• Image generation parameters– fully specify projection on the sky but simplify FITS WCS– naxis, cframe, equinox, crpix, crval, cdelt, rotang, proj– defaults are derived from POS, SIZE
• Others– Intersect (covers, enclosed, center, overlaps)– service defined, e.g., filter or bandpass name
NVO Summer School, Aspen Sept 14 2004 8
SIA - Interface Summary
• Query response– Response is a VOTable– One candidate image per table row– Includes standard metadata, including WCS
• title, date, pos, naxes, naxis, scale, format, etc.
– FITS WCS parameters• subset but includes CD matrix
– spectral bandpass metadata– service processing metadata
• did service interpolate pixels?
NVO Summer School, Aspen Sept 14 2004 9
SIA - Interface Summary
• Get Data– The image "access reference", a URL, is
used to fetch the dataset– URL often points to a service which
generates data on-the-fly (OTF)• e.g., image cutout or mosaic
– A separate get is required for each image– Note the query and get may be performed
by different clients• multiple get operations may proceed concurrently• Use of URL permits caching of images
NVO Summer School, Aspen Sept 14 2004 10
SIA - Interface Summary
• Staging Data– Included in SIA interface design, but not yet implemented
– Asynchronous staging of data is required for large computations
• e.g., a large image mosaic, or generation of 10000 cutouts
– Interface• same as for synchronous image access (same query, getData)• adds accessImage method, messaging, polling, multiple
clients• third party delivery possible, including MySpace
NVO Summer School, Aspen Sept 14 2004 11
SIA - Future Work
• Advanced queries– query on additional image metadata, e.g., collection,
bandpass, time– syntactical queries (ADQL), queries on virtual tables
• Extended data model– metadata standardization (UCD normalization)– dataset characterization, identification, provenance– image subtypes, e.g., image cube, synoptic imagery
• Query response– intelligent ranking of query response (like Google)– logical grouping of related images, e.g., multi-band survey
data– metadata extension mechanism, e.g., as for AVO demo
• Data Access– Staging of data, authentication
NVO Summer School, Aspen Sept 14 2004 12
Simple Spectral Access
• Provides access to "spectral" data– similar to SIA but deals with tabular spectrophotometric
data• Service-oriented data discovery
– query service to discover data• Access to dataset metadata
– can get dataset metadata without retrieving actual dataset– uniform interface based on standard data models
• Access to actual dataset– data may be virtual, i.e., computed on demand– uniform interface to any type of spectral data– hides details of how data is stored or represented
externally
NVO Summer School, Aspen Sept 14 2004 13
SSA - Basic Usage
• Simplest possible query– HTTP GET, e.g., http://myvo.nvo.org/SSAPServlet?
POS=12.0,10.0&SIZE=0.2– other query types, e.g., WS, or ADQL in the future, also possible
• Query response– VOTable describing spectral datasets, one per row– for each dataset there is an *access reference*, a URL
• Get Data – simply fetch the dataset at the given URL– returned data adheres (normally) to a standard data model and data
format• Data Format
– A returned 1D spectrum may be a simple VOTable (or text file, or FITS binary table, etc.) with some general metadata followed by a simple spectrum table with wavelength, flux, and uncertainty columns.
• Once again, although basic usage is simple, the interface can do more than is immediately apparent.
NVO Summer School, Aspen Sept 14 2004 14
SSA - Interface Scope
• SSA deals with several types of data– Spectral Energy Distributions (SEDs) – 1D spectra– time series
• Why this grouping?– common spectrophotometric data model– all are sampled, spectrophotometric, tabular
data
NVO Summer School, Aspen Sept 14 2004 15
SSA Data Model
• Sampled spectrophotometric sequence– projected at constant time results in 1D spectrum– projected at constant spectral value results in time
series– projecting both results in a photometry point
• A SED is:– a collection of these three types of objects– at a constant point on the sky (usually!)– typically spanning a wide range of spectral values
• Summary– a SED attempts to describe the full spectral energy
distribution of an object, encompassing as much of the emitted energy as possible
NVO Summer School, Aspen Sept 14 2004 16
SED Composition
spectrum segment
Photometry point
NVO Summer School, Aspen Sept 14 2004 17
NVO Summer School, Aspen Sept 14 2004 18
OTF Generation of Spectral Data
• Spectral Archives– spectral data resembles catalog data as much as
image data– most spectral data access will probably be to pre-
computed data
• Virtual Data examples– spectrum from an image cube– SED from multi-band image data (plus catalog
data etc.)– time series from synoptic imagery or catalog data– spectrum or time series from radio Pulsar data
NVO Summer School, Aspen Sept 14 2004 19
SSA Data Formats
• Concepts– science data model (SDM)
• semantic model for the data - what IS this data– export data format (EDF)
• expresses the SDM in a specific data representation• identically the same SDM regardless of representation
• Formats– native XML– VOTable– FITS binary table– text table, e.g., CSV– plus graphics, HTML– spectral data can also be viewed as an image, with
restrictions
NVO Summer School, Aspen Sept 14 2004 20
SSA Service Interface
• Each class of data gets a separate interface– SED, spectrum, time series– similar but separate access interfaces preferred
• Similar to SIA– query, query response, getData– additional query parameters
NVO Summer School, Aspen Sept 14 2004 21
SSA Query
• Required parameters– POS, SIZE, FORMAT– region is circular, as for cone search, unlike SIA– FORMAT provides more options than just FITS for
the EDF, including XML (native and VOTable), and text
• Optional parameters– time, bandpass, collection, ID, rank– aperture, verbosity
NVO Summer School, Aspen Sept 14 2004 22
SSA Query Response
• VOTable– one table row per candidate dataset– access reference (URL) used to fetch data– component data models included directly as
objects– uses GROUP, UTYPE from VOTable 1.1