Reduced data products in the ESO Phase 3 archive

(Status: 02 August 2017)

The ESO Phase 3 archive provides access to reduced and calibrated data products. Allthese data are stored in standard formats and can be queried via a unified query interfacehere:http://archive.eso.org/wdb/wdb/adp/phase3 main/form. The data are either provided by exter-nal users such as the ESO Public Survey (see Table 1) consortia and the ESO Large Programmeteams, or produced by ESO using science grade reduction pipelines (see Table 2). Here we givean overview of the types of data currently provided. For details see the ESO Science DataProduct Standard and the release descriptions provided for each data collection available atthe following URLs, from the community http://eso.org/rm/publicAccess#/dataReleases and the in-house processed products http://www.eso.org/sci/observing/phase3/data streams.html.

Calibrated images, flux maps and source lists

The VISTA and VST public imaging surveys provide reduced, photometrically and astromet-rically calibrated images. The VISTA/VIRCAM and VST/OmegaCAM instruments withtheir mosaic CCDs produce non-contiguous images of the sky in a single exposure. The gapsbetween the single detectors of OmegaCAM are much smaller than those of VIRCAM. Thosenon-contiguous images (also called “pawprints”) are stored in multi-extension FITS files, whereeach extension corresponds to one of the 16 (VIRCAM) or 32 (OmegaCAM) images producedby the single detector chips.

The building blocks of the VISTA surveys are the “tile” images. The tile is a combinedimage of the field of view with no gaps (see Fig. 1,2). Tile images resulting from the coadditionof several individual images are termed as deep tiles (e.g. UltraVISTA, VIDEO). Tile imagesare provided as single-extension FITS files. The VST surveys follow a similar approach ofcombining multiple exposures. However, the resulting contiguous images are either re-griddedas single-extension tile images (e.g. KiDS) or stored as multi-extension FITS files (e.g. ATLAS,VPHAS+). The tabular data of sources extracted from imaging data products are named“source lists”. Typically, each image has a corresponding single-band source list. Ancillary dataproducts are associated to scientific data products without being directly searchable via thequery interface. For imaging data, weight maps giving the statistical significance of each pixelare associated to the images (see Fig. 2, right panel; Fig. 3, upper right panel). Other optionaldata products, sometimes associated to imaging data, are gain maps specifying the number ofelectrons that contributed to each pixel value of the image or masks to mark corrupted regionsof the image (Fig. 3, lower panel). These ancillary data are stored in separate FITS files andautomatically delivered together with the selected science files for download.

Since 2016, the flux maps at a wavelength of 870 µm of 420 sq. deg of the Milky-Wayinner plane from the APEX Telescope Large Area Survey of the Galaxy (ATLASGAL) surveyare available from the ESO archive. The variance map, source lists and associated catalogs arealso provided.

A recent addition to the portfolio of reduced images in the ESO Phase 3 archive are theHigh Acuity Wide field K-band Imager (HAWK-I) near infrared images and source lists, whichare produced as part of the ESO in-house reprocessing and also as science products from ESOProgrammes, see for example the deep HAWK-I images for the Hubble Frontier Fields. TheHAWK-I products include multi-extension stack exposures (4 detector chips), mosaiced tilesand source lists, plus variance and confidence maps as ancillary files.

Calibrated spectra and data cubes

The public spectroscopic surveys at the VLT and the NTT provide one-dimensional reduced andwavelength calibrated spectra in FITS binary table format (Fig. 4). Information associated toscience spectra is stored within separate columns of the same FITS-file as the science spectrathemselves. Each data array (i.e. wavelength, flux, error, quality...) is provided in a single cellof the one row contained in the binary table. Wavelengths can be calibrated using wavelengthlists measured in dry air (at standard temperature and pressure) or in vacuum, according tothe value captured by the unified content descriptor for the wavelength array in the product’sheader.

Reduced spectra of selected instruments are also produced in house by ESO using sciencegrade pipelines. Until now archival data from UVES, XSHOOTER, HARPS, FLAMES/GIRAFFEand FEROS have been reprocessed and published. Newly obtained spectra are automaticallyreduced shortly after the observation. They are immediately accessible to the PIs via the userportal account and released to the public after the proprietary period.

An important addition to the ESO in house processed products are the Integral Field UnitMulti-Unit Spectroscopic Explorer (MUSE) reduced data cubes. Each MUSE data cube hasa three-dimensional image with two spatial and one spectral coordinates, in its first extension;in the second extension contains the error information. The MUSE products are wavelength(in vacuum) and flux calibrated, and each of them come with an associated additional fits filethat contains the two-dimensional white-light collapsed data cube image. See Fig. 5 for anexample of a three-dimensional calibrated cube and the corresponding white-light image, withadditional color coding of the two-dimensional velocity field of the ionized gas in a galaxy.

Catalogues

Catalogues may consist of sources extracted and band-merged from imaging data products,redshifts and additional physical parameter from spectra, or weak lensing shear measurements(e.g. KiDS release 3.1). These catalogues can be queried for single or multiple sources via thePhase 3 catalogue query interface (http://www.eso.org/qi/) by performing a cone or box search.Alternatively, the catalogues can also be downloaded partly or as a whole from the Phase 3main query form of the ESO archive, together with the image or spectral products from whichthe measurements are extracted. Simple catalogues are stored in one single binary table. Largercatalogues are stored in multiple files. For imaging surveys, each data file contains the extractedband-merged catalogue of one corresponding survey tile (see Fig. 6, lower panel). In additionto the multi-band source catalogues, imaging surveys with multi-epoch observations providelight curve catalogues and catalogues of variable objects (e.g. VVV, VMC and PESSTO, seeFig. 6, upper panel).

Table 1: Types of reduced data products provided by the ESO Public Surveys (August 2017).

Survey Single FITS Multi-extension FITS Single-band 1D spectra multi-band/highimages images source lists level catalogues

VHS X X X XVIKING X X X XUltraVISTA X X XVIDEO X X X XVVV X X XVMC X X X X

KiDS X X XVPHAS+ X X XATLAS X X X

PESSTO X X XGaia-ESO X XLEGA-C X XVANDELS X

ATLASGAL X X X

Table 2: Types of reduced data products provided by the ESO in house reprocessing (August2017).

Data Stream Single FITS Multi-extension FITS Single-band 1D spectra 3D cubesimages images source lists

UVES XXSHOOTER XHARPS XFLAMES/GIRAFFE XFEROS X

HAWKI X X X

MUSE XMUSE DEEP X

Figure 1: Contiguous tile, formed by combination of six overlapping pawprints taken withVIRCAM (VISTA/VIRCAM User Manual, VIS-MAN-ESO-06000-0002).

Figure 2: Examples for data products from VISTA surveys: Tile image (left panel)and associated weightmap (right panel) obtained with VIRCAM in the course of the VMCPublic Survey. Since the tile image has been combined from several overlapping pawprints, thesignal-to-noise per pixel is not homogeneous across the area.

Figure 3: Examples for data products from VST surveys: Tile image (upper left panel),associated weightmap (upper right panel) and associated mask (lower panel) obtained withOmegaCAM in the course of the KiDS Public Survey. The mask marks excluded regions of theimage.

650 660 670 680

20000

40000

60000

WAVE (nm)

FLUX (adu)

ADP.2013-11-11T14_11_58.683.fits(FLUX_1-1)

Figure 4: Examples for 1D spectra: NIR spectra of the supernova SN2012ec (Barbarino etal. 2015, MNRAS, 448, 2312) taken with SOFI in the course of the PESSTO Public Survey(left panel). Optical spectrum of a star in an open cluster taken with GIRAFFE in the courseof the Gaia-ESO Public Survey (right panel).

Figure 5: Example of a 3D cube and white lamp image: example of a 3D cube for apeculiar galaxy with ionized gas disk (left panel), and white lamp image for the same galaxy,with the ionized gas distribution that is color coded according to the approaching/recedingvelocities. Observation acquired during MUSE commissioning in 2014 (Bacon et al. 2014, TheMessenger 157, 13).

Figure 6: Examples for high-level scientific catalogues: Light curves of classical Cepheidsobserved with VIRCAM in the course of the VMC survey (upper panel, Ripepi et al. 2012,MNRAS, 424, 1807). The VMC catalogues published via Phase 3 contain light curves ofall objects in different bands, the properties of variable sources as well as the properties ofknown Cepheids in the observed fields. Lower panel: Two-color diagram constructed froma multi-band tile catalogue of the VIKING Public Survey (VIKING catalogue DR1, Releasedescription). In addition to the calibrated photometry in the different bands the cataloguecontains more information like the shape of the sources. Here point-like stellar sources areplotted in blue, while extended sources are plotted in red.