EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 1

Chajnantor (5000m)

Paranal (2600 m)

La Silla (2400 m)

Paul Eglitis and Dieter Suchar

Historical lessons, inter-disciplinary comparison, and their application to the future evolution of the ESO Archive Facility and Archive Services

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 1

ESO, Karl-Schwarzschild Str. 2, 85748 Garching near Munich, Germany

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 2

Introduction

Inter-disciplinary thoughts

The ESO Science Archive Facility

Physical experiences: storage and infrastructure

Future growth requirements

Lessons Learnt

Future evolution

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 3

Archive Requirements: Inter-disciplinary Variation and Similarity

Space Mission

Science

Stable (Mission)

On Opportunity

IPDA, SPASEEUROPLANET

Earth Observation

Service, Science

Stable (Long Term)

Periodic

GEOSS, INSPIRE, GIGAS ,HMA

Ground‐Based Astronomy

Science

Variable

Variable

FITS, VO, IVOA

Particle Physics

Science

Variable

In‐house++

E‐GRID

Customer

Configuration

Data Timeliness

Data Volume

Interoperability

Always more!

…

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 4

CHILE

USER INTERFACES

SECONDARYARCHIVE

LocalDatabases

OBSERVATORIES

PRIMARYARCHIVE

ReplicatedDatabase

Infrastructure

HQ

ReplicatedDatabase

Infrastructure

DATAHANDLING

Rep

licat

ion

STAGING

TravelingDisk

On-LineData Transfer

ESO SCIENCE ARCHIVE

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 5

DISK

0.250.52

Hardware Changes

Inexpensive Upgrades

CD DVD* *

****

GB

Year

Rac

ks

Physical Archive History

cited.be

Paranal

La Silla Chajnanator

Future

VLT Very Large Telescope

CRIRES CRyogenic high‐resolution InfraRed Echelle Spectrograph

FLAMES Fibre Large Array Multi Element Spectrograph

FORS1 FOcal Reducer/low dispersion Spectrograph 1

FORS2 FOcal Reducer/low dispersion Spectrograph 2

HAWK‐I High Acuity Wide field K‐band Imager

ISAAC Infrared Spectrometer And Array Camera

NACO Nasmyth Adaptive Optics System (NAOS)Near‐Infrared Imager and Spectrograph (CONICA)

SINFONI Spectrograph for INtegral Field Observations in the Near Infrared

UVES Ultraviolet and Visual Echelle Spectrograph

VIMOS VIsible MultiObject Spectrograph

VISIR VLT Imager and Spectrometer for mid Infrared

X‐SHOOTER a multi wavelength (300‐2500nm) medium resolution spectrograph

VLTI Very Large Telescope Interferometer

VLTI MIDI MID‐infrared Interferometric instrument

VLTI AMBER Astronomical Multi‐BEam combineR

VISTA Visible and Infrared Survey Telescope for Astronomy

MASCOT Mini All‐Sky Cloud Observation Tool

NTT      ESO 3.6m      MPG/ESO 2.2mWFI Wide Field Imager

FEROS The Fiber‐fed Extended Range Optical Spectrograph

EFOSC2 ESO Faint Object Spectrograph and Camera

HARPS High Accuracy Radial velocity Planet Searcher

SOFI Son of ISAAC

ALMA The Atacama Large Millimeter/submillimeter ArrayAPEX Atacama Pathfinder Experiment telescope

VST VLT Survey Telescope

E‐ELT European Extremely Large Telescope

“Multi-mission” Concept in Archival

Images Spectra Calibration and Quality Information

Data products managed in the ESO Data Centre and Archive Facilities

Leading observatories and state-of-the-art instrumentation

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 7

ESO Archive System 2009

Design and construction of the ESO Data Centre

Requirements:• Construction at different physical location• On an area of 7 offices in an existing building• Modular approach: Dismantle, move and rebuildthe Data Centre at ESO premises.

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 9

Archive Holdings Past and Projected

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 10

Paranal and the ESOVery Large Telescope

La Silla

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 11

VISTA-survey-telescope

Credit: Steven Beard/UKATC/ESO

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 12

Credit: ALMA

ALMA

~ 1 rack/yr

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 13

E-ELT 17 PB/yr ?

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 14

Lessons Learnt

Preconditioning: physical constraints of data flow a.o.

domain practices bias archive solutions.

Physical installation: in-house, out-source (all or part)?

Industry support: risks, costs, maintenance continuity.

Engineering changes: return on investment, cost of change,

operational costs.

Art-of-balance: especially when a facility serves multiple

purposes, e.g. Operations, Preservation, Exploitation.

Interoperability: many parallel efforts are ongoing.

Long-term view and inter-disciplinary initiatives like the PV-Series critical.

EGLITIS and SUCHAR | PV2009 ESAC | 03.12.2009 | Slide 15

ESO Archive Future Evolution“History of effective response to rapid changes has led to system heterogeneity”

Homogenisation of metadata and interfaces

Development of new HMIs

Standards compliance

Converge on next storage technology:

Analyse current usage of disk-based system in detail

Assessment of growth and access requirements

End-User, Operations, Maintenance and Development

requirements

The ESO Science Archive is a complex part of the complete operational Data Flow System at ESO.

Thank you

Acknowledgement: observatory images courtesy of the ESO Education and Public Outreach Department. See http://www.eso.org/gallery