Paolo Padovani, May 23, 2007, e-Science in Chile - 1 e-Astronomy: the Virtual Observatory and its implementation Paolo Padovani Head, Virtual Observatory

Paolo Padovani, May 23, 2007, e-Science in Chile - 1

e-Astronomy: the Virtual Observatory and its implementation

Paolo Padovani

Head, Virtual Observatory Systems Department, ESO, Garching bei München, Germany

&EURO-VO Facility Centre Scientist

• Astronomy in a nutshell

• e-Astronomy: the Virtual Observatory

• The Virtual Observatory world-wide, in Europe, and at ESO


The Ultimate Goal


The way Astronomy works

• Telescopes (ground- and space-based, covering the full electromagnetic spectrum) Observatories

• Instruments (telescope/band dependent) Observatories/Consortia

• Publications Journals

• Data curation (metadata + tables & catalogues) Data curators

• Data analysis software (instrument dependent) Observatories/Consortia/Researchers

• Active Archives Observatories/Agencies


… and Public Outreach


Astronomy from an e-Science Perspective: the Good News

• Observational data normally stored in digital repositories (“astronomical archives”), freely available on-line after < 1 year

• Results published in academic journals, all available on-line Table of contents and abstracts freely available Full content in general freely available after 3 years Some tables immediately available through data curators

• One single entry point for journals: ADS • Two-way links between most archives and publications• Data curators (object metadata + catalogues) on-line; some links asawith archives and publications • Analysis software available from Observatories/Archives on-line• Press release and outreach material (pictures, movies) on-line


Astronomy from an e-Science Perspective: the “less Good” News

• Different astronomical archives have widely different access/search interfaces and standards/conventions; serving mainly raw data

• Widely specialized, complex analysis software for various sub-branches; steep learning curve, but multi-wavelength now the norm

• Publication - Archive links point to raw, unprocessed data • Object metadata not homogeneously defined; links with archives

and publications not complete • Press release and outreach material disconnected• Huge data collections: download and data analysis on desktop

problematic/impossible (TB dataset: ~ 1 week at 10 Mbps)• Information avalanche!


Telescope Collective Area Increase

Mountain & Gillett 1998


ESO Archive Growth


Astronomical Publication Growth

White 2007


Instruments

Data Headers

Telescopecontrol

Logs

Observatories

Outreach Products

Calibrations

Raw Data

Processed Data

AnalysisData

Products

Catalogues

Software

Computing

Publications

Papers

TablesFigurescourtesy of

P. Quinn


The Virtual Observatory

• The Virtual Observatory (VO) is an innovative, still evolving, system to: take advantage of astronomical data explosion allow astronomers to interrogate multiple data centres in a seamless and transparent way and to utilize at best astronomical data permit remote computing and data analysis foster new science

• Web: all documents inside PC; VO: all astronomical databases inside PC • VO democratization of astronomy!• All of the above requires the various players to speak the same language VO standards and protocols are defined and adopted within the International Virtual Observatory Alliance (IVOA), which includes 16 VO projects world-wide


16

ChinaIndia

Canada

Spain

Italy

Armenia

FranceGermany

Hungary

Japan

Korea

USA

Russia

UK

Australia Europe


• Mission: To facilitate the international coordination and collaboration necessary for the development and deployment of the tools, systems and organizational structures necessary to enable the international utilization of astronomical archives as an integrated and interoperating virtual observatory

• Needs: standardization of data and metadata, data exchange methods, and registry (list of available services and what can be done with them)

• Organized in working groups which tackle various aspects (e.g., data access, data modelling, query language, registries, etc.)

• IVOA Executive Board includes representatives from all VO projects• Works by telecons, “TWiki” pages, and bi-annual meetings (last one in

Beijing [May 2007])• VO projects starting to move from development to operational

The IVOA: http://ivoa.net

http://ivoa.net/


VO in EUROPE: the EURO-VO Project http://www.euro-vo.org

• Successor to the Astrophysical Virtual Observatory (AVO), which was a 5 M€, Phase A study (2001 - 2004/5) on the scientific requirements and technology for building the VO in Europe, 50% funded by European Community (Fifth Framework Programme [FP5])• Includes 8 partners: ESO, European Space Agency (ESA), plus six national nodes: INAF (Italy), INSU (France), INTA (Spain), NOVA (Netherlands), PPARC (UK), and RDS (Germany)• Has three components: Data Centre Alliance, Technology Centre, Facility Centre

http://www.euro-vo.org/



An alliance of European data centres who will populate the EURO-VO with data, provide the physical storage and computational fabric and who will publish data, metadata and services to the EURO-VO using VO technologies

A distributed organization that coordinates a set of research and development projects on the advancement of VO technology, systems and tools in response to scientific and community requirements

An operational organization, that provides the EURO-VO with a persistent, centralized registry for resources, standards and certification mechanisms as well as community support for VO technology take-up and scientific programs. EURO-VO’s “public face”


The EURO-VO Project: Components and funding situation

• Data Centre Alliance (DCA) co-funded by the European Community at the 1.5 M€ level (FP6) for 2 years since Sept. 2006; 8.5 FTE/yr. Lead by CDS, Strasbourg. Main activities:

Community Workshop (March 2007 at ESAC) Data Centre Workshop (June 2007 at ESAC)

• VO-TECH co-funded by the European Community at the 3.3 M€ level (FP6) for 3(4) years since Jan. 2005; 21 FTE/yr. Lead by AstroGrid, UK. Main activities:

“Design Studies” meetings every 6 months• Facility Centre (FC), located at ESO, co-managed by ESO & ESA; support at “best-effort” level [~ 2 FTEs] (FP7 proposal May 2 2007). Main activities:

1st EURO-VO technical Workshop (June 2005 at ESO) Selection of EURO-VO Science Advisory Committee; held first two meetings EURO-VO Web pages EURO-VO Research Initiative


• The European Organisation for Astronomical Research in the Southern Hemisphere• Intergovernmental organisation (founded 1962) to 'establish and operate an astronomical observatory in the southern hemisphere, equipped with powerful instruments, with the aim of furthering and organizing collaboration in astronomy…'.• Three observational sites, all in Chile: medium-sized telescopes (La Silla, north of La Serena), Very Large Telescopes [VLT] (Paranal, Antofagasta), APEX (in operation) and ALMA (giant array of 12-m submillimetre antennas under construction) (Llano de Chajnantor, San Pedro de Atacama). • 13 member states + agreement with Chile: 10% of telescope time and an annual fund for the development of astronomy

ESO


ESO and the VO

• ESO operates one of the largest astronomical archives in the world• ESO is committed to become a key provider of data and resources in

the Virtual Observatory: Founding member of IVOA Lead partner of Astrophysical Virtual Observatory Lead partner of EURO-VO

• To this aim, ESO has created (Nov. 2004) the Virtual Observatory Systems Dept. (17 people) to manage ESO’s VO activities but especially with the goal of turning its archive into a powerful scientific resource for the ESO and world-wide community


High-level VO Work on the ESO Archive

1. Build a “VO layer”: “translate” any locally defined parameter to the standard (IVOA

compliant) ones hide any observatory/telescope/instrument specific detail and

work in astronomical units provide the right metadata2. Improve archive interface3. Creation of highly processed (“science-ready”) data: very important

for VO users 4. Collection, validation, and publishing into archive (according to

detailed guidelines) of highly processed community data [paradigm change]

5. Same from Large Programs and upcoming Public Surveys (VST & Vista) [now required]


Summary

• Astronomy has done the groundwork to become e-Astronomy

• Some more work is required to integrate and make interoperable the various components the Virtual Observatory

• Goal: all astronomical databases “one click away” democratization of Astronomy!

• The Virtual Observatory is an international effort. ESO plays a major role in Europe (EURO-VO) and world-wide (IVOA)


Organisation for Economic Co-operation and Development (OECD) comments on the VO

Findings• The Virtual Observatory concept is a bold community-led response to

the challenges the astronomical community faces in data management and storage. Impressive progress has been made and the momentum of the International Virtual Observatory Alliance will ensure sustained progress, provided the agency level support and funding is available.

Recommendations• New projects and facilities must take the data management, storage,

maintenance, and dissemination into account at the earliest planning stages, consulting potential users in the process. Agencies should recognise that this is an important long term issue and should co-ordinate plans, provide adequate funding on a long-term basis, and support development and maintenance of the needed infrastructure. Agencies should encourage the broadening of the existing VO collaboration into a fully representative global activity.

Documents

Paolo Padovani, May 23, 2007, e-Science in Chile - 1 e-Astronomy: the Virtual Observatory and its implementation Paolo Padovani Head, Virtual Observatory