Overview of the SKA Project and its Current StatusMiles Deegan, SKA Organisation, Jodrell Bank, UK

ISC17, 19th June 2017

2

Advert

• Immediately after this session:• SKA the Ultimate Big Data Machine, BOF 4• Nicolas Erdody and Happy Sithole are leading this • Starts @17:00

3

Summary

• Introductions• The Project• The Science• SKA Computing• SKA Regional Centres• Conclusions

4

Introductions: SKA people

• Miles Deegan, HPC/HPDA Domain Specialist at SKA HQ• John Taylor, University of Cambridge (UK), SDP

Consortium• Chris Broekema, ASTRON (NL), SDP Consortium• Other SDP colleagues in the room

5

Square Kilometre Array3 sites; 2 telescopes + HQ

1 Observatory

Design Phase: late 2013 – 2018, €200M; 600 scientists+engineers

Phase 1Construction: 2018/9 – 2024

Construction cost cap: €674.1M (inflation-adjusted)Operations cost: under development (power is a big part)

MeerKat integratedObservatory Development Programme (€20M/year planned)SKA Regional centres out of scope of centrally-funded SKAO.

Phase 2: start mid-2020s ~2000 dishes across 3500km of Southern Africa

Major expansion of SKA1-Low across Western Australia + further telescopes after that

6

Telescopes in AU & SA – remote!

• SKA1-LOW: 50 – 350 MHz• Phase 1: ~130,000 antennas • across 65km

SKA1-Mid: 350 MHz – 24 GHzPhase 1: 200 15-m dishes across 150 km

Construction: 2018 – 2024; Cost cap: €675M

International Design Teams• Project Management, System Engineering and Architecture based at Jodrell Bank, UK - SKA

Office is the Design Authority, will lead procurement and then the operational phase• >600 scientists & engineers in institutes and industry in Member countries of the SKA

Participating Organisations

9

SKA Organisation: 10 countries, more to join

Australia (DoI&S)Canada (NRC-HIA)China (MOST)India (DAE)Italy (INAF)Netherlands (NWO)New Zealand (MED)South Africa (DST)Sweden (Chalmers)UK (STFC)

Interested Countries:• France• Germany• Japan• Korea• Malta• Portugal• Spain• Switzerland• USA

Contacts:• Mexico• Brazil• Ireland• Russia

10

• Portugal: Just joined; funding for computing design work

• Germany: • Industry engagement, prototyping dishes; Future governance work engagement• Reports in the press of (re)joining as an Associate Member

• France: Accelerated re-examination of Astrophysics section of National Science Infrastructure Roadmap. Engagement with industrial partners, e.g. Bull/Atos

• Spain: Spanish State Secretary has met the D-G, supportive of joining SKA in near future.

• Switzerland: Swiss State Secretary requested observer status at SKA Board - granted; Recent meetings in Lausanne re. computing; indication will join when an IGO.

• Japan and Korea: Attending SKA Board Meetings; visiting scientists

• USA: Establishing radio astronomy strategy for Astro2020. Ongoing discussions NRAO/NSF community and potentially DoE

• Why all this interest?• Primarily the science, but ICT innovation is a big driver

• More participants = Higher Budget? Possibly

11

Future SKA governance structure• SKA Org is currently a not-for-profit limited by guarantee UK company• We aim to transition to being an Inter-Governmental Organisation• IGO = ‘Convention’ agreed between governments

• Government commitment: Long-term political stability, funding stability• A level of independence in structure• Availability of ‘supporting processes’ through Privileges and Immunities from

members: functional support for project• ‘Freedom to operate’, specifically through procurement process, employment

rules and so on

New SKA HQ building: work underway

15

£16.5M investmentCompletion in June 2018

16

17

Overall project timeline – to be confirmed

Key dates:• Convention signing July 2017 -• CDRs Q4 2017 – Q2 2018

• IGO enter into force with 5 members• SKA1 Construction approval early 2019

18

Service Level AgreementsMemorandum of Understanding

In Scope

Operations Scope

19

The Science

20

Cosmic Dawn(First Stars and Galaxies)

Galaxy Evolution(Normal Galaxies z~2-3)

Cosmology(Dark Energy, Large Scale Structure)

Cosmic Magnetism(Origin, Evolution)

Cradle of Life(Planets, Molecules, SETI)

Testing General Relativity(Strong Regime, Gravitational Waves)

Exploration of the Unknown e.g. FRBs

Extremely broad range of science!

21

Epoch of Reionization

22

SKA Computing and Software

23

System Overview

8.8 Tbits/s

7.2 Tbits/s

2x 5 Tbits/s

~50 PFlop ~250 PFlop

300 PB/yr

SKA Regional Centres

2Pbits/s

24

• SKA is a software telescope• Very flexible and potentially easy to reconfigure• Major software development, maintenance and ongoing

improvement challenge for 50 years

• HPC piece – Science Data Processor• SDP needs ~25 PetaFLOPS sustained over a scheduling period• Current estimate is that SDP needs 250 PFLOP/sec peak

• Memory bandwidth is ~200 PetaBytes/sec• Low arithmetic intensity• Roofline models, detailed parametric modelling and prototyping to

confirm• Real-time as well as batch, complex buffering, all within a live

running experiment• Watts/Gflop ~40x better than Tianhe-2

• + Pulsar Search (CSP) is an additional 50 PFLOP/s of peak processing

25

A Pre-SKA Computing Challenge: Precursors (and pathfinders)

26

Software

• Budget of ~€90M on manpower for software development across the whole telescope.

• Need professional practices for development, testing, integration and deployment.

• Need to unify the processes across the world-wide team of developers.

• Need world-leading expertise in a number of areas.• Engaged SEI (CMU)

• Delivered system will not be static• SDP hardware and software will be updated regularly• Key input for development will be the scientific and software

community through the regional centres (more later)

27

SKA Regional Centres (SRCs)

28

SRC Overview

• SKA Board agreed the principle of SRCs• Not part of SKA, or funded by SKA, but:

• Essential to get most out of the science products produced by SDP

• Coordinated with assistance from SKAO and accredited with SKAO

• Requirements analysis, technical work underway – SKA Regional Centre Coordination Group (SRCCG)

• Personnel from member countries plus the likes of CERN and LSST

29

Regional Centre Concept:• ~300 PB/annum

• Might be more

Science Processing

Centre

Science Processing

Centre

Regional Centre 1

Regional Centre 2

Regional Centre N

Users

Users

Users

Users

Users

Users

Users

RSA

AUS

Observatory User CommunitySR

C Sc

ienc

e G

atew

ay

30

Regional Centre Network• 10 year IRU per 100Gbit circuit early 2020s onwards• Guesstimate of Regional Centre locations

US$1-3M/Year

US$1-3M/Year

US$1-3M/Year

US$.2-.5M/Year

US$.2-.5M/Year

US$.1-.2M/YearUS$.1M/Year

US$.1M/Year

US$.2-.5M/Year

v

US$2-3M/Year

US$1-2M/Year

US$1-2M/Year

US$.2-.5M/Year

US$.2-.5M/Year

US$.1-.2M/YearUS$.1M/Year

US$.1M/Year

US$.2-.5M/Year

Major NREN Paths – SKA relevant

Fibre Cable Systems – SKA relevant

US$1M

/Year

31

What will SRCs do?• Provide a nexus for resources in a member country or region

• Scientific expertise and software expertise• Access to computing resources to further process SDP products - but direct

ownership is not a requirement• How much compute and storage? Plenty• Developments so far: EU H2020, AENEAS and EOSC driver; Sino-Australia

collaboration• Provide support for scientific development

• Provide future, subject-specific pipelines• Provide access to data

• Ensure security and adherence to SKA data policies• Play a defined role in hosting and distributing archives• Perhaps process precursor and pathfinder data

• Provide local (time zone) user support, proposal access, information, training and outreach activities

• Liaison with SKA Observatory and NRENs• Need to ensure sufficient and affordable network capacity is procured and

provisioned in a timely fashion• A lot of things in common with the CERN LHC computing grid

• CERN upgrade will see data volumes comparable to SKA phase 1• Starting collaboration with colleagues in areas such as OpenStack and data

management

32

Construction, Operations and €s• Budgets not tied down but at least 60M Euros on SDP HW;

40M Euros on SDP software development• Ongoing operational costs and development (new algos etc) being

worked on• Construction late 2018/2019 onwards for ~ 5 years• Telescopes built out in a series of array releases with

computing needs scaling over time• Refresh rate of SKA HW?

• TBD• SRCs – size and funding

• Again TBD• But expect in some parts of the world nascent SRCs in early 2020s• Shared infrastructure with other communities and SKA countries

with the region

33

Conclusions• SKA will be the world’s primary radio telescope in the metre

and centimetre bands and be one of the largest science projects of the 21st Century

• It is a huge computing and software challenge• “Traditional HPC” is not a good match because the problem

is I/O and memory bandwidth dominated with real-time as well as batch computing and complex buffering and data transfer problems

• RFPs will not procure HPL machines• Exascale development programmes must address data-intensive

science as well as the more traditional HPC disciplines• Keen to learn more this week and at SC17• And seek co-design opportunities

• Software complexity is also beyond what has been achieved previously, certainly in astronomy

• Quality is paramount and good processes essential

Thank You for your timeQuestions?