Embed Size (px)
Citation preview
Presented at:
APAN44 – HPC Session
Presented by:
Chianne Chong
Strategic Policy & Planning
August 2017
Agenda
• Overview of NSCC
• Strategic Projects
• Outreach & Talent Development
• Next Phase of NSCC
• Discussion
2
Introduction to NSCC
4
National Petascale Facility
The National Supercomputing Centre Singapore is established to
support the high performance science and engineering computing
needs for academic, research and industry communities in Singapore
Our facility is linked by high bandwidth multi-gigabit networks to
provide high speed access to users everywhere locally & globally
4
Our Stakeholders
Introduction: Vision & Objectives
5
Vision of NSCC
Support
Singapore’s
R&D
Initiatives
1 Attract
Industrial
Research
Collaborations
2
Objectives of NSCC
Enhance
Singapore’s
Research
Capabilities
3
Democratising Access to Supercomputing
Milestones / Tender Process
6
Tender awarded to Fujitsu Asia Pte Ltd in June 2015 for: • PetaScale Supercomputer; • Data Centre retrofitting; and • Managed Services (3 years)
Nov 14 Jan-Jun 15
Q3 16
Govt Funding &
Co-funding
(A*STAR, NUS,
NTU & SUTD)
Tender Opened,
Closed & Awarded
Unveiling of
Supercomputer
ASPIRE 1
αβ testing
Q4 16
NSCC Data Centre @ Fusionopolis
Visit us @
One-North,
Singapore
ASPIRE 1 HPC Hardware
8
EDR Interconnect
• EDR (100Gbps) Fat
Tree within cluster
• InfiniBand
connection to remote
login nodes at
stakeholder campuses
13PB Storage
• I/O bandwidth up to
500GB/s
• GPFS and Lustre
File System
1 PFLOP System
• 1,288 nodes (dual socket,
12 cores/CPU E5-2690v3)
• 31,320 cores
• 128 GB DDR4 RAM/node
• 11 Large memory nodes
(1x6TB, 4x2TB,
6x1TB)
Accelerator nodes
• 128 nodes with GPUs
• 1 x Tesla K40 per node
Visualisation nodes
• 2 nodes R940 graphic workstations
• Each with 2 x NVIDIA Quadro K4200
• NVIDIA Quadro Sync support
Supporting up to 274,363,200 core-hours per year
Direct streaming of Sequence Data from GIS to Supercomputer in NSCC
NGSP Sequencers at B2
(Illumina + PacBio)
NSCC
Gateway
STEP 2: Automated
pipeline analysis once
sequencing completes.
Processed data resides
in NSCC
500Gbps
B/W
between BP
and FP
Data Manager
STEP 3: Data manager index
and annotates processed
data. Replicate metadata to
GIS. Allowing data to be
search and retrieved from GIS
Compute Tiered Storage
POLARIS, Genotyping &
other Platforms in L4~L8
Tiered Storage
STEP 1: Sequencers
stream directly to
NSCC Storage
(NO footprint in GIS)
Compute
1 Gbps per sequencer 1 Gbps per
machine
100 Gbps
10 Gbps
NSCC
GIS
2km
• NSCC leverages on long-
haul InfiniBand switches
which deliver high-
bandwidth and low-latency
between different
geographic locations
10
High-speed InfiniBand for Data Transfers
A*STAR Login Nodes
NSCC
NUS Login Nodes
NTU Login Nodes
Genome Institute of Singapore
1TB Large Memory Application Node
InfiniBand 40Gbps
• Between stakeholder remote
login nodes/fat nodes and
NSCC HPC centre
• Also for Data Transfer
Nodes overseas InfiniBand 100Gbps
Leveraging on Singapore Advanced Research and Education Network
(SingAREN) and SingAREN-Lightwave Internet Exchange (SLIX) 11
Remote Login Nodes / Fat Nodes
• Remote Login nodes @ NTU & NUS – allow students & staff to utilise NSCC HPC resources as if they were locally at NSCC via high-speed low latency InfiniBand connectivity
• Large memory node @ GIS – serves as direct gateway between GIS’ sequencers and NSCC’s compute and storage facilities
• Connectivity @ SUTD – allow students & staff to NSCC HPC resources via leased circuit
NTU Dark Fibre
2km,
40Gbps
40km,
40Gbps
Leased Circuit
A*STAR @Fusionopolis
A*STAR Biopolis (GIS)
NSCC
NUS
Smart Login via Federated Identity Management (FIM) for Stakeholders
• Leveraging on SingAREN’s Singapore Access Federation
(SGAF) service, NSCC enables seamless access to its
resources for our stakeholders
12
• SGAF uses Shibboleth-based authentication and authorisation framework
• The authentication is performed by the user's home institution (maintains control of its users' information). NSCC grants authorised service access based on specific user attributes.
https://user.nscc.sg
13
Envisaged High-Speed InfiniBand Fabric
Duke-NUS SingHealth Academy
Ministry of Health
Integrated Health Information System
National Cancer Centre Singapore
42km
OUTRAM
Genome Institute of Singapore
BIOPOLIS
30
km
Up to 100Gbps GE
Up to 500Gbps GE
WOODLANDS 10/40/100Gbps InfiniBand/IP
SELETAR
CHANGI
NOVENA
OUTRAM
ONE-NORTH
JURONG
Lee Kong Chian School of Medicine
Tan Tock Seng Hospital
NOVENA SUTD
CHANGI
Rolls Royce
SELETAR
WOODLANDS
Republic Polytechnic
JURONG
Keppel
SingAREN
Global Switch
A*STAR Biopolis
NTU
SUTD
NUS
A*STAR
Fusionopolis
In Partnership with
Regional Network Connectivity (via SingAREN)
to
North America
100G Links
London, EU
Singapore
Los Angeles, USA
100Gbps co-funded with
10Gbps co-funded with
100Gbps TransPAC/Pacific Wave 100Gbps
co-funded with NICT
Tokyo, JP
100Gbps SINET
NSCC STAR-N Co-funded International Links
To connect Singapore to established HPC centres worldwide
To position Singapore as a Strategic Hub in the region
In Partnership with
NSCC Steering Committee
16
Mr Peter Ho
Chairman
Prof Tan Sze Wee
Executive Director,
Science & Engineering
Research Council,
A*STAR, Singapore
Prof Patrick Tan
Professor Duke-NUS,
Deputy Executive
Director,
Biomedical Research
Council,
A*STAR, Singapore
Prof Satoshi Matsuoka
Professor
Tokyo Institute of
Technology, Japan
Prof Thomas Dunning
Professor Emeritus
University of Illinois
Urbana-Champaign, USA
Prof Lai Choy Heng
Professor NUS,
Deputy Director
Centre for Quantum
Technologies,
Singapore
Mr David Gledhill
Managing Director &
Head, Group Tech &
Ops
DBS Bank Singapore
Mr Aziz Merchant
Executive Director
Komtech Singapore
Prof Lam Khin Yong
Chief of Staff
Nanyang Technological
University, Singapore
Mr Quek Gim Pew
Chief Defence Scientist
Ministry of Defence
Singapore
Strategic Projects
• To provide HPC resources for National R&D initiatives
(Research Innovation & Enterprise 2020 / Science &
Technology Research - $19B)
• To identify those which HPC can have maximum impact
Strategic Focus
18
Advanced
Manufacturing and
Engineering (AME)
To develop technological capability that support the
growth & competitiveness of our manufacturing and
engineering sectors
Health and Biomedical
Sciences (HBMS)
To advance human health & wellness, and create economic value for
Singapore and Singaporeans
Services and Digital
Economy (SDE)
To leverage innovation to create economic
opportunities and meet national priorities
Urban Solutions and
Sustainability (USS)
To develop a sustainable & liveable city through
integrated solutions for Singapore and the World
Domain Areas
AI / DEEP LEARNING
Allow companies to improve customer experiences, automate work processes and detecting & deterring security intrusions
DATA ANALYTICS
Enable organisations to explore and gain more and deeper insights into more complex, intelligent
data models.
SCIENTIFIC ANALYSIS Molecular Modeling provides scientists and researchers a powerful tool to design new chemical and materials.
COMPUTATIONAL FINANCE
Perform HPC modelling of market conditions, pricing model, risk models, and contingencies to allow financial institutions to accurately meet real-time goals.
DIGITAL MEDIA PRODUCTION Accelerate rendering with high realism, reduces time to market for producers and increases the quality of production for users.
LIFE SCIENCES Accelerate biomedical discoveries through high performance applications in genomics, so as to improve effectiveness of clinical treatments & personalised medicine.
CLIMATE MODELLING Contribute to atmospheric science & improve weather forecasts accuracy by broadening the range of parameters in the simulations.
MANUFACTURING Enhance modeling, simulation and analysis to speed up the design cycle for a faster time-to-market for new and advanced products.
GenomeAsia 100K consortium
20
• Aim: To sequence 100,000 genomes from various South, North & East Asia populations
• Goal: To accelerate precision medicine and clinical applications for Asian patients
• NTU acts as the host and NSCC provides the HPC resources
• 68TB of genome data from USA and Korea aggregated and hosted on NSCC platform
• Using NSCC HPC resources, total compute time to
process genome data for 100,000 genomes will only take up 1.1% of the 4-year
project, thus providing more time for novel downstream analysis and experiments
Large-Scale Earthquake Simulations
• Leveraging on ASPIRE 1, to revise the original model on Parkfield tremors by incorporating slow earthquakes and to simulate the super shear ruptures, which can break the fault region at the speed of 3000m/s.
• With the use of a numerical model, the tremors in California USA is shown to have an interesting double pattern. The results of this work was published in the Nature Magazine in 2016.
21
CFD Research - Advanced Simulation and Forensic Capabilities
KOMtech has extensively used ASPIRE 1 to optimise the designs of rigs and vessels by tapping on CFD, which is able to numerically capture most environmental loads imposing on a vessel as well as the response of the vessel. Through CFD analysis, platform design can be optimised to achieve best performance and facilitate the product development process.
• Development & Enhancement of Various In-House Designs
• Building Advanced Simulation Capability
• Forensic Investigation
22
Green Data Centre
23
• Development of an interactive 3D visualization of the
thermal profile of high performance server clusters in
NSCC
• Ability to monitor the thermal dynamics of servers and
spot outliers or anomalies
• Eventual aim is to improve the energy efficiency at
NSCC’s data centre
OUTPUT
SIMULATION (NSCC)
Technology Centre for Offshore and Marine Singapore (TCOMS)
24
COUPLED EXPERIMENTAL - NUMERICAL WAVE FLUME TEST
EXPERIMENT (TCOMS)
Summary of NSCC Utilisation Statistics
CPU Utilisation (by organisation)
Number of users
> 2,000 (as of 12 July 2017)
Average % CPU utilisation
~ 90% (as of 12 July 2017)
CPU hours consumed
> 270 M (since commissioning in May 2016)
Industry,
Others
6.69%
GPU hours consumed
> 31M (since commissioning in May 2016)
Storage used
> 7.5PB (since commissioning in May 2016)
• International Collaborations with Supercomputing Centres (RIST, SDSC, ICAIRS, HKU, Compute Canada, ANU/NCI, PAWSEY, JAXA)
Research Organisation for
Information Science &
Technology (20 Jun 2016)
with Dr. Motoi Okuda
(Deputy Head of Kobe Centre,
RIST)
International Collaborations
iCAIRs (20 June 2016)
with Joe Mambretti
University of San Diego (UCSD) (20 Jun 2016)
HKU (3 Aug 2016)
with Prof. Mark Ellisman with Prof. Andy Hor
Japan Aerospace Exploration
Agency Supercomputing
Division (19 Jun 2017)
with Dr. Fujita Naoyuki
(Manager, JAXA Supercomputer
Division)
Outreach
27
Conduct
Regular Training
Workshops
Workshops
• Introductory
• Optimisation Techniques
• Parallel Profiling &
Debugging
• Advanced Job
Management
Local & Overseas
Conference
Participation
Roadshows /
Collaborations /
Industrial
Engagement
NTU
NUS
A*STAR
MOU with Industry (Mar 2016)
Industrial/Government
Engagements
19–22 Jun 2017
Frankfurt,
Germany
13–16 Mar 2017
Singapore
3–4 May 2017
Singapore
SUTD
12–17 Nov 2017
Denver, USA
26 Aug –1 Sep
2017
Dalian, China
26–29 Mar 2018
Singapore
Awards Ceremony at Frankfurt Messe auditorium (21 June 2017)
Talent Development
NTU student team , supported by NSCC, wins Deep Learning Excellence Award for solving the Captcha Challenge and achieving the Highest Degree of Model Accuracy at 6th Annual ISC-HPCAC Student Cluster Competition of International Supercomputing Conference (ISC’17) in Frankfurt Germany
Supercomputing Frontiers 2017
• 13 – 17 March 2017
• About the Conference – Organised by National Supercomputing Centre (NSCC), Supercomputing Frontiers
2017 is Singapore’s annual international conference that provides a platform for thought leaders from both academia and industry to interact and discuss visionary ideas, important global trends and substantial innovations in supercomputing
– We are proud to have brought together over 450 delegates and speakers from over 12 countries to explore global trends and innovations in high performance computing over this 4-day event.
• Keynote Speakers
29
GORDON BELL
Microsoft Research,
USA
ALESSANDRO CURIONI
IBM Research Laboratory,
Switzerland
THOM H. DUNNING, JR.
Northwest Institute for Advanced
Computing, USA
HAOHUAN FU NSCC Wuxi, China
Inaugural SCF & NSCC Awards 2017
• SCF Awards – Singapore Distinguished Service
Award Lim Chuan Poh, A*STAR
– Singapore Visionary Award Dr. Raj Thampuran, A*STAR
• NSCC Awards – Outstanding HPC Scientific Award
Data Storage Institute, A*STAR
– Outstanding HPC Industry Award Keppel Offshore & Marine Technology Centre
– Outstanding HPC Innovation Award Institute of High Performance Computing, A*STAR
– Special Honourable Mention for Best Integrated Vertical HPC for Offshore & Marine
30
31
32 32
Gathering the Best of HPC in Asia
Industry HPC Technology Update
Scientific (SCF Asia) Paper Presentations
Education Student Competition Sharing
Tutorials
Exhibitions
Strategic Showcase Strategic verticals on use of HPC such as biomedical & health, advanced manufacturing & engineering, offshore & marine, etc.
Co-Located HPC Events
Conference on Next Generation Arithmetic (CoNGA)
Asia Research Platform Workshop (ARP)
Asia Pacific Advanced Networks (APAN) 45
NSCC: The Next Phase
Next Phase
34
SCF & NSCC
Awards
Q4 16 Q1 17
Unveiling of
Supercomputer
NSCC 2.0
The next
supercomputer
for Singapore
Next Steps for NSCC
• Apart from just being a Service Provider to having a
Strategic focus (being a catalyst for HPC activities in
Singapore)
• Plans ahead:
– To develop a HPC talent development plan
– To explore and demonstrate new capabilities for industry
– To focus on strategic projects – e.g. areas of genomic &
precision medicine, offshore & marine, aerospace, industrial
processes, modelling & simulation, and big data analytics
– To connect local HPC ecosystem
– To foster international collaborations
