Highlights of the 49th TOP500 List

ISC 2017, Frankfurt,

June19, 2017

Erich Strohmaier

41ST LIST: THE TOP10 # Site Manufacturer Computer Country Cores Rmax

[Pflops] Power [MW]

1 National Supercomputing Center in Wuxi NRCPC

Sunway TaihuLight NRCPC Sunway SW26010,

260C 1.45GHz China 10,649,600 93.0 15.4

2 National University of Defense Technology NUDT

Tianhe-2 NUDT TH-IVB-FEP,

Xeon 12C 2.2GHz, IntelXeon Phi China 3,120,000 33.9 17.8

3 Swiss National Supercomputing Centre (CSCS) Cray

Piz Daint Cray XC50,

Xeon E5 12C 2.6GHz, Aries, NVIDIA Tesla P100 Switzerland 361,760 19.6 2.27

4 Oak Ridge National Laboratory Cray

Titan Cray XK7,

Opteron 16C 2.2GHz, Gemini, NVIDIA K20x USA 560,640 17.6 8.21

5 Lawrence Livermore National Laboratory IBM

Sequoia BlueGene/Q,

Power BQC 16C 1.6GHz, Custom USA 1,572,864 17.2 7.89

6 Lawrence Berkeley National Laboratory Cray

Cori Cray XC40,

Intel Xeons Phi 7250 68C 1.4 GHz, Aries USA 622,336 14.0 3.94

7 JCAHPC Joint Center for Advanced HPC Fujitsu

Oakforest-PACS PRIMERGY CX1640 M1,

Intel Xeons Phi 7250 68C 1.4 GHz, OmniPath Japan 556,104 13.6 2.72

8 RIKEN Advanced Institute for Computational Science Fujitsu

K Computer SPARC64 VIIIfx 2.0GHz,

Tofu Interconnect Japan 795,024 10.5 12.7

9 Argonne National Laboratory IBM

Mira BlueGene/Q,

Power BQC 16C 1.6GHz, Custom USA 786,432 8.59 3.95

10 Los Alamos NL / Sandia NL Cray

Trinity Cray XC40,

Xeon E5 16C 2.3GHz, Aries USA 301,0564 8.10 4.23

•  SW26010 processor (Chinese design, ISA, & fab) •  1.45 GHz •  Node = 260 Cores (1 socket)

–  4 – core groups –  32 GB memory

•  40,960 nodes in the system •  10,649,600 cores total •  1.31 PB of primary memory (DDR3). •  125.4 Pflop/s theoretical peak •  93 Pflop/s HPL, 74% peak •  15.3 Mwatts water cooled •  3 of the 6 finalists for

Gordon Bell Award@SC16

SUNWAY TAIHULIGHT

0

2

4

6

8

101993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

2017

TOP10 REPLACEMENT RATE

109

0

50

100

150

200

250

300

3501993

1994

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

2017

REPLACEMENT RATE

AVERAGE SYSTEM AGE

0

5

10

15

20

25

1995

1996

1997

1998

1999

2000

2001

2002

2003

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

2017

Age[M

onths]

7.6month

PERFORMANCE FRACTION OF THE TOP5 SYSTEMS

0%2%4%6%8%10%12%14%16%18%

199419961998200020022004200620082010201220142016

1

2

3

4

5

0

20

40

60

80

100

1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016

RANK AT WHICH HALF OF TOTAL PERFORMANCE IS ACCUMULATED

SYSTEM AGE (IN LIST COUNT)

0

100

200

300

400

500

4

3

2

1

0

PERFORMANCE DEVELOPMENT

0.11

10100100010000

100000100000010000000

1000000001E+09

199419961998200020022004200620082010201220142016

59.7GFlop/s

400MFlop/s

1.17TFlop/s

93PFlop/s

432TFlop/s

749PFlop/s

SUM

N=1

N=500 1 Gflop/s

1 Tflop/s

100 Mflop/s

100 Gflop/s

100 Tflop/s

10 Gflop/s

10 Tflop/s

1 Pflop/s

100 Pflop/s

10 Pflop/s

1 Eflop/s

PROJECTED PERFORMANCE DEVELOPMENT

0.1

10

1000

100000

10000000

1E+09

1E+11

19941996199820002002200420062008201020122014201620182020

SUM

N=1

N=500

1 Gflop/s

1 Tflop/s

100 Mflop/s

100 Gflop/s

100 Tflop/s

10 Gflop/s

10 Tflop/s

1 Pflop/s

100 Pflop/s

10 Pflop/s

1 Eflop/s

PERFORMANCE DEVELOPMENT

0.11

10100100010000

100000100000010000000

1000000001E+09

199419961998200020022004200620082010201220142016

59.7GFlop/s

400MFlop/s

1.17TFlop/s

93PFlop/s

432TFlop/s

749PFlop/s

SUM

N=1

N=500 1 Gflop/s

1 Tflop/s

100 Mflop/s

100 Gflop/s

100 Tflop/s

10 Gflop/s

10 Tflop/s

1 Pflop/s

100 Pflop/s

10 Pflop/s

1 Eflop/s

June2008

PROJECTED PERFORMANCE DEVELOPMENT

0.1

10

1000

100000

10000000

1E+09

1E+11

19941996199820002002200420062008201020122014201620182020

SUM

N=1

N=500

1 Gflop/s

1 Tflop/s

100 Mflop/s

100 Gflop/s

100 Tflop/s

10 Gflop/s

10 Tflop/s

1 Pflop/s

100 Pflop/s

10 Pflop/s

1 Eflop/s

PERFORMANCE DEVELOPMENT

0.11

10100100010000

100000100000010000000

1000000001E+091E+10

1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016

59.7GFlop/s

400MFlop/s

1.17TFlop/s

93PFlop/s

432TFlop/s

749PFlop/s

SUM

N=1

N=500

1 Gflop/s

1 Tflop/s

100 Mflop/s

100 Gflop/s

100 Tflop/s

10 Gflop/s

10 Tflop/s

1 Pflop/s

100 Pflop/s

10 Pflop/s

1 Eflop/s

June2008

June2013

PROJECTED PERFORMANCE DEVELOPMENT

0.1

10

1000

100000

10000000

1E+09

19941996199820002002200420062008201020122014201620182020

SUM

N=1

N=500

1 Gflop/s

1 Tflop/s

100 Mflop/s

100 Gflop/s

100 Tflop/s

10 Gflop/s

10 Tflop/s

1 Pflop/s

100 Pflop/s

10 Pflop/s

1 Eflop/s

ANNUAL PERFORMANCE INCREASE OF THE TOP500

1

1.5

2

2.5

3

1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 2014 2016

−Moore’sLaw

−TOP500Trend

UnitedStates,34%

China,32%

Japan,6%

Germany,6%

France,3%

UnitedKingdom,3% Italy,2%

Korea,South,2% Others,12%UnitedStates

China

Japan

Germany

France

UnitedKingdom

Italy

Korea,South

COUNTRIES / SYSTEM SHARE

UnitedStates,52%

Japan,10%

Germany,8%

UnitedKingdom,6%

China,5%France,4%

Canada,1%Italy,1%

Korea,South,1% others,12% UnitedStates

Japan

Germany

UnitedKingdom

China

France

Canada

Italy

COUNTRIES / HISTORICAL SHARE

0

100

200

300

400

5001993

1995

1997

1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

China

Korea,South

Italy

Canada

France

UnitedKingdom

Germany

Japan

UnitedStates

COUNTRIES

0

100

200

300

400

5001993

1995

1997

1999

2001

2003

2005

2007

2009

2011

2013

2015

2017

India

Taiwan

Australia

Russia

China

Europe

Japan

USA

PRODUCERS

0

1

10

100

1,000

10,000

100,000

2000

2002

2004

2006

2008

2010

2012

2014

2016

TotalPerform

ance[Tfl

op/s] US

EUJapanChina

PERFORMANCE OF COUNTRIES

HPE,143,29%

Lenovo,88,18%

CrayInc.,57,11%

Sugon,46,9%

IBM,30,6%

Inspur,20,4%Huawei,19,4%

Bull,17,3%Dell,15,3%

Fujitsu,11,2%

PenguinC.,10,2%

Others,44,9% HPE

Lenovo

CrayInc.

Sugon

IBM

Inspur

Huawei

Bull

VENDORS / SYSTEM SHARE

#ofsystems,%of500

HPE,124,17%

Lenovo,74,10%

CrayInc.,160,21%

Sugon,30,4%IBM,60,8%Inspur,14,2%Huawei,13,2%Bull,24,3%

Dell,25,3%Fujitsu,38,5%

PenguinC.,14,2%

NRCPC,94,13%

others,78,10% HPE

Lenovo

CrayInc.

Sugon

IBM

Inspur

Huawei

Bull

VENDORS / PERFORMANCE SHARE

SumofPflop/s,%ofwholelist

Cray,20,40%

IBM,6,12%HPE/SGI,5,10%

Fujitsu,5,10%

Lenovo,2,4%

NUDT,2,4%Dell,2,4%

Bull,2,4%

Penguin,2,4% Others,4,8% Cray

IBM

HPE/SGI

Fujitsu

Lenovo

NUDT

Dell

Bull

VENDORS (TOP50) / SYSTEM SHARE

0102030405060708090

100110

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

2017

System

s

PEZY-SCKepler/PhiXeonPhiMainIntelXeonPhiClearspeedIBMCellATIRadeonNvidiaPascalNvidiaKeplerNvidiaFermi

ACCELERATORS

0

50

100

150

200

250

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

2017

TotalPerform

ance[Pfl

op/s] PEZY-SC

Kepler/Phi

Clearspeed

ATIRadeon

IBMCell

XeonPhiMain

IntelXeonPhi

NvidiaPascal

NvidiaFermi

NvidiaKepler

PERFORMANCE OF ACCELERATORS

PERFORMANCE SHARE OF ACCELERATORS

0%5%10%15%20%25%30%35%40%

2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

FracSo

nofTotalTOP5

00

Performan

ce

•  Both projects worked for several year to unify measurement and reporting approaches (EEHPC-WG: Energy-Efficient HPC Working Group ).

•  Ultimately this lead us to combine data collection and curation in one site and system.

•  Both lists will continue to be published at the same time (ISC and SC). •  We are working on combining past data-sets and sites. •  Both sites will be hosted and maintained by the ISC Group.

TOP500 - GREEN500

Computer Rmax/Power

Tsubame 3.0, SGI ICE XA, Xeon14C2.4GHz IntelOmni-Path TeslaP100SXM2 14.11

kukai, ZettaScaler-1.6 GPGPU System Xeon14C1.7GHz InfinibandFDR TeslaP100 14.05

AIST AI Cloud, NEC 4U-8GPU Xeon10C1.8GHz InfinibandEDR TeslaP100SXM2 12.68

RAIDEN GPU subsystem, NVIDIA DGX-1 Xeon20C2.2GHz InfinibandEDR TeslaP100 10.60

Wilkes-2, Dell C4130 Xeon12C2.2GHz InfinibandEDR TeslaP100 10.43

Piz Daint, Cray XC50 Xeon12C2.6GHz Ariesinterconnect TeslaP100 10.40*

Gyoukou, ZettaScaler-2.0 HPC system Xeon16C1.3GHz InfinibandEDR PEZY-SC2 10.22**

RCF2, SGI Rackable C1104-GP1 Xeon12C2.2GHz InfinibandEDR TeslaP100 9.80

NVIDIA DGX-1/Relion 2904GT Xeon20C2.2GHz InfinibandEDR TeslaP100/QuadroGP100 9.46

DGX SaturnV, DGX-1 Xeon20C2.2GHz InfinibandEDR TeslaP100 9.46

MOST ENERGY EFFICIENT ARCHITECTURES

[Gflops/WaU]**Systemswithderivedpower*Poweropemizedraeo:HPL-13%;Power-28%

POWER CONSUMPTION

0123456789

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Power[M

W]

TOP10

TOP50

TOP5002.7xin5y

2.6xin5y

3.0xin5y

POWER CONSUMPTION

0123456789

101112

Power[M

W]

TOP10

TOP50

TOP500

POWER EFFICIENCY

0500

1,0001,5002,0002,5003,0003,5004,000

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Linp

ack/Po

wer[Gflo

ps/kW]

TOP10

TOP50

TOP500

ENERGY EFFICIENCY

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Linp

ack/Po

wer[Gflo

ps/kW]

TOP10TOP50

TOP500

Max-Efficiency Tsubame3.0

BlueGene/QCell

MicAMDFirePro

TsubameKFCNVIDIAK20x–K80

ZeUaScaler-1.6c

DGXSaturnV

ENERGY EFFICIENCY

0

2,000

4,000

6,000

8,000

10,000

12,000

14,000

16,000

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017

Linp

ack/Po

wer[Gflo

ps/kW]

TOP500Average

Max-Efficiency Tsubame3.0

BlueGene/QCell

MicAMDFirePro

TsubameKFCNVIDIAK20x–K80

ZeUaScaler-1.6c

DGXSaturnV

•  Longstanding interest to augment HPL with other benchmarks.

•  Starting to publish HPCG numbers with this list. •  Submission still go to Jack and Mike first. •  ~109 measured system •  47 HPCG entries which made the TOP500

(not necessarily the top47 HPCG measurements!). •  Adding ability to resort and filter to our web-lists. •  Top10 …

TOP500 - HPCG

41ST LIST: THE TOP10 # T Site Manufacturer Computer Country HPCG

[Pflop/s] Rmax [Pflop/s]

HPCG/ Peak

HPCG/ HPL

1 8 RIKEN Advanced Institute for Computational Science Fujitsu

K Computer SPARC64 VIIIfx 2.0GHz,

Tofu Interconnect Japan 0.6027 10.5 5.3% 5.7%

2 2 National University of Defense Technology NUDT

Tianhe-2 NUDT TH-IVB-FEP,

Xeon 12C 2.2GHz, IntelXeon Phi China 0.5801 33.9 1.1% 1.7%

3 3 Swiss National Supercomputing Centre (CSCS) Cray

Piz Daint Cray XC50,

Xeon E5 12C 2.6GHz, Aries, NVIDIA Tesla P100 Switzerland 0.4700 19.6 1.9% 2.4%

4 7 JCAHPC Joint Center for Advanced HPC Fujitsu

Oakforest-PACS PRIMERGY CX1640 M1,

Intel Xeons Phi 7250 68C 1.4 GHz, OmniPath Japan 0.3855 13.6 1.5% 2.8%

5 1 National Supercomputing Center in Wuxi NRCPC

Sunway TaihuLight NRCPC Sunway SW26010,

260C 1.45GHz China 0.3712 93.0 0.3% 0.4%

6 6 Lawrence Berkeley National Laboratory Cray

Cori Cray XC40,

Intel Xeons Phi 7250 68C 1.4 GHz, Aries USA 0.3554 14.0 1.3% 2.5%

7 5 Lawrence Livermore National Laboratory IBM

Sequoia BlueGene/Q,

Power BQC 16C 1.6GHz, Custom USA 0.3304 17.2 1.6% 1.9%

8 4 Oak Ridge National Laboratory Cray

Titan Cray XK7,

Opteron 16C 2.2GHz, Gemini, NVIDIA K20x USA 0.3223 17.6 1.2% 1.8%

9 10 Los Alamos NL / Sandia NL Cray

Trinity Cray XC40,

Xeon E5 16C 2.3GHz, Aries USA 0.1826 8.10 1.6% 2.3%

10 15 NASA/ Ames Research Center/NAS HPE

Pleiades SGI ICE X,

Xeon E5 10C 2.4-2.8GHz, Infiniband FDR USA 0.1750 5.95 2.5% 2.9%

11 9 Argonne National Laboratory IBM

Mira BlueGene/Q,

Power BQC 16C 1.6GHz, Custom USA 0.1670 8.59 1.7% 1.9%

•  BoF 14: The Green500: Trends for Energy-Efficient Supercomputing –  Tuesday: 03:45 pm – 04:45 pm –  Substance 1+2  –  Wu Feng, Virginia Tech

•  BoF 19: HPCG Benchmarks Updates

–  Wednesday: 11:30 am – 12:30 pm –  Substance 1+2 –  Michael Heroux, Sandia

MORE INFO ABOUT THESE TOPICS AT ISC