Environmental (Friendly) Supercomputing on SuperMUC · D. Kranzlmüller Cracow, 26 October 2016 5...

1

Environmental(Friendly)SupercomputingonSuperMUC

DieterKranzlmüller

Munich NetworkManagementTeamLudwig-Maximilians-UniversitätMünchen(LMU)&LeibnizSupercomputing Centre (LRZ)of the Bavarian Academy of Sciences and Humanities

FlashFlood Genoa,Italy, 2011

D.Kranzlmüller Cracow,26October2016 2

http://www.drihm.eu/images/video/DRIHM_final.mp4

2

FlashFloods

n Formswiftlydueto(extremely)highrainfallrates

n Littleornopriorwarning

n Devastatingconsequences(casualties,economiclosses,...)

D.Kranzlmüller Cracow,26October2016 3

UNISDR– TheUnitedNationsOfficeforDisasterRiskReduction

D.Kranzlmüller Cracow,26October2016 4

https://www.unisdr.org/

3

GAR– GlobalAssessmentReportonDisasterRiskReduktion 2015

D.Kranzlmüller Cracow,26October2016 5

http://www.preventionweb.net/english/hyogo/gar/2015/en/home/GAR_2015/GAR_2015_6.html

NumberofDisastersperRegion

D.Kranzlmüller Cracow,26October2016 6

http://www.emdat.be/disaster_trends/index.html

4

MunichRe– LossEventsWorldwide2014

D.Kranzlmüller Cracow,26October2016 7

http://www.preventionweb.net/files/41773_munichreworldmapnaturalcatastrophes.pdf

FlashFloods

n Formswiftlydueto(extremely)highrainfallrates

n Littleornopriorwarning

n Devastatingconsequences(casualties,economiclosses,...)

n Monitoringandforecastingoffloods:– EuropeanFloodAwarenessSystem(EFAS)– GlobalFloodDetectionSystem(GFDS)– GlobalFloodAwarenessSystem(GloFAS)

n Problem:spatialresolution50-100kmè Flashfloodsremainundetected

D.Kranzlmüller Cracow,26October2016 8

5

TheEUProjectSeriesDRIHM*

D.Kranzlmüller Cracow,26October2016 9

PossibleSolution– EnvironmentalComputing

n Combinemeteorology,hydrology,hydraulicsthroughcomputerscience

n Increasespatialandtemporalresolution(dataquality)– RegionalClimateModels(RCM)

n Computeensemblesofforecaststocoverallpotentialoutcomes

n Startandfinishcomputationintimetoprovideleadtimeforevacuationmeasures

è Simulateensemblesofforecastswithhigh-resolutionon

high-performancecomputing(HPC)infrastructuresondemandwhentriggeredbyincreasedrainfallrates

D.Kranzlmüller Cracow,26October2016 10

Notinthis talk

6

LeibnizSupercomputingCentreoftheBavarianAcademyofSciencesandHumanities

D.Kranzlmüller Cracow,26October2016 11

Withapprox.230employeesformorethan100.000studentsandformorethan30.000employeesincluding8.500scientists

• EuropeanSupercomputingCentre• National SupercomputingCentre

• RegionalComputerCentreforallBavarianUniversities• ComputerCentreforallMunichUniversities

Photo:ErnstGraf

LeibnizSupercomputingCentreoftheBavarianAcademyofSciencesandHumanities

n EuropeanSupercomputingCentre

n NationalSupercomputingCentre

n RegionalComputerCentreforallBavarianUniversities

n ComputerCentreforallMunichUniversities

D.Kranzlmüller Cracow,26October2016 12

SGI UV

SGI Altix

Linux Clusters

SuperMUC

Linux Hosting and Housing

7

SuperMUC@LRZ

D.Kranzlmüller Cracow,26October2016 13

Video: SuperMUC rendered on SuperMUC by LRZ

http://youtu.be/OlAS6iiqWrQ

Top500SupercomputerList(June2012)

D.Kranzlmüller Cracow,26October2016 14

www.top500.org

8

LRZSupercomputers

D.Kranzlmüller 15

SuperMUC PhaseII

Cracow,26October2016

SuperMUC Phase 1 + 2

D.Kranzlmüller Cracow,26October2016 16

9

SuperMUC System@LRZ

Phase2(LenovoNeXtScale WCT):

• 3.6PFlops peakperformance• 3072LenovoNeXtScale nx360M5WCT

nodesin6computenodeislands• 2IntelXeonE5-2697v3processorsand64

GBofmemorypercomputenode• 86,016computecores• NetworkInfiniband FDR14(fattree)

CommonGPFSfilesystemswith10PBand5PBusablestoragesizerespectivelyCommonprogrammingenvironment

Directwarm-watercooledsystemtechnology

Phase1(IBMSystemxiDataPlex):

• 3.2PFlops peakperformance• 9216IBMiDataPlex dx360M4nodesin18

computenodeislands• 2IntelXeonE5-2680processorsand32

GBofmemorypercomputenode• 147,456computecores• NetworkInfiniband FDR10(fattree)

D.Kranzlmüller Cracow,26October2016 17

PowerConsumptionatLRZ

D.Kranzlmüller Cracow,26October2016 18

0

5.000

10.000

15.000

20.000

25.000

30.000

35.000

Stro

mve

rbra

uch

in M

Wh

10

CoolingSuperMUC

D.Kranzlmüller Cracow,26October2016 19

SuperMUCPhase2@LRZ

Cracow,26October2016

HighEnergyEfficiencyü UsageofIntelXeonE52697v3processorsü Directliquidcooling

- 10%poweradvantageoveraircooledsystem- 25%poweradvantageduetochiller-lesscooling

Photos:TorstenBloth,Lenovo

ü Energy-aware scheduling- 6%poweradvantage- ~40%poweradvantage- Totalannualsavingsof~2Mio.€

forSuperMUCPhase1and2D.Kranzlmüller 20Slide:HerbertHuber

11

LRZ Application Mix

n Computational Fluid Dynamics: Optimisation of turbines andwings, noise reduction, air conditioning in trains

n Fusion: Plasma in a future fusion reactor (ITER)n Astrophysics: Origin and evolution of stars and galaxiesn Solid State Physics: Superconductivity, surface propertiesn Geophysics: Earth quake scenariosn Material Science: Semiconductorsn Chemistry: Catalytic reactionsn Medicine and Medical Engineering: Blood flow, aneurysms, air

conditioning of operating theatresn Biophysics: Properties of viruses, genome analysisn Climate research: Currents in oceans

D.Kranzlmüller Cracow,26October2016 21

Results (Sustained TFlop/s on 128000 cores)

D.Kranzlmüller Cracow,26October2016 22

Name MPI #cores Description TFlop/s/island TFlop/smaxLinpack IBM 128000 TOP500 161 2560Vertex IBM 128000 PlasmaPhysics 15 245GROMACS IBM,Intel 64000 MolecularModelling 40 110Seissol IBM 64000 Geophysics 31 95waLBerla IBM 128000 LatticeBoltzmann 5.6 90LAMMPS IBM 128000 MolecularModelling 5.6 90APES IBM 64000 CFD 6 47BQCD Intel 128000 QuantumPhysics 10 27

12

PartnershipInitiativeComputationalSciencesπCS

n Individualizedservicesforselectedscientificgroups– flagshiprole– Dedicatedpoint-of-contact– Individualsupportandguidanceandtargetedtraining &education– PlanningdependabilityforusecasespecificoptimizedITinfrastructures– EarlyaccesstolatestITinfrastructure(hard- andsoftware)developments

andspecificationoffuturerequirements– AccesstoITcompetencenetworkandexpertiseatCSandMath

departmentsn Partnercontribution

– EmbeddingITexpertsinusergroups– Jointresearchprojects(includingfunding)– Scientificpartnership– equalfooting– jointpublications

n LRZbenefits– Understandingthe(currentandfuture)needsandrequirementsofthe

respectivescientificdomain– Developingfutureservicesforallusergroups– Thematicfocusing:EnvironmentalComputing

D. Kranzlmüller Cracow,26October2016 23

SeisSol - Numerical Simulationof Seismic WavePhenomena

D.Kranzlmüller Cracow,26October2016 24

Picture:AlexBreuer(TUM)/ChristianPelties(LMU)

Dr.ChristianPelties,Departmentof Earthand EnvironmentalSciences (LMU)Prof.MichaelBader,Departmentof Informatics (TUM)

1,42Petaflop/son147.456Coresof SuperMUC(44,5%of PeakPerformance)

http://www.uni-muenchen.de/informationen_fuer/presse/presseinformationen/2014/pelties_seisol.html

13

Conclusions

n EnvironmentalComputingneedsIT-Infrastructures(includingHPC)

n EnergyEfficiency isanimportantparttomaximizescientificthroughput

n Computationalscienceneedstobeanintegralpartofteachingdomainscientists– LearnhowtogetaccesstoHPCinfrastructures– LearnhowtoprogramHPCinfrastructureswithincreasingcomplexity,

heterogeneityandscalability– efficiency,reliability,portabiliy

n TheLRZPartnershipInitiativeComputationalScience(piCS)triestoimproveusersupport

http://www.sciencedirect.com/science/article/pii/S1877050914003433

D.Kranzlmüller Cracow,26October2016 25

TheEUCopernicusEarthObservationPlatform

n TheCopernicusSentinel Missions are „game changers“for EarthObservation:watch the heartbeat of the planet

D.Kranzlmüller Cracow,26October 2016 26

Sentinel 2Sentinel 1

10m,every 2daysca.3PByte permonth

raw data

10m,every 2..5daysca.4.5PByte permonth

raw data

SlidecourtesyWolframMauser

14

Complex ImageAnalysisderivesEnvironmentalParameters

n Example:quantitativesatellite image analysis of wheat fields

D.Kranzlmüller Cracow,26October2016 27

Chlorophyllcontent [µg/m²]Satellite image

Parameters,e.g.:Plantspecies,biomass,chlorophyll,pests,phenology,…

SlidecourtesyWolframMauser

EnvironmentalComputing

n Human-Environment-Relation- ObservationsandSimulationsforalternativeGlobalFutures

n Massivecomputingresourcesareneededtocreateacyber-environmentalsystem inwhichtherealandthevirtualworldcaninteract:– toturnremotesensingimagedatastreamsintomeaningfulenvironmental

informationforeachfarmerontheglobe– toidentifyleastinvasivewaysforagriculture– todoublefoodproductionandshowthe

globalenvironmentalbenefitsithas– tosimulateandassessthetotal

environmentandthehumaninterventionsbeforetheyoccur

– toexplorealternativefutureenvironmentsandtheirsustainabilityandqualityoflife

D.Kranzlmüller Cracow,26October2016 28

SlidecourtesyWolframMauser

15

D.Kranzlmüller Cracow,26October2016 29

Environmental (Friendly) Supercomputingon SuperMUC

Dieter Kranzlmüllerkranzlmueller@lrz.de

Photo:Karl Behler