Environmental (Friendly) Supercomputing on SuperMUC · D. Kranzlmüller Cracow, 26 October 2016 5 ... n Solid State Physics: Superconductivity, surface properties n Geophysics:

1

Environmental(Friendly)SupercomputingonSuperMUC

DieterKranzlmüller

Munich NetworkManagementTeamLudwig-Maximilians-UniversitätMünchen(LMU)&LeibnizSupercomputing Centre (LRZ)of the Bavarian Academy of Sciences and Humanities

FlashFlood Genoa,Italy, 2011

D.Kranzlmüller Cracow,26October2016 2

http://www.drihm.eu/images/video/DRIHM_final.mp4

2

FlashFloods

n Formswiftlydueto(extremely)highrainfallrates

n Littleornopriorwarning

n Devastatingconsequences(casualties,economiclosses,...)


UNISDR– TheUnitedNationsOfficeforDisasterRiskReduction


https://www.unisdr.org/

3

GAR– GlobalAssessmentReportonDisasterRiskReduktion 2015


http://www.preventionweb.net/english/hyogo/gar/2015/en/home/GAR_2015/GAR_2015_6.html

NumberofDisastersperRegion


http://www.emdat.be/disaster_trends/index.html

4

MunichRe– LossEventsWorldwide2014


http://www.preventionweb.net/files/41773_munichreworldmapnaturalcatastrophes.pdf

FlashFloods

n Formswiftlydueto(extremely)highrainfallrates

n Littleornopriorwarning

n Devastatingconsequences(casualties,economiclosses,...)

n Monitoringandforecastingoffloods:– EuropeanFloodAwarenessSystem(EFAS)– GlobalFloodDetectionSystem(GFDS)– GlobalFloodAwarenessSystem(GloFAS)

n Problem:spatialresolution50-100kmè Flashfloodsremainundetected


5

TheEUProjectSeriesDRIHM*


PossibleSolution– EnvironmentalComputing

n Combinemeteorology,hydrology,hydraulicsthroughcomputerscience

n Increasespatialandtemporalresolution(dataquality)– RegionalClimateModels(RCM)

n Computeensemblesofforecaststocoverallpotentialoutcomes

n Startandfinishcomputationintimetoprovideleadtimeforevacuationmeasures

è Simulateensemblesofforecastswithhigh-resolutionon

high-performancecomputing(HPC)infrastructuresondemandwhentriggeredbyincreasedrainfallrates


Notinthis talk

6

LeibnizSupercomputingCentreoftheBavarianAcademyofSciencesandHumanities


Withapprox.230employeesformorethan100.000studentsandformorethan30.000employeesincluding8.500scientists

• EuropeanSupercomputingCentre• National SupercomputingCentre

• RegionalComputerCentreforallBavarianUniversities• ComputerCentreforallMunichUniversities

Photo:ErnstGraf

LeibnizSupercomputingCentreoftheBavarianAcademyofSciencesandHumanities

n EuropeanSupercomputingCentre

n NationalSupercomputingCentre

n RegionalComputerCentreforallBavarianUniversities

n ComputerCentreforallMunichUniversities


SGI UV

SGI Altix

Linux Clusters

SuperMUC

Linux Hosting and Housing

7

SuperMUC@LRZ


Video: SuperMUC rendered on SuperMUC by LRZ

http://youtu.be/OlAS6iiqWrQ

Top500SupercomputerList(June2012)


www.top500.org

8

LRZSupercomputers

D.Kranzlmüller 15

SuperMUC PhaseII

Cracow,26October2016

SuperMUC Phase 1 + 2


9

SuperMUC System@LRZ

Phase2(LenovoNeXtScale WCT):

• 3.6PFlops peakperformance• 3072LenovoNeXtScale nx360M5WCT

nodesin6computenodeislands• 2IntelXeonE5-2697v3processorsand64

GBofmemorypercomputenode• 86,016computecores• NetworkInfiniband FDR14(fattree)

CommonGPFSfilesystemswith10PBand5PBusablestoragesizerespectivelyCommonprogrammingenvironment

Directwarm-watercooledsystemtechnology

Phase1(IBMSystemxiDataPlex):

• 3.2PFlops peakperformance• 9216IBMiDataPlex dx360M4nodesin18

computenodeislands• 2IntelXeonE5-2680processorsand32

GBofmemorypercomputenode• 147,456computecores• NetworkInfiniband FDR10(fattree)


PowerConsumptionatLRZ


0

5.000

10.000

15.000

20.000

25.000

30.000

35.000

Stro

mve

rbra

uch

in M

Wh

10

CoolingSuperMUC


SuperMUCPhase2@LRZ

Cracow,26October2016

HighEnergyEfficiencyü UsageofIntelXeonE52697v3processorsü Directliquidcooling

- 10%poweradvantageoveraircooledsystem- 25%poweradvantageduetochiller-lesscooling

Photos:TorstenBloth,Lenovo

ü Energy-aware scheduling- 6%poweradvantage- ~40%poweradvantage- Totalannualsavingsof~2Mio.€

forSuperMUCPhase1and2D.Kranzlmüller 20Slide:HerbertHuber

11

LRZ Application Mix

n Computational Fluid Dynamics: Optimisation of turbines andwings, noise reduction, air conditioning in trains

n Fusion: Plasma in a future fusion reactor (ITER)n Astrophysics: Origin and evolution of stars and galaxiesn Solid State Physics: Superconductivity, surface propertiesn Geophysics: Earth quake scenariosn Material Science: Semiconductorsn Chemistry: Catalytic reactionsn Medicine and Medical Engineering: Blood flow, aneurysms, air

conditioning of operating theatresn Biophysics: Properties of viruses, genome analysisn Climate research: Currents in oceans


Results (Sustained TFlop/s on 128000 cores)


Name MPI #cores Description TFlop/s/island TFlop/smaxLinpack IBM 128000 TOP500 161 2560Vertex IBM 128000 PlasmaPhysics 15 245GROMACS IBM,Intel 64000 MolecularModelling 40 110Seissol IBM 64000 Geophysics 31 95waLBerla IBM 128000 LatticeBoltzmann 5.6 90LAMMPS IBM 128000 MolecularModelling 5.6 90APES IBM 64000 CFD 6 47BQCD Intel 128000 QuantumPhysics 10 27

12

PartnershipInitiativeComputationalSciencesπCS

n Individualizedservicesforselectedscientificgroups– flagshiprole– Dedicatedpoint-of-contact– Individualsupportandguidanceandtargetedtraining &education– PlanningdependabilityforusecasespecificoptimizedITinfrastructures– EarlyaccesstolatestITinfrastructure(hard- andsoftware)developments

andspecificationoffuturerequirements– AccesstoITcompetencenetworkandexpertiseatCSandMath

departmentsn Partnercontribution

– EmbeddingITexpertsinusergroups– Jointresearchprojects(includingfunding)– Scientificpartnership– equalfooting– jointpublications

n LRZbenefits– Understandingthe(currentandfuture)needsandrequirementsofthe

respectivescientificdomain– Developingfutureservicesforallusergroups– Thematicfocusing:EnvironmentalComputing

D. Kranzlmüller Cracow,26October2016 23

SeisSol - Numerical Simulationof Seismic WavePhenomena


Picture:AlexBreuer(TUM)/ChristianPelties(LMU)

Dr.ChristianPelties,Departmentof Earthand EnvironmentalSciences (LMU)Prof.MichaelBader,Departmentof Informatics (TUM)

1,42Petaflop/son147.456Coresof SuperMUC(44,5%of PeakPerformance)

http://www.uni-muenchen.de/informationen_fuer/presse/presseinformationen/2014/pelties_seisol.html

13

Conclusions

n EnvironmentalComputingneedsIT-Infrastructures(includingHPC)

n EnergyEfficiency isanimportantparttomaximizescientificthroughput

n Computationalscienceneedstobeanintegralpartofteachingdomainscientists– LearnhowtogetaccesstoHPCinfrastructures– LearnhowtoprogramHPCinfrastructureswithincreasingcomplexity,

heterogeneityandscalability– efficiency,reliability,portabiliy

n TheLRZPartnershipInitiativeComputationalScience(piCS)triestoimproveusersupport

http://www.sciencedirect.com/science/article/pii/S1877050914003433


TheEUCopernicusEarthObservationPlatform

n TheCopernicusSentinel Missions are „game changers“for EarthObservation:watch the heartbeat of the planet

D.Kranzlmüller Cracow,26October 2016 26

Sentinel 2Sentinel 1

10m,every 2daysca.3PByte permonth

raw data

10m,every 2..5daysca.4.5PByte permonth

raw data

SlidecourtesyWolframMauser

14

Complex ImageAnalysisderivesEnvironmentalParameters

n Example:quantitativesatellite image analysis of wheat fields


Chlorophyllcontent [µg/m²]Satellite image

Parameters,e.g.:Plantspecies,biomass,chlorophyll,pests,phenology,…


EnvironmentalComputing

n Human-Environment-Relation- ObservationsandSimulationsforalternativeGlobalFutures

n Massivecomputingresourcesareneededtocreateacyber-environmentalsystem inwhichtherealandthevirtualworldcaninteract:– toturnremotesensingimagedatastreamsintomeaningfulenvironmental

informationforeachfarmerontheglobe– toidentifyleastinvasivewaysforagriculture– todoublefoodproductionandshowthe

globalenvironmentalbenefitsithas– tosimulateandassessthetotal

environmentandthehumaninterventionsbeforetheyoccur

– toexplorealternativefutureenvironmentsandtheirsustainabilityandqualityoflife



15


Environmental (Friendly) Supercomputingon SuperMUC

Dieter Kranzlmü[email protected]

Photo:Karl Behler

Documents

Environmental (Friendly) Supercomputing on SuperMUC · D. Kranzlmüller Cracow, 26 October 2016 5 ... n Solid State Physics: Superconductivity, surface properties n Geophysics: