HPCSummit,Prague

ExaFLOW

ExaFLOW

ExaFLOW–EnablingExascaleFluidDynamicsSimula>onsErwinLaure,KTH,Stockholm,SWEDEN

HPCSummit,Prague

ExaFLOW

ExaFLOW 2

ExaFLOW

àAddresscurrentalgorithmicboGleneckstoenabletheuseofaccurateCFDcodesforproblemsofprac>calengineeringinterest

HPCSummit,Prague

ExaFLOW

ExaFLOW

TheAirbusChallenge,DavidHills,2008

3

AnAirbus310cruisingat250m/sat10000mTeraflopsmachine(1012Flops):8·105yearsResultinoneweek:4·1019flopsmachine(40EFlops)(basedonJohnKim’ses>mate,TSFP-9,2015)

HPCSummit,Prague

ExaFLOW

ExaFLOW

Navier–Stokesequa>ons

DatafromMira(2013),millioncorehours•  Engineering/CFD 525 19%•  Subsurfaceflow&reac>vetransport 80 3%

•  Combus>on 100 4%•  Climate 280 10%•  Astrophysics 28 1%•  Supernovae 105 4%

1118 40%

4

(frac>onofNavier-Stokesbasedsimula>ononcurrentsupercomputers)

HPCSummit,Prague

ExaFLOW

ExaFLOW

ExaFLOWOverview

ThemaingoaloftheprojectistoaddresscurrentalgorithmicboGleneckstoenabletheuseofaccurateCFDcodesforproblemsofprac>calengineeringinterest.Thefocuswillbeondifferentsimula>onaspectsincluding:

•  accurateerrorcontrolandadapJvemeshrefinementincomplexcomputa>onaldomains,

•  solverefficiencyviamixeddiscon>nuousandcon>nuousGalerkinmethodsandappropriateop>misedprecondi>oners,

•  strategiestoensurefaulttoleranceandresilience,•  heterogeneousmodellingtoallowfordifferentsolu>onalgorithmsindifferentdomainzones,

•  parallelinput/outputforextremedata,employingnoveldatareduc>onalgorithms(feature-basedin-situanalysis),

•  energyawarenessofhigh-ordermethods,•  4differentcodes.

5

HPCSummit,Prague

ExaFLOW

ExaFLOW

ExaFLOWPartners

•  KTHStockholm,PDCandMechanics(Coordinator)•  ImperialCollege,London,CFD•  UniversityofSouthampton,Aerodynamics•  UniversityofEdinburgh,EPCC•  UniversityofStuGgart,HLRSandAerodynamics•  EPFLausanne,Mathema>cs

•  McLarenRacing,UK•  Automo>veSimula>onCenterStuGgart

6

HPCSummit,Prague

ExaFLOW

ExaFLOW

AlgorithmicDevelopment

•  3mainobjec>vestodevelopthemathema>calandalgorithmicknowledgeneededtotacklekeyobjec>vesandenableexascale-levelCFDsogware:– ObjecJve1:errorcontrolthroughadap>on,heterogeneousmodellingandresilience

– ObjecJve2:strongscalingatexascale– ObjecJve3:techniquesforI/Oatexascale

7

HPCSummit,Prague

ExaFLOW

ExaFLOW

WP1:6monthprogresshighlights

8

Workingimplementa>onofaspectralerrores>matortodriveah-adap>veprocessforObjec>ve1(KTH)

Ini>alformula>onofhybridCG-HDGononenodeundergoingini>altes>ngforObjec>ve2,

aimtoreducecommunica>onboGleneck(ICL)

Preprintofinves>ga>onintoresiliencemeasuresforhard/sogerrorsduringrun>meforObjec>ve1(EPFL)

HPCSummit,Prague

ExaFLOW

ExaFLOW

Powerandenergyprofilingtobuildbaselines

9

•  BuildingpowerandenergyprofilesofNektar++andNek5000toestablishbaselineuponwhichtomeasureimprovements.

•  Usingfulltestcasestoexercisecompletecode.•  Energy-to-solu>onand>me-to-solu>onasafunc>onofprocessor

frequency(leg);lookingforop>maltrade-offs.•  Varia>onseenbetweenotherwiseiden>calrunsofthesamecodeand

test-case(right);lookingtoquan>fyuncertaintyinmeasurements.

HPCSummit,Prague

ExaFLOW

ExaFLOW

AddingmodernparallelI/OtoNektar++

•  CodeboGleneckfoundinI/O.–  Slowscheckpoint/restartandfinalresultswrites.

•  Previouslymul>plefiles,perprocesswri>ng.•  NowusingHDF5atopMPI-IOforparallelI/Otosinglefile.

–  Non-trivialmappingofelementsoneachprocesstofile.–  AllowscheckpointonXnodes,restartonZnodes;X!=Z.–  Fitproblemtoavailableresources,evenwhensolu>onrequires>1jobsofdifferentlengths/widths.

•  Buildsontopofworkdoneinpriorproject,nowreadytomergeintotrunk.

•  Givesimplementa>ontobringtootherprojectcodesdoingsimilarI/Owithnon-trivialmappings.

10

HPCSummit,Prague

ExaFLOW

ExaFLOW

l  Fivecomputa>onally-demandingusecasessuitablefordemonstra>ngtheneedforexascalecapabili>eshavebeencreated:

l  l  -NACA4412(compressible)-Sotonl  -NACA4412(incompressible)-KTHl  -Jetincrossflow-KTH/StuGgartl  -AutomoJve/flowpastacar-ASCSl  -ImperialFrontwing–Imperial/McLarenl  l  SomeofthesehavealreadybeenpublishedonlineontheExaFLOWprojectwebsite.TheotherswillsoonbemadeavailableviaDeliverable3.1.

l  l  Quan>ta>vemeasurestoensurecorrectflowphysicsisreproducedagercodeop>misa>onshavebeendefined.

l  l  Computa>onalrequirementshavebeeninves>gatedandinternalevalua>onofusecasesisunderway.

l  11

PilotCases

HPCSummit,Prague

ExaFLOW

ExaFLOW

l  Vor>cesbeinggeneratedl  fromaNACA4412airfoil:

12

Turbulent straight pipe simulation(part of the jet in cross flow case):

Preliminary results of strong scaling from the incompressible NACA-4412 case (3.2 billion solution points):

Geometry for Imperial front wing use case:

HPCSummit,Prague

ExaFLOW

ExaFLOW

WorkPackages

13

1.  Algorithmicimprovementstowardsexascale(IC)2.  Efficiencyimprovementstowardsexascale(UEDIN)3.  Valida>on&usecases(SOTON)4.  Dissemina>onandexploita>on(USTUTT)5.  Management(KTH)

HPCSummit,Prague

ExaFLOW

ExaFLOW

ConclusionsandOutlook

•  Fluidmechanicsisaprimeexampleforexascale•  ExaFLOWwilladdresssomeoftheissueswhenitcomesto

pracJalapplicaJons–  errorcontrolandadap>vemeshingforlargerandmorecomplexsimula>ondomains;capableofdynamicremeshingifnecessary.

–  Heterogeneousmodelling–  Resilience&faulttolerance–  datahandling,complexfeatureextrac>on(in-situ)andsharingofsimula>ondata.

•  Enhancingcommunitycodes(Nek5000,Nektar++,SBLI);Open-sourcedevelopmentofallcomponents

14