ARCHERUserSurvey

2015

2

1. DocumentInformationandVersionHistoryVersion: 1.0Status Final

Author(s): AnneWhiting

Reviewer(s) AlanSimpson,AndyTurner

Version Date Comments,Changes,Status Authors,contributors,

reviewers0.1 2016-03-07 Initialdraft AnneWhiting0.2 2016-03-22 Furtherupdates AnneWhiting0.3 2016-04-13 Updates AnneWhiting0.4 2016-04-18 Reviewed AndyTurner0.5 2016-04-19 Post-reviewupdates AnneWhiting0.6 2016-04-19 Reviewed AlanSimpson1.0 1016-04-19 VersionforEPSRC AlanSimpson,AnneWhiting

3

2. DescriptionoftheSurveyTheARCHERUserSurveyclosedon17thFebruary2015.230responseswerereceivedfromARCHERusers.Thesurveyaskedforratings(onascaleof1to5)withthefollowingquestions:

1. PleaserateyouroverallexperienceoftheARCHERService(required)[VeryUnsatisfied(1)–VerySatisfied(5)]

2. HastheARCHERhardwareconfigurationmettherequirementsofyourresearch?(required)[Notmetanyrequirements(1)–Exceededrequirements(5)]

3. HasthesoftwareonARCHERmettherequirementsofyourresearch?(required)[Notmetanyrequirements(1)–Exceededrequirements(5)]

4. IfyouhaveusedtheARCHERhelpdesk,pleaserateyourexperience[VeryUnsatisfied(1)–VerySatisfied(5)]

5. IfyouhaveusedtheARCHERdocumentation,diditprovidetheinformationyourequired?[DidnotprovidetheinformationIrequired(1)–ProvidedalltheinformationIrequiredandmore(5)]

6. IfyouhaveusedtheARCHERwebsite,pleaseratethequalityofthecontentandeaseofnavigation[Verypoor(1)–Excellent(5)]

7. PleaserateyourexperienceofanyARCHERTrainingyouhaveused(eitheronlineorface-to-face)?[VeryUnsatisfied(1)–VerySatisfied(5)]

8. IfyouhaveattendedanyARCHERwebinarsorvirtualtutorials,didyoufindthesessionworthwhile?[Acompletewasteoftime(1)–Extremelyinterestinganduseful(5)]

9. IfyouhaveattendedanyARCHERonlinetrainingmaterial(e.g.OnlineDrivingTest,screencasts),howusefuldidyoufindthematerial?[Ofnouse(1)–Extremelyuseful(5)]

Onlythefirstthreequestionswerecompulsoryforallsurveyresponders,butover93%ofrespondersalsoprovidedfeedbackinatleastsomeoftheoptionalquestions.Userswerealsoprovidedwiththeopportunitytooffercommentsorsuggestionsunderalloftheaboveheadingsandprovidedwithspaceforanyothercommentsorsuggestionsattheendofthesurvey.ThesurveywasconstructedusingGoogleFormsandembeddeddirectlyintotheARCHERwebsite.

4

3. SelectedQuotesThefollowingquotesreflectthetoneofthemajorityofresponderstothesurveywithregardtotheARCHERservice:

“IfindtheARCHERservicetoberesponsiveandhelpful,easytouse,andflexible.Excellentservice.”“TheonlinedrivingtestwashowIobtainedaccesstoARCHER,andIthinkitisabrilliantideaforwideningaccess.ThishasallowedmetoputtogetheraneCSEapplicationforthenextcall,andhopefullyacceleratemyresearchinfuture.”“Archerreallyisagreatservice.IhaveusedothersupercomputersbuttheassistanceandtheguidanceonArcherissignificantlybetter.”“TheUKNationalSupercomputerserviceisanessentialtoolinsupportofscientificresearchwithintheUK.”“Anexcellentservice!Thankyouforallyourhardwork!”

Quotesonthehelpdesk(whichalsoreflectonthecentralisedCSEteam)echothehighratingsforthisaspectinparticularareshownbelow:

“Justtore-iteratethehelpdesk/supportmakesarchersobeneficialandhasreallysavedmeanimmenseamountoftimewhentryingtocompilesoftware,orwhenaproblemhasariseninacalculation.”“Fantastic.Theyknowwhentobringexpertsinonparticularpackagespecificquestionsalso.”“Alwayshelpful,andIlikethefactthattheycheckwithyoubeforeclosingajobtomakesureeverythingisfixed.““Allhelphasbeenpromptanddirectedtowardsimprovingmyexperience.Somesuggestionstoimproveperformancehavebeenofferedwithoutmyrequestandthat'salsoprovedveryuseful.”“Thehelpdesktendstobequicktorespond,efficient,andhelpful,evenwithfairlycomplexrequests.Greatjob!It'salwaysapleasuretoworkwiththehelpdesk.”

Therewereonlytwocommentsonthefilesystemsproblems:

“DiskfailuresandprolongeduseofRaidCheckdiagnosticsgreatlyreducedtheperformanceofthesystemandthethroughputofmyjobsoverthepast12months.““Theissueswiththefilesystemshavehadanadverseeffectonourresearchwork,mostlyintermsofatimelost.“

AfulllistofthecommentscanbefoundinSection5.

5

4. RatingsAllquestionsaskedresponderstoratetheirsatisfactionwitheachparticularaspectofthesurveyonascaleof1to5with1representing“VeryUnsatisfied”and5representing“VerySatisfied”.Table1summarisestheratingsforeachaspectandrevealstheallaspectsoftheARCHERServiceareratedhighlybyusers.Thenumberofresponseswasupfrom153in2014to230in2015.Table2showstheresponsestothe2014surveyforcomparisonpurposes.ServiceAspect

TotalResponses MeanScore(outof5) MedianScore(outof5)

OverallSatisfaction 230 4.3 4Hardware 230 4.1 4Software 230 4.0 4Helpdesk 198 4.5 5Documentation 215 4.1 4Website 221 4.2 4Training 147 4.1 4Webinars 102 3.9 4Onlinetraining 104 4.0 4Table1:SummaryofscoresfordifferentaspectsoftheARCHERService2015ServiceAspect

TotalResponses MeanScore(outof5) MedianScore(outof5)

OverallSatisfaction 153 4.4 4Hardware 153 4.1 4Software 153 4.0 4Helpdesk 198 4.5 5Documentation 142 4.1 4Website 144 4.1 4Training 81 4.1 4Webinars 41 3.6 4Onlinetraining - - -Table2:SummaryofscoresfordifferentaspectsoftheARCHERService2014Table33showsthatthemeanratingsin2015fordifferentaspectsoftheserviceareverysimilartotheequivalentratingsinthepreviousyear.AllaspectsoftheARCHERservicecontinuetoreceiveveryhighsatisfactionratingsfromtheusers.Inparticular,theHelpdeskcontinuestostandoutasthehighestratedaspectoftheserviceinbothsurveys,withanextremelyhighrating.Thisistestamenttothehardworkofallservicepartners(SP,CSEandCray)inensuringthatresponsestotheusersthroughthehelpdeskaretimely,accurate,usefulandpolite.ServiceAspect

2014MeanScore(outof5) 2015MeanScore(outof5)

OverallSatisfaction 4.4 4.3Hardware 4.1 4.1Software 4.0 4.0Helpdesk 4.5 4.5Documentation 4.1 4.1Website 4.1 4.2Training 4.1 4.1Webinars 3.6 3.9Onlinetraining - 4.0Table3:Comparisonofmeanscoresfrom2014and2015UserSurveysfordifferentaspectsoftheARCHERService

6

AscanbeseenfromFigure1,theoverallsatisfactionwiththeARCHERserviceisextremelyhighwithonly8respondersratingtheservicebelow3ona1-5scalefrom“VeryUnsatisfied”to“VerySatisfied”,i.e.3percent.Themeanratingis4.3andthemedianratingis4.Theseratingsareverysimilartotheratingfrom2014UserSurveywherethemeanratingwas4.4andthemedianratingwas4.

Figure1:DistributionofscoresforoverallsatisfactionwiththeARCHERservice(230responsesintotal).

Similarly,forthehardwareandsoftware(Figure2andFigure3respectively),theoverallsatisfactionwiththeserviceishighwithonly5usersratingthehardwarebelow3and9usersratingthesoftwarebelow3.Thesingleratingof1(“VeryUnsatisfactory”)forthehardwareonARCHERwasaccompaniedbyacommentstatingthatthehardwareisfinebuttheturnaroundforNERCjobswasnotgoodrecently.Theratingof1forthesoftwareonARCHERisfromthesameuserwhogavethehardwareascoreof1andhasanegativecommentaboutrecentjobturnaround.Theresponsehasbeensubmittedanonymouslysowecannotfollowupthecommentswiththeresponder.Themeanratingforhardwareis4.1(medianis4)andthemeanratingforthesoftwareis4.0(medianis4).Theseratingsareexactlythesameasthosefromthe2014UserSurvey.

Figure2:DistributionofscoresforsatisfactionwiththeARCHERhardware(230responsesintotal).

0

20

40

60

80

100

120

1 2 3 4 5

Respon

ses

Racng

0

20

40

60

80

100

120

140

1 2 3 4 5

Respon

ses

Racng

7

Figure3:DistributionofscoresforsatisfactionwiththeARCHERsoftware(230responsesintotal).

ThesatisfactionratingsfortheARCHERHelpdeskshowed4responseswithascoreunder3andameanratingof4.5(medianis5).Theseareidenticaltothemeanandmedianinthe2014UserSurvey.Ofthe198responses128,65%,gaveascoreof5(“Excellent”).Oneoftheuserswholeftascoreof1indicatedthattheydidnotwishtobecontactedregardingtheirresponsetothesurveyandtheotherhasbeencontactedforfeedbackastheydidnotleaveacomment.

Figure4:DistributionofscoresforsatisfactionwiththeARCHERhelpdesk(198responsesintotal).

ARCHERdocumentation(fig5,mean=4.1,median4)andwebsite(fig6,mean=4.2,median4)showthesamehighlevelofsatisfactionasthatshownfortheoverallserviceandhavehighrespondentrates.TheresultsforARCHERtraining(fig7,mean=4.1,median=4)arehighandconsistentwiththecoursesurveyresultspresentedintheCSEServicequarterlyreports.Therearenocommentsfromuserswithscoresunder3andanumberofrespondershavescoredfortrainingwithacommentthattheyhadnotattendedanytraining.Asfortheotheraspects,thesescoresareverysimilartothescoresfromthe2014UserSurvey.Thewebinarsandonlinetraininghaveamuchlowerresponderrate(possiblyduetothefactthatthetechnicalnatureofthewebinarsisofinteresttoasubsetofARCHERusers)butshowahighsatisfactionrating(figs8and9,mean=3.9and4.0respectively,median=4).Therewassome

0

20

40

60

80

100

120

140

160

1 2 3 4 5

Respon

ses

Racng

0

20

40

60

80

100

120

140

1 2 3 4 5

Respon

ses

Racng

8

apparentresponderconfusionbetweentrainingandonlinetrainingandwebinars,andthusthequestionsinthesurveyfor2016willberefinedtoavoidthis.

Figure5:DistributionofscoresforsatisfactionwiththeARCHERdocumentation(215responsesintotal).

Figure6:DistributionofscoresforsatisfactionwiththeARCHERwebsite(221responsesintotal).

0

20

40

60

80

100

120

1 2 3 4 5

Respon

ses

Racng

0

20

40

60

80

100

120

1 2 3 4 5

Respon

ses

Racng

9

Figure7:DistributionofscoresforsatisfactionwiththeARCHERtraining(147responsesintotal).

Figure8:DistributionofscoresforsatisfactionwiththeARCHERwebinars(102responsesintotal).

0

10

20

30

40

50

60

70

1 2 3 4 5

Respon

ses

Racng

0

5

10

15

20

25

30

35

40

45

1 2 3 4 5

Respon

ses

Racng

10

Figure9:DistributionofscoresforsatisfactionwiththeARCHEROnlineTraining(104responsesintotal).

0

5

10

15

20

25

30

35

40

45

50

1 2 3 4 5

Respon

ses

Racng

11

5. ListofcommentsThenumberinbracketsindicatestheIDoftherespondent.Hardware• Itisnotpossibletousepartofprocessorsonanode:thisconstrainsscalabilitytoamultipleof24,

i.e.thenumberofprocessorsonanode(14)• Itwouldbeextremelyusefultohaveafewnodeswithmorememory(17)• Lesscrashing(23)• ThefocusontraditionalCPUs,ratherthanaccelerators,isgreatforscientistsneedingtospend

timeonscience,notrewritingtheirsoftware(wheretheiralgorithmisamenable)forGPUs.Formyapplications,interconnectisnotthemostimportantthing,andthefastestcorespossibleisthepriority.(28)

• WouldbenicetohaveaccesstoafewGPUsfortrialpurposes.(32)• Bigmemorynodesareveryuseful-moreofthesewouldbegreat.(34)• Verypleasanttouse,thesystemfastandthequeuesaregenerallyquitequickundernormal

usageloads.(40)• Pleasestopdoingmaintenancemidweek...itcauseshugequeuebacklogsanddestroystheflow

oftheweekattimes(48)• Thehardwareandinterconnectisexcellent,andmuchbetterthantheregionalN8machine

(scalabilityis~200coresonN8vs6000coresonARCHER).(50)• Theissueswiththefilesystemshavehadanadverseeffectonourresearchwork,mostlyinterms

ofatimelost.(55)• Thereisinsufficientstoragespacetopost-processanydata.Datatransfersareslow.(58)• Samearchitectureandsoftwareacrosscomputeandserialqueues.Longeraccess(>12hrs)and

betterperformanceofserialqueuenodes.Infutureinmemorycomputingwouldbeusefultoreducedatasizeandoutputtodisk.(60)

• RAMonindividualnodesareonlowerside.Mostcodesscaleswellwith8GBpercoreRAM.Also,ifgeneralqueueallow72hourjobs,itwillbehelpful.(62)

• Anywaytobetterhandlemanysmallfiles?(66)• Longqueueandwaitingtimefortheinteractivesessions(67)• unknown,yet.(68)• IoftenusethelargememoryCPUsavailableonArcherandduethesmallernumberofthose

specificCPUsthewaitingtimesaremuchlongercomparedtothestandardCPUs.PossiblyIwouldliketoseeanincreaseofthenumberofsuchCPUs.(73)

• Niceandfast!(74)• Itwouldbeniceiftherewaslesslagwhenaccessingfile.(77)• Todate,IhaveonlyusedARCHERfortrainingcourses.(80)• CurrentlytryingoutDDTreverseconnection.Itworkedbrieflywhichwasgoodbutonlyatoy

codefortesting.Itsgreatthatwehavethisfacility(89)• Itwouldbeniceiftherewasadefault6monthextensionofCPUhoursbeyondtheendofa

grant.Manypapersaresubmittedattheendofagrantandextratimeforreviewercommentswouldbeuseful.(91)

• IhaveproducedasubstantialnumberofsimulationsonARCHERthisyear.MostlyIhavebeenveryimpressedwiththespeedofjobsgoingthroughthequeue.However,therehavebeeninstanceswherejobshavewaitedforseveraldaysbeforebeingprocessedandthishasheldupmyresearch.(102)

• ThejobsIrunonArchertypicallyperformbestwithnumbersofcoresthatarepowersof2.With24corespernode,itisnotpossibletoreachapowerof2anduseallavailablecoresatthesametime.Itwouldbemoreusefuliffuturesystemshadsomethinglike16or32corespernode.It

12

wouldalsobeusefultoincreasethenumberofhighmemorynodestoatotalof512.TherearesomejobsIwouldliketorunthatwouldbenefitfromthisnumberofhighmemorynodes.(104)

• ARCHERcanprovidemoreabilityforthepost-processingnodessinceIfoundthatthespeedformypost-processingwithOpenFOAMutilitiesisalwaysveryslow.ThesejobsaresubmittedtoserialqueueinArcher.(108)

• Veryreliable,goodperformance(109)• Greatforrunninglargejobs.Howeverbecausethequeuesareoftenlongitisn'tthatpracticalto

debugsmallerjobsetc.(119)• NoneatpresentasIhaveonlyjustbeentrainedonhowtouseit(121)• Transferspeedsbetween/workdiskto/nercand/nerctoJASMINevenusinglightpatharequite

slow.Thathasbeenthemajorlimitationofhowmuchsimulationswecanperformwithingivendaysratherthanthecomputationspeedorlengthofqueue.That'squiteashameisn'tit?(126)

• Thecomputetimesandqueuetimesaremanageable.However,thefilesystemisoftenextremelysluggishduringweekdayswhenpeopleuseARCHER.Jobsthatnormallytake~2hrscansometimeshitthe5hrwalltimeduetoslowread/writespeeds.(127)

• ExcellenthardwareformyGroup'sneeds!(128)• ForbigdatatasksmoreuseoftheMOMnodesforamotherprogram(i.e.forextremelyparallel

quantummontecarlo)wouldbeniceinthefuture.Otherwisegreat.(131)• It'sbehelpfultohavealarger/workspace(132)• TheonlyissueIhaveisthatreadingandwritingfilesseemsquiteslow.ParticularlywhenIwant

totarupasetofresultssoIcancopythemelsewhere.It'safairlyminorpoint.(133)• Stable,fast,goodamountofmemorypercore.Onlyslightcaveatisthelackofdiskspace.(138)• Stable,fast,goodamountofmemorypercore.Onlyslightcaveatisthelackofdiskspace.(143)• Availabilityof>2GB/corememoryaswellastheexistenceoflargememorynodehasbeen

fundamentaltosustainourresearch.(146)• Thehardwareisfine,whenjobsrun.However,overthepastfewmonthstheturnaroundforour

NERCjobshasbecometerrible.WithdecentturnaroundArcherwouldbeexcellentforourwork.(151)

• Fastercoresandfasternetworking(probablylowerlatencyratherthanbandwidth)wouldgivebetterperformanceforourmassiveatomisticsimulations.Don’tgetmewrong,ARCHERisfantasticbutour40millionatomsimulationsscalelinearlyto3072coresbutonly80%efficientat6144.SoIcantgive"5Exceededrequirements"althoughIlovethemachine.(152)

• Youmaywanttoexperimentaddingacceleratorsonsomenodes.Icanseethatyoumaygainexperienceonthisfield,whichmaybecomemoreimportantinthefuture.Mysuggestionhowever,istokeeptheARCHERastheverylargecomputerclusterthatitisrightnow,andupgradeitwithfastermemoryandmorepowerfulprocessors.(153)

• Goodperformancepernode(155)• DiskfailuresandprolongeduseofRaidCheckdiagnosticsgreatlyreducedtheperformanceofthe

systemandthethroughputofmyjobsoverthepast12months.Additionally,thescheduler'sstrongpreferenceforlarge(>200node)jobsmeansthatuserswho,likeme,requiresmallernumberofcoresforlongerperiodsoftimeareatadisadvantage.Manyofmysimulationsrequireweekstorun,butuseonly16-64nodes.Throughputforthesejobsispoor.(160)

• Itmaybeasoftwareissueratherthanhardware.Thefilesystemslowsdownfrequentlyandonceseveralfilesareinafolder(about1000orso)thefilesystembecomesdifficulttoworkwith(161)

• Morestoragespacewouldbeevenbetter.(171)• IamveryhappywithARCHERandhavenothadaproblemyet.(178)• Thehardwareisverygood,mydown-votingismoreintermsofavailability.Toworkefficiently

I'dneedveryquickaccesstoadevelopmentqueueavailablefordebugging,can'twaitfor2hinthedevelopmentqueueto"sortouttheedges"...Alsoforproductionrunsqueuetimeshavebeenridiculouslylongthisyear.(182)

• Iwaswondering,whetherArcherisabletohavealargememoryloginnodeforsomesimplepost-processinguseifpossible.(183)

13

• InthelastyearstheBarcelonaSupercomputingCenterhasmaintainedsimultaneouslyaCPUbasedmachine(Marenostrum)andaGPUbasedmachine(Minotauro).WiththegrowingpopularityofGPUbasedsimulationcodes,thismaybeasuitableoptiontoexploreinthefuture.(186)

• Asalways,wecoulduseabiggerandfastercomputer...(187)• Prioritisediskspacebeforenumberofcores.ItisveryeasytogenerateTBsofdatawiththe

currentsizeofthemachine,butthereisn'tthediskspaceavailabletoholdit.Thesharednodesareutterlyuselessforcompilingcode,andmakemenotwanttousetheserviceatall.Asthesharednodesareexactlythat,andareoftenoverloaded,whatshouldtake~10minutestocompiletimesoutatanhourormore.(188)

• Matlabonthefewpost-processingnodeswouldbequithelpful.(195)• Archerhardwareexcellent,especiallylargememcores.(198)• justwewaitsometimesforlongtimeinthequeue,ifthereisanysolutionforthisproblemwe

willbegrateful.(208)• Verypowerful.(209)• JustasIneedittobe.(216)• IusetheRDFalotatthemoment.It'sveryusefulformodeldatastorage.IwillusetheCPUs

laterthisyearagain,forUKESM.(218)• Goodarchitecture,butcoulddowithbeingabitlarger.I/Ocanbeextraordinarilyslow,and

there'salotofnoiseinruntimesstill(+/-20%istypical,butsometimesmuchlarger)anditwouldbegoodtosolvethis.(225)

• Pleaseopensomequeuewithlongerwalltime.Forlong(10days)simulations,the24hwalltimemeanswastingourtimeinrestart(229)

• Hardwareisgood.Thenumberofprocessorspernode(24)canmakedomaindecompositionawkward,sinceapowerof2isusuallypreferred,butthisisasmallmatter.(230)

14

Software• InstallationofoftenrequiredMatlab(1)• Sometimespackageupgrades(whichoftenbreakcode)arenotaswelldocumentedasremovals.

(13)• ThelinearequationsolverthatIusedseemedslowincomparisontoothersystems,butIonly

foundthisoutaftertheeventasitwasn'tamajorpartofmycode.Somedocumentationpointingtoperformanceofdifferentlinearalgebratasksondifferentcompilerswouldbeuseful.(18)

• YoucouldaddBerkeleyGWasapreinstalledmodule.(19)• Apermanenttestqueueisneededwithshortwall-times.Withthecurrentsetup,testingisvery

difficultasevensmalljobsmayqueueforinexcessof24hours.(22)• TheCraycompilershaveoftencausedtrouble,andIhaveinsteadhadtorelyontheGNU

programmingenvironment.Librariesandpackageshoweverhavenevercausedmeproblems.(28)

• I'dbehappyifMidnightCommander(unixfilemanager)operatesinafullregime,i.e.withthesubshellwhichactuallyworks.(29)

• Cantakeaverylongtimeforsmallerjobstostartrunning-sometimesupto5days.Wouldbegoodtotweaktheschedulingsystemtomakethiswaitabitshorter.(34)

• Paraviewrunsslowwithlargedatasets,butthismaybeinevitable.(36)• TheonlycommentthatIhaveisthatIwouldlikeautomatichighlightingontheheadnodeforfile

typese.g.tar.gzandfordirectoriestomakeitalittlesimplertodistinguishbetweendifferentitems.(40)

• ARCHERrdfcouldbenefitfromhavingafewmoresoftware.Forexample,NCLcouldbeinstalledaswellasasoftware(otherthangs)forviewingpdfdocuments.(44)

• IwouldneedtousesoftwaretoperformQMcalculationslikeGuassian,QChem,Orca.IunderstandtheseprogramsdonotparallelizeveryefficientlybutitiswhatIneedfordoingmyresearch(48)

• Generallythecompilerisgood,butthecurrentdefaultversionoftheCrayCcompilerisbuggyanddoesn'tcompilemycode(itcompilesonallotherplatforms/compilerswithoutissue).ThenextversioncompilesthecodeOK.(50)

• SomesoftwarewasoutofdatebutonceIemailedthehelpdesk,theyupdatedittothelatestversionquicklyenough.(53)

• Thepythonchangeshavebeenabitofapain-hopefullyit'llbestablenow.(56)• ThecodeI'musingisLAMMPS(nottheprovidedversionbutonewhichincludesmodifications).It

isapitythatLAMMPSdoesnotcompileproperlywiththedefaultcompiler,anditisrecommendedtoswitchinsteadtotheGNUcompilerasthissurelyaffectsperformance.Idon'tknowwhetherthisisacompilerproblem,aLAMMPSproblemorsomethingelse.(57)

• Samesoftware/compilersforcomputeandpostprocessingnodes.Graphical/visualparalleldebugging+performanceanalysiswouldbeusefuloncomputenodes-couldbedoneoninteractivenodesifsamearchitecture+softwareenvironmentisused.Itcanbedifficulttodebug/testusingsmallnon-productiontyperuns.(61)

• Isitpossibletohaveanestimationthewaitingtimeaftersubmittingajob?(67)• Compilersandenvironmentmodulessystemisexcellent.ARCHERisbyfartheeasiest

environmentIhaveusedtocompilescientificsoftware,andagoodselectionisavailable(74)• Todate,IhaveonlyusedARCHERfortrainingcourses.(80)• Maybeitisalreadypossible,andifsopleaseacceptmyapologiesforthiscomment.Ithinkthatit

wouldbeusefulforsomepeople(IknowactuallyquiteafewofthemIworkwith,particularlyinthebio-physics,photo-physics,moleculardynamicssectors)togetaccesstovery-longqueues.Somethinglikelastingaweek.Theydonotneedlargesystems,orlargenodes(materialsscientistslikemedo)justafewCPUsworkinguninterruptedlyforaweekortwo.Theyarenotwillingoritisnotinherentlypossibletoimplementmiddlecheckpointsintheirsoftware.

15

Furthermore,youwouldgetmoreusersfromEngineeringbackgroundbyaddingsomemoremulti-physicssoftwareorfiniteelementssoftware.(86)

• IuseonlymysoftwarebutIhaveaccesstoallthelibsIneedtocompileit(91)• supportabitslow,pythonmodulesstrugglingtokeepupwithe.g.releasesonUbuntuLTS?(92)• Morecompletepythoncomputesetofmodulesforscientificcomputingwouldbeusefuland

savesometimeinstallingthemlocally.(101)• Itwouldbeveryusefulifuserswereallowedtorunscreenontheloginnodes.Also,Iwasasked

recentlytostopusing"ls--color"asitslowsdownthefilesystem.ThisisabitsadconsideringhowusefulitisandthatthisisallowedoneveryothersupercomputerIhaveeverused.(104)

• I'mgladthatARCHERhasPython!(105)• Verygood.(108)• Mostofthesoftwareweuseisourownbutalllibrariesweneedwereavailable.(109)• tmux!(115)• tmux!(116)• Wheneveranythinghasn'tbeenavailableithasbeenaddedquickly.(119)• itisgoodtohavethemakefilesforsomeoftheinstalledsoftwarepackagesavailableattheweb

page;thisservesasastaringpointforowncompilationswherechangesinthesourcecodehavetobeaccountedfor(123)

• Ispent3monthstryingtodiscoverwhatfeatureofmycodeleadtosomenodesrunningmoreslowlythanothers,onlytohavethaterraticperformancebehaviourvanishasaresultofa`systemupgrade'.(124)

• LackofIDL,ncdump,ncview,ncohavebeenthelimitationofARCHER.ThishasbeenareasonwhyIneedtotransferdatatoJASMINorlocalcomputerwhichtakestime.(126)

• Afewdaysago,IhadanincidentwhenIcouldnotcompilemycodeduetoproblemswiththepathtoalibrary.Onthefollowingdaytheproblemseemedtohavebeenresolvedasthesamecodecouldcompilewithoutanyproblems(althoughIdidnotnotifyanyonefromtheARCHERSupportTeam).(128)

• Onesmallthingwouldbeifyourpersonalqstatlistedhowmanyjobsoutofyourtotalyouhavesubmittedsofar-i.e.14/16jobsinqueue.(131)

• Theerrormessagesthatyougetwhenajobfailsareratherimpenetrable-particularlywhenitisjobcontrolfault.(133)

• WenormallyuseRandrgdalpackagetoprocessgeographicaldatabutitrequiresexternallibrariessuchasGDALandPROJ4.WoulditbepossibletohavetheselibrariesinstalledontheArcher?(134)

• Itendtoself-installallnecessarysoftware.(138)• nocomments.Allavailableasneeded.(139)• IdeallyIwouldliketobeabletousethepost-processingsoftwareParaviewremotelyinparallel

forvisualizingresultsfromOpenFOAMCFDsimulations.(141)• WouldbeusefultobeabletorunParaviewonArcherforpost-processing.Thiswouldsavehaving

toregularlydownloadlargequantitiesofdata.(142)• CurrentlyArcherhaslimitedsupportforIntelcompilertools.ItwouldbegreatifthelatestIntel

CompilerSuiteandperformanceanalysistoolsarealsosupported.(147)• Fortrancoarraycornercasesneedmoreresearch.Someunexplainedbehaviourencountered.

Craysupporthasbeenexcellent,butthisseemsanareawhereimprovementscanbemade.(148)• Againthemachineisgreatwhenjobsrun.However,theturnroundhasbecometerrible.Even

quitesmalljobs(64cores)queueformanydays(includingweekends).Notsureifthisasoftwareissueorhardwareorpriority....(151)

• Thereisagreatup-to-dateamountofsoftwareinstalled.Thankyou!(153)• ProvidingcustommodulestootherARCHERusersisstillrathercumbersome.(154)

16

• Thesoftwareisupdatedtoofrequently,particularlythecompilersuitesandthesystemlibraries(e.g.,MPICH),requiringrecompilationofcode.Noreasonsareprovidedtousersfortheupdatesexcepttoincreaseversionnumbers.Iwouldpreferamorestablesoftwareenvironment.(160)

• Themessingwiththepythonstackhasbeenabitfrustrating,butotherwiseit'sgreat.(173)• Iusegromacsanditisworkingverywell(178)• Frequentlybrokenmodulesafterupdatecycles,issueswithpythonlibraries,guilibraries...(182)• great!(183)• IamgenerallyhappywiththesoftwareonARCHER,andanythatisn'tavailablehasbeeninstalled

reasonablywell.Iwouldsaythatsometimesonlythegnuversionisinstalled,whenitwouldbegoodtohaveaversioncompiledunderthecceinstead.(188)

• ModuleconflictsbetweenscientificlibrariesandMPIlibrariesareunfortunate(butdifficulttoavoid).ThereisnotoolthatcanproduceacommunicationtimelinealaVampir.(201)

• SomesortofmoreautomatedarchivingfromworktotheRDFwouldbeuseful.(203)• ReallysatisfiedwithcoherentpackagesofMPI,FFTWandotherlibrariesfordifferent

programmingenvironment.Ifonlythiswasavailableonsmalleruniversityclusters.Unfortunately,theparallelprofilersturnedouttobebuggyandtheydidn'tworkwithmycodeineCSE05-14.Wehavespentalotoftimewiththem.(205)

• Morejobscriptsforthosescientificsoftwaresunderindividualwebpages.(209)• I'mnotanadvanceduser,don'thaveintensivedemands.Soyes,hasmetmyexpectations.(211)• Itwouldbenicetobeabletorunsmallshortscriptstomovefilesaround.(215)• Inmyexperience,ARCHERisstrippedbacktojustwhatisneeded,andIamhappytoanalyse

elsewhere(220)• Havingagraphicsessionandsometooltovisualiseatomistructureswouldhelpourjo(229)

17

Helpdesk• Thehelpdeskisexceptional,allmyinteractionswiththemhavebeenpromptandveryhelpful.

(13)• Fantastic.Theyknowwhentobringexpertsinonparticularpackagespecificquestionsalso.(22)• Alwayshelpful,andIlikethefactthattheycheckwithyoubeforeclosingajobtomakesure

everythingisfixed.(34)• supportwasveryhelpfulinmakingnecessaryscriptstorunjobsonArcherandintestingthe

scriptstounderstandoptimalscalability(35)• Ionlyeverhadtousethehelpdeskonceasthedocumentationonthewebsitecoveredalmost

everythingthatIhaveneededsofar.ThetimethatIdidcontactthehelpdesktheresponsewasquickanditsolvedtheissuethatIwashaving.(40)

• Veryfastresponseandveryhelpfulatsolvingtheproblem.(45)• Veryhelpfulstaffandquickreply.(49)• Didwhattheysaidtheywouldwithinthetimeframetheysaidtheywould.(53)• Greatresponsesandresponsetime.(56)• Fastandhelpful.(58)• Onlinevideos/webinarsuseful,pleasekeepthesegoing.(60)• Makeitpossibletoreplythroughthewebpageforhelpdeskqueries.(66)• veryefficient(68)• Theresponsesarealwaysveryfastandhelpful.(73)• Actuallynothingtosay...youaregreat!Thequerysystemworksperfectlyandefficientlyasnever.

SinceIamaskedandsinceIdiditrecently,Iwouldstandardizetheprocedureofbuyingcomputationaltimeforexternalusersandcenters,tobeabletoaccessArcherbypaying(Imean,otherthanapplyingforgrants).(86)

• Waswaitinganadditional(beyondduedate)twomonthstohavequeryresponse.(92)• Thesupporthasbeenandcontinuestobeexcellent.WithoutthesupportgiventomeIwould

definitelyhavestruggledtoconductmyresearch.(97)• Allhelphasbeenpromptanddirectedtowardsimprovingmyexperience.Somesuggestionsto

improveperformancehavebeenofferedwithoutmyrequestandthat'salsoprovedveryuseful.(102)

• Ihaveusedthehelpdeskalot,withanythingfromquick1linequeries,tomorecomplicatedcompilinganddebuggingquestions.Theteamhavebeeninvaluabletomyresearch(103)

• Thehelpdesktendstobequicktorespond,efficient,andhelpful,evenwithfairlycomplexrequests.Greatjob!It'salwaysapleasuretoworkwiththehelpdesk.(105)

• Thehelpdeskcanalwayssolvemyproblems.Veryhelpfulandprofessional.(108)• Someofmystudentshavecomplainedthatresponsetimehasbeenslowinsomeinstancesand

norealsolutiontoaproblemwasprovided.(109)• Helpdeskalwaysreplyveryquicklyandareveryfriendlyandhelpful.(111)• Veryhelpful.(119)• quickandkindhelp!Thanks!(125)• Helpdeskhasbeenextremelyhelpful.Ireallyappreciateit.(126)• Oftenquickandhelpfulreplies.(127)• Thisworkswell.(133)• AsarecentARCHERuser,I'veaskedanumberofquestionsandrequiredconsiderablesupport

fromtheARCHERhelpdeskandtheresponseshavealwaysbeentimelyandincrediblyhelpful,thankyou.(141)

• Havebeenveryhelpfulwhenansweringallofourqueries.(142)• Helpdeskqueriesareansweredquicklyandinformatively.TheARCHERhelpdeskisoneofthe

mostusefulsupportservicesIhaveused.(160)• Theyhaverepliedpromptlyandhelpedmewhenrequired(178)• Usuallygood,inonecasedidn'treceivefurtherfeedbackandafteragoodwihle(aweekorso)

foundtheissuemyself,fedbackthesolutionbutneverheardback...(182)• great!(183)• Itseemstoworkfine.(188)

18

• Thehelpdeskhasalwaysbeenquicktorespondtomyquieries(203)• Veryefficient.(209)• Notusedsufficientlytoprovideusefulcommentary(220)• Ihaven'treallyuseditthatmuch.(225)• Responsewasveryquickandhelpful.(230)

19

Documentation• Moredocumentationonoptimisingcodeperformancewouldbeuseful.Particularlyfor

estimatingtheamountofAUstobeusedpriortorequestingtime.(9)• Needtoimproveinsomeaspectssuchasthereisnodetaileddocumentationforsubmittingjobs

withdependencyandrestartjobs.(10)• Ithinkthattheweblinkssometimesledmeincircles.(18)• Iverymuchenjoyedtheintroductorytutorialvideos.(19)• Possiblyabitmoreinformationonthebigmemorynodes.(34)• Ihaveonlyhadtousethehelpdeskoncetofindtheanswertoaproblemasthedocumentation

onthewebsiteisverygood.(40)• Thestructuringofthedocumentationisabitconfusingattimes:someinformationisintheuser

guide,someinthebestpracticeguideetcanditisnotclearwherethebestplacetolookis(41)• TheYoutubevideosarehelpfulandthereisalotofwritteninformationbutitcanbehardtofind

whatyou'relookingforinthewrittendocumentation.Atipsandtrickssectioncouldbehelpful.(43)

• Ihavefoundithardtofindinformationonthequeueingsystemandwhatqueueacceptscertainjobs.Particularlyrecentlywhenthelongqueuechangedtogreaterthan25hours-Ifoundthisbytrialanderror.(45)

• Thedocumentationisexcellent.(50)• Mostlyveryclear.(60)• AddhowtheusedAUsarecomputed(mycalculations,usingwhatIthinkhowitisdone,are

factor2off).havenotdiscussedwithhelpdesksofar.(68)• ThedocumentationofGlobusandGrid-FTPintheDataManagementGuidewouldbenefitfrom

anupdate.Moredetailedinstructionsoninstallationoftherequiredtoolsandcertificatesontheclientsystemwouldespeciallybehelpful.(71)

• Irecentlyfoundthetaskfarmjobsinstructionsextremelyhelpful.(73)• Itcouldbealittleeasiertofindthings,buttheinformationiswell-writtenandofferedatabout

therightdepth(74)• Itisverydifficulttofindoutwhichlibrariesareavailable.Itwouldalsobeusefultohavesome

moreinformationonhowthingsarelaidout(e.g.locationoflibrariesandincludepaths).(84)• Justperfect.Whatyouneedinaclearandconcisestyle.(86)• Iknowitisdifficulttokeepituptodatebutreferencingaversionofsoftwarethatisnolonger

availablecouldbebetter.Ifstatingversionnumbersandmodulesoneshouldreallyprovidethedatethatitwasvalid.i.e.thedefaultversionasof....(89)

• Thishasbeenforthebpsscriptsmainly(91)• Thereisnotaspecificdocumentationforlibrariesandmodulesinstalledonarcher.Sometimes

onewanttoknowspecificenvironmentalvariablesandpathsforspecificmodules,butthereisnodocumentationforthat.(94)

• itwouldbenicetohaveitorganizedinamoreuser-friendlyway,witheasyaccesstotheimportantinformation,butmaintaininghigh-levelhelp,possiblyupdatedwiththehelpofusers.Forinstance,aspecialCASTEPorVASPsectionwithbenchmarks,suggestionsandutilitiesfromthecommunitywillbegreat.Ithinkthesectionwithhelpforwindowsusertoconnectandbackupfilescouldbeupgradedwithsuggestionsforspeedingupconnectionanddatatransferaswellaslinkstousefultoolssuchasxming.(96)

• I'velookedatthedocumentationforsubmittingmultiplejobsusingarraysandfounditbaffling.Therewereveryfewcluesastohowtostartwithsubmittinginthisfashion.(102)

• Thereareafewtoomanyguidesavailable,suchastheQuickStartGuide,theUserGuide,andtheBestPracticesGuide.Thedivisionofinformationbetweenthesethreeisnotatallclear,especiallybetweentheUserandBestPracticesGuide.Thesewouldbebetteroffcombined.(104)

• Thedocumentsareveryhelpfulandcansolvemostofmyquestions.(108)• Thedocumentationisgood-clear,rightlevelofdetail,wideranging.However,it'snotalways

easytofindwhatyouneed-it'snotclearwhatisintheuserguideandwhat'sinthebestpracticeguide,soit'seasytomissthingsthatareinthebestpracticeguide.(114)

20

• Sometimesabithardtofindwhatyou'relookingfor.e.g.runningtheserialnodes,accessingthepostprocessingbitofarcher.(119)

• Noteasytogettheinformation...asearchsystemorabetterorganisationasawikicouldbeverygood.(122)

• Thedocumentationonthewebcanbebetter.TherehavebeenafewoccasionswhenIfollowedthedocumentation,didn'twork,consultedthehelpdesk,helpdeskofferedhelpandupdatedthedocumentation.Thatmeansthedocumentationwasnotinitiallycorrectornotupdated.(126)

• IcanfindmostthingsOK.TheonlythingthatIfindhardtofindisthedetailsofthecurrentlyavailablequeuelimits,andtherecommendedsizesofjobstomaximiseutilisation.(133)

• Idon'trelyonthedocumentationoftenanymorethesedays,butit'stothepointandusefulwhenIdo.(138)

• Availabilityofwikicontaininguptodateinformationoncompilationofsupportedmodellingpackagesisawell-definedmeritoftheARCHERdocumentation(146)

• Simplifydocumentationsothatit'seasytopulloutspecifictasks(e.g.compilationagainstspecificlibrariesetc)(149)

• Thedocumentationisscatteredthroughoutseveral"guides",butitisnotclearwhy.Theinformationcouldbemoreusefullyorganisedintoasingledocument.RecentlyIencounteredanissuewithrunningmixed-modeMPI/OpenMPjobswhencompilingsoftwarewiththeIntelcompiler,inwhichthejobwoulduseonlyoneOpenMPthreadnomatterhowmanywerespecifiedthroughthe'aprun'command.Thesolutiontothisproblemwasinthedocumentation,butlistedonlyinthesectionon"pureOpenMP"jobs,whichIhadskippedoverbecausemyjobwasamixed-modeone.Includingthisinformationinthecorrectsectionwouldhavesavedmetwoweeksoftestinganddebuggingmycode.(160)

• Again,usuallygoodbutnotalwayscatchingupwiththeupdatecycles(182)• Thedocumentationisgood,itwouldbebettertoprovidesomekeywordsortingfunction.(183)• WheneverI'veneededtouseit,I'vefoundwhatIneed.(188)• Theuseofthequeuesisstillnotentirelycleartome.Istheallocationofnodesdependenton

totalresourcerequest(i.e.nodehours)orjustnumberofnodesusedinajob.i.e.forlongrunsthatneedtoberuninsequence,isitbettertoruntheseinthelongqueue,orinthestandard?Thisismainlyrelatedtoaproblemforwhenyouneedoneexperimenttoruninconsecutivedependentchunks(i.e.climatemodelexperiments),ratherthanhavingmanyindependentexperimentsthatcansubmittedtothestandardqueue.Sometimes,iffeelslikeotherusersareatanadvantageastheycanhavemanyexperimentsrunningatonce,whileasinglerunagesinthequeues.(203)

• Ifoundalotofusefulthingsithedocumentation.StillIhadtoasktheusersupportforsomedetails,butmostusersprobablydon'tneedthose.(205)

• Thedocumentationisveryusefulforme,especiallynewtoARCHER.(209)• Generallyhelpfulsofar,onlybeenauserforaboutamonth.(211)• Itdidn'tmentiontheworkdirectory!(215)• Notusedsufficientlytoprovideusefulcommentary(220)• TheguidanceonusingDDTwasalittleunclear,nothelpedbytherebeingboth"ddt"and

"allinea"modules.(225)

21

Website• TheonlywayIcanfindthingsisthroughsearch,whichIsupposeisokay.(16)• TheservicestatusinformationisnotasdetailedasitwasonHECToR,whichprovidedacountof

whichqueueswerebusiest.Thiswasusefulwhenconsideringwhethertosplitlargejobsorrunwhole.(28)

• IwastryingtofindinformationonhowmanyAUsusedcertainsoftwarepackagemodules(e.g.CASTEP)onARCHER,togetaroughestimateonhowmuchARCHERtimegoestowardsCASTEP,LAMMPS,etc(beingawarethatpeoplemayforgotheARCHERmodulesinfavouroftheirowncompiles.Anyway,Icouldnotfindsuchastatisticonmoduleuse.(37)

• Safecouldbeimproved.Therightbuttoncanbehardtofindtoperformanoperation.Fornewuserstogainprojectaccessitisnotobviouswhattheuserortheprojectmanagerwillsee/do.(60)

• Navigationcouldbeeasier,butagoodrangeoffunctions.(74)• It'susuallyeasyenoughtofindthematerialIneedviagoogle/searchonwebpage.(75)• Findinginformationiscomplicated.ArcherSAFEusedtobeparticularlynon-trivialtolocate.(77)• Itismoredifficultthannecessarytoaccessthecoursedescriptionsfromthecourse/event

schedulingpage.Description/registration/schedulinginformationshouldbeincludedinthesameplace.(80)

• itwoulddefinitelybenefitofanupdateinstyle(abittoomuchold-fashioned)andcontent.(96)• Sometimesthecompilation/jobscribsareoutofdatebuthelpwasalwaysathandwhen

contactingthehelpdesk.(97)• Thetextisalittleconfrontational,butnavigationisgood.(102)• Ifindtheoverallstatus/livereportsveryhelpful(103)• Itwouldbeveryusefultoshowinformationoncompletedorexpiredallocationsontheuser-

specificpage.Onceallocationsarefullyusedorexpired,theydisappearcompletelyasiftheyneverexisted.Suchinformationisusefulwhenwritingproposalsandisalsousefulwhenanallocationstopsbeingactivebeforeauserexpects.(104)

• Sometimesabithardtofindwhatyou'relookingfor.e.g.runningtheserialnodes,accessingthepostprocessingbitofarcher.(119)

• Atextbookhasanindex.Websitesrequireyoutoknowwheretheinfoisbeforeyoustartlookingforit.Archer'swebsiteisnodifferent.(124)

• Servicestatusdoesnotalwaysshowstheproblem.BylookingatthegraphsdownthepageIcanseeit'squitecrowded,butitdoesnotprovideanyclueabouthowmuchlongermyjobneedstobeonthequeue.(126)

• Thiscanbetrickytonavigate-especiallytheSAFEtool...notentirelyclearhowcertainprocesses(e.g.getanewpassword)aremeanttohappen.(137)

• Thewebsiteisveryuseful.However,theremainingbudgetsectiononlyseemstoupdateoccasionallywhichhasledtoover-usingtheavailablebudgetonoccasion.(142)

• Navigatingbetweenthemainsiteandarcher-safecouldbeeasier.(152)• Someusefullinksandinformation,sometimeshardtofindcertainpagesforexample,ifthere

wasalinkinanemailandIwantedtolookthepageupagainlaterwithoutthelinkfromtheemail,usingthesearchtoolonthearcherwebsiteitcanbehardtofindthatpagesometimes.(181)

• great!(183)• Somethingshardtofindbecausenotsurewhatwordsyouuse.Onceaccustomedtothe

languageallwasfine.(187)• Nicelayout.(209)• Good(211)• Couldbemadeclearer,especiallythedocumentation,currentformatlooksmessy(214)• I'vealwaysfoundthewebsitetohavewhatIneed.Inpart,thisprobablystemsfromthelimited

wayinwhichIneedtointeractwithARCHER(e.g.runninglargeoceanjobsonit).(220)• Uptodate.cool!=D(229)

22

Training• Iappreciatethehighfrequencyofwebinars.(19)• notused,butnowIcannotunclicktheanswer.(37)• Ihavenotcompletedanyofthese,ofmyownfault,theyarewelladvertisedthroughthemailing

lists.Onlygivena3becauseIhavenevertakenadvantageofthisfacility.(40)• Audioqualitycanbeanissuewhenfollowingvideoguides.(43)• Therangeoftrainingavailableisgreat.(50)• Onlinevideos/webinarsuseful,pleasekeepthesegoing.(60)• IattendedtheScientificPythoncourseparticularlyforthesectiononintegratingPythonwith

otherlanguages,e.g.C,Fortran.However,asthecoursehadrunovertimethissectionwasalmostcompletelyskipped.Iwouldbeinterestedinmoremedium-levelcourse,orhands-onworkshopsforpeoplewithsomeHPC/codingexperiencebutnotadvancedusers,onthingslikedatamanagement/analysisusingbatchprocessing,particularlyhowtousedifferentsoftwarepackagestocompletedifferentpartsoftheanalysis.(Iamawareofhowtodothiswithbash+gnuplot,buthavenoideahowtointegratesay,datageneratedinfortranwithanalysisdoneinMatLab/Mathematica/Origin.)(80)

• UnfortunatelyIdidnotusethem,butIwillgiveitagointhefuture.Ibelieve,lookingatthetrendsaround,thatdeliveringlecturesonbasicandadvancedFORTRAN(orCandC++)andontheuseofGPGPUandCUDAwillbeenoughfornormalusers.Furthermore(andIamnotsurehereagainalreadyyouofferthispossibility)youcanincreasetheinterestofprogrammingcourses(sometimesboring)ifyouaddspecificonesonapplications(Iamthinkingaboutfluid-dynamics,astrophysics,high-performancemoleculardynamicsetc...)(86)

• HavenotattendedsomeastheyarebasedinEdinburgh.(89)• Iwouldhavelovedtoparticipatetothecomputationalsoftwaretraining,butitisdifficultforPIs

toattendafullday(ormore)oftraining.EventsinLondonoreventsonlineworkbetterforme.Iwouldincrediblyappreciateifthematerial,examples,tutorialsandpresentationsvideosandslideswouldbemadeavailabletotheARCHERcommunity,especiallytousersthatcouldnotattendthesessions.(96)

• IattendedthePythonandFortranTrainingsessionandtheywereveryuseful.Mayberunningdifferentlevelsoftrainingmightbeworthwhileinthefuture.(97)

• TheylookveryinterestingalthoughIhaven'tactuallyattendedany.Moreonpython.(119)• Veryinformative(121)• Iattendedone.itwasverygood.Ijustfeltnotenoughdaysandpracticalworks.(122)• IhaveonlywatchedoneTutorialfromtheARCHERwebsite.TherestinformationIneededI

foundinwrittenform(ontheARCHERwebsite).(128)• Iwishsomeofthesewereeasiertogetto...(133)• MPIinPythonmightbeausefuladditiontotheMPIcourse(whichcurrentlyincludesCand

Fortran)(137)• Ihavenotreceivedrecenttraining,soIgiveaneutralscore.(138)• IhaveattendedseveralTrainingsessionsandtheyhavebeenreallyusefultome.(147)• Ihavenotattendedtoanyofthewebinars,asIhavelongexperienceinHPCand

supercomputing,buthavelookedatthewebsiteandthematerialslookgreat.However,youcouldtakeastepforwardanddosomeworkshopsonnumericalapproaches,specificlibrariessuchasPETScorTrilinos...YoucanfindinspirationintheworkshopsandtrainingsessionsthatareofferedintheSuperComputingconferenceinthestates.(153)

• IfoundtheAdvancedMPIcourseabittoofastpaced+toomuchnewinformationcovered.Thiswasfedbackonthefeedbackformforthecoursealongwithsomesuggestionsforimprovementetc.(156)

• Itisveryusefulindeed!thanks!(183)• -Foryourinfo,Ihaven'tattendedanyARCHERtrainingsofar,butitisusefulforARCHERto

providethese.(184)• Didn'tdothismyself,butheardgoodstoriesfromothers.(187)• Couldyouimproveyourannouncementsaboutyourtrainingcoursestobemoreinternational.I

meantherearemanyinternationalstudentsneedyourcoursesbuttheydon'tknowaboutyourservices.(208)

23

• Itcoversmanyusefultrainingcoursesforme.(209)• Ihaven'tbeentoany,butI'veheardgoodreports.(225)

24

Webinars• Thiswasveryuseful.(18)• Someoftheintroductionstoagiventopicwereabittoobasicforme,butIsupposethecluewas

inthetitle...(53)• Thewebinarsoftwareisabitclunky,asimplewebinterfacewouldbenice.(74)• TheHPCoverviewcourse(recommendedasthestartingpointfornewusersplanningtotakethe

HPCdrivingtest)washardtofocusonremotely.Thereweretechnicaldifficultiesatthebeginning(videosonlyshowedpresenterwithslidesinthebackground,ratherthanavideoofthepresentationnexttotheslidesthemselves).Thepresentersdidagoodjobofinformingparticipantsoftheissuesandupdatingtheslidesviathecoursemailinglist,butIfounditdifficulttosetasidethetimeduringtheworkdaytowatchthe3hourpresentationaftertheineffectivefirstsession.(80)

• Oftencannotattendastheyclashwithotherwork..HaveusedYouTubechannel.Workedwellforme.Althoughononeoccasionthepresentationwasdifficulttofollowduetofontsize.(89)

• GoodforaskingquestionsbutgenerallyItendtoswitchoffhalfwaythrough.(97)• Somewereuseful,someabittoobasic.Thearchiveofmaterialsfrompasttutorialsisveryuseful.

(114)• It'sgreatthatthewebinarsareavailableonyoutubeaftertheeventdate.(158)• Usefulandinteresting!(183)• Didn'tdothismyself,butheardgoodstoriesfromothers.(187)• Lateafternoontimeslotsformanyofthemissomewhatinconvenient.Otherwise,excellent(198)• Veryuseful(209)

25

OnlineTrainingMaterial• Honestly,IonlydidthedrivingtesttogettheextrakAUs,levelwasverybasic.(20)• Possiblyhaveextensionstothedrivingtestformorecomplextasks.(34)• OneofmystudentsrecentlyworkedthroughtheinitialtrainingforArcher'snewuserandhewas

verysatisfiedbytheinstructions.(73)• Needtobemorecarefulwhenreferencingspecificversions(89)• OnlinedrivingtestisagreatwaytogetpeopleinvolvedinArcherandHPC!(115)• OnlinedrivingtestisagreatwaytogetpeopleinvolvedinArcherandHPC!(116)• ItooktheOnlineDrivingTest,andthinkthatitisagreatideatoasitintroducesthefutureuser

nicelywhilemotivatinghim/hertolearnhowthemachineisconfiguredbeforeusingit.(153)• Itisextremelyusefultomycodingandparallelismimplementation.(183)• IdidtakethedrivingtestIthink,butIcan'trememberitbeingparticularlyusefulI'mafraid.(188)• extendawardlimits(twice-triplekAuallocations)ifresourceswouldallowthat.(195)• Needmoreexamplesforexplanation.(208)• Itisveryusefultoprovidestep-by-stepinformation.(209)• Drivingtestwasgood.IrepeatedittillIgot100%,couldn'tfigureoutwhetherthiswasrequired

ornot,don'tthinkIreaditanywhere(211)

26

Othercomments• OverallmyexperiencewiththeARCHERservicehasbeenverypleasant.(13)• I'vehadafewMAJORproblemswithARCHERoverthepastyear.First,thequeuesystemis

absolutelydisgraceful.Havingtoqueuemorethan48hoursforasmall,3-minuterunisridiculous.Ifoundthatsubmittingtheseshortrunstothelongqueuesolvedmyproblem,whichIrealizeisnotwhatyouhadinmindforthelongqueue,butwhenyouareundertimeconstraints,sometimesthisisnecessary.ThefactthatARCHERwasdownforthebetterpartoftwomonthsisanationaldisgrace.YouaresupposedtobetheUK'sbestandfastestsupercomputer.ThatwasthekeytimeformetorunsimulationsformyPhD.HavingmymodelspostponedfortwomonthsnearlycostmemyPhDasIhadnooptionofextendingmyfunding.Ihopeyouhavelearnedfromthoseproblemsandhaveasysteminplacetopreventthisfromhappeninginthefuture.(16)

• Itwouldbegreattobeabletousetheshortqueuealldaylongandnotonlybetween09:00-16:00.(19)

• Archerreallyisagreatservice.IhaveusedothersupercomputersbuttheassistanceandtheguidanceonArcherissignificantlybetter.(27)

• Thestandardqueuesareoftenpainfullylong.Myjobstake6hourstoruniftheyaresuccessful,butinthelastfewmonthshavetypicallyqueuedfor12-24hoursbeforestartingtorun.24hoursisalongtimetowaitforsomethingtocrashassoonasitstartstorun,whichisacommonoccurrencewhentestinganewsetup.Isthereanywaythatthequeuingsystemcouldbechangedtoreducewaitingtimesforuserswithrepeatedlyfailingjobs?(38)

• Itisaterrificresource(42)• Theonlyfrustratingthingiswhenthequeuesalwaysbecomeblockedjustbeforearenewalof

budget.IknowthereisnotmuchthatcanbedonetopreventthisbutitstheonlyfrustratingthingaboutusingArcher.(45)

• TheonlinedrivingtestwashowIobtainedaccesstoARCHER,andIthinkitsabrilliantideaforwideningaccess.ThishasallowedmetoputtogetheraneCSEapplicationforthenextcall,andhopefullyacceleratemyresearchinfuture.(50)

• Archershouldmovetowardshigh-throughputcomputation,muchmoreusefulforthevastmajorityofresearch,thanthecurrentsinglempiprocesspreference.(76)

• Pleasegivemetheprizesof2000kAUonArcher;-)!!!(86)• Justtore-iteratethehelpdesk/supportmakesarchersobeneficialandhasreallysavedmean

immenseamountoftimewhentryingtocompilesoftware,orwhenaproblemhasariseninacalculation.(97)

• Ihaveafewcriticalcomments.1)Whyisthe'short'testingqueueonlyopenfrom9-5UKtime?Iamnotalwaysabletodomytestingworkduring"normalbusinesshours".IhavetobeflexibleastowhenandwhereIwork.SometimesIneedtoruntestswhileworkingfromsomeothertimezone-whyshouldIbepenalisedforworkingfromabroad?Havingatestingqueuethatisonlyopenfrom9-5hampersmyprogressandisveryfrustrating.Itisalsoveryunusual-IhaveworkedonalotofHPCsystems,andARCHERistheonlymachineIhaveencounteredwithsuchastrangetime-limitedqueue.Itreallyisbizarre.2)Theattitudethat"ARCHERisnotatestingplatform"doesnotrespecttheneedsofocean/atmosphere/cryospheremodellers.Ourmodelsare*always*indevelopment.Wetypicallyhavetocompileandrun*each*case(e.g.differentcompile-timephysicalpackages,adjointsetups)andtestit.Whyshouldn'tIbeabletotestmysetupwithashortrun?Mytypical"testrun",justtomakesurethatmyinputsareconfiguredcorrectly,islessthan1000coresforabout10minutes.Ishouldn'thavetowaitonthequeueforhoursjusttoperformthissmalltest.Whydoesn'tARCHERrespectourneedsforthisveryminimalkindof"testing"?Inshort,Ireallyneedalarger"short"testingqueuethatisopen24hoursaday.3)Whatcanbedoneaboutthe"rushtoburnhours"inFebruaryandMarch?ARCHERisnearlyunusableforthosemonthsbecausebothNERCandEPSRCusersaretryingtouseupalltheir

27

hoursfortheyear.Whycan'tthoseallocationyearsstartandendatdifferenttimes?4)It'stoocumbersometoshareinfobetweenprojects(e.g.n01,n02).Ihavecollaboratorsinbothn01andn02,andsometimessharingfilesonARCHERbetweenprojectsisarealheadache.Therearetoomanywalls.5)PleaseclearlypublishanychangesinyourSSH/SCPpolicies.Sometimeinthepastyear,adecisionwasmadetodisallowoutgoingSSHconnections.Thismadeitimpossibleformetodomywork,sinceIuseTAF(aFastOpt.comproduct),andTAFusesoutgoingSSHconnectionsforserver-to-servercommunication.IwastedalotoftimewonderingwhyTAFwasn'tworkingforme.TheARCHERhelpdeskdidrespondtomyrequestfairlyquickly,andtheissuewasfixedoverthenextcoupleofweeks.Ifpossible,pleasebemoreexplicitaboutthesethingsinthefuture.Forexample,perhapsinsteadofjustkillingtheSSHsessions,youcouldhaveanerrormessageintheterminalexplainingthatthesessionisbeingkilled(andwhy)?(105)

• OntheSAFEwebpage,itisveryusefultohavethetablewhichsummarisestheprojectsyouareinvolvedin.Remainingbudgetisreportedonthistable,anditwouldbereallyusefultohavereportedhowmuchofyourtotalallocatedtimethiswas,andwhenitrunsout.(111)

• Itwouldbeveryhelpfultogetawarningmessageifyoutrytosubmitajobtotheshortqueueoutofthe9-5hours;alsoifyousubmitajobbefore5pmwhichislikelytobestillrunningat5pm.It'seasytoforgetaboutthelimitedhoursforthisqueue.(114)

• Itwouldbeinvaluabletohaveaseparateresourcewithbetterqueuesforsmallerruns.(119)• Ifthereisawaytoincreasebandwidthforthefilesystem,pleasedoso.I'vehadseveralcrashes

duetoextremelyslowread/writespeedsformyjobs,andpostprocessingdataismuchsloweronARCHERthanonothermachines.(127)

• IfindtheARCHERservicetoberesponsiveandhelpful,easytouse,andflexible.Excellentservice.(132)

• Ihavefoundthatmanagingthebudgetofprojectshasbeenalittledifficultduetoadelay(~24hours)inupdatingtheremainingbudget.Ihavefoundthatthequeuingperiodscansometimesbequitelong.IunderstandthatreducingthewalltimeofsimulationsgivesprioritybutinmyworkItendtorunrelativelyfewlongjobssoendupwithlongqueuetimes.Theuploadanddownloadspeedoffilescanbeincrediblyslowascandeletingfiles.ForsimulationsgeneratingTBsofdatathiscantakeareallylongtime.(141)

• Downloading/uploadingdatafromArchercanseemverysloworevenstallattimes.Whenthishappensitisoftenquickertostopthedownloadandrestartitagain.Furthermore,deleting/copyingdataonArcherseemstobeslow.(142)

• Itwouldbegreatifthereisfurthersupportforshorterdurationdebugqueues(147)• welldone!(148)• Theserviceisworsenowthaneverinthepastduetothejobturnaround.Ireallyhopethatcan

besortedsoon.ItisironicasArcherhasbeenupgraded,buttheserviceisworse.Therestofthefacility(machineandstaff)isexcellent.(151)

• Keepuptheexcellentwork&Thanks(152)• Pleasecommunicatesystemissuestousersmorefrequentlyandinamoretimelymanner.I

understandthatthereisabalancetobestruckbetween"bombarding"userswithemailsandkeepingusersinformed,butARCHERalreadyhasdifferentlevelsofmailingliststhatcouldbeusedforthispurpose.Overthepastyear,therewereseveralinstancesofovernightsystemfailures/machinerebootsthatwerenotcommunicatedtousers.Thoseofuswhohadjobsrunningatthetimereceivedcrypticerrormessages,whichinitiallyledustoblamesoftwareerrorsratherthanahardwarecrash.OnlyafteremailingtheARCHERhelpdesktoinquiredidwediscoverthattheentiremachinehadgonedownovernight!Hardwareerrors(particularlydisks)andsubsequentmaintenancehavemarredperformanceofthesystemoverthepastyear.(160)

• 16joblimitonqueueisfrustrating--Iunderstandtheneedforthrottling/controllingthenumberofjobs/coresactiveperuser,butthequeueshouldhandlethat,notrejectjobsbasedonahard(andquitesmall)limit.(167)

• Wonderfulservice,myresearchwouldn'tbethesamewithoutit!(169)• Anexcellentservice!Thankyouforallyourhadwork!(172)

28

• Thetimetakentoqueuejobstorunisquitefrustrating.A1.5hrjoboftenneedstobesetupthedaybeforetorun.(175)

• someofthemoreusefulworkshopsfillupreallyquickly,e.g.pythonrelatedones-woulditbepossibletohavetheseclassesmorethanonceinayear?(181)

• Great-Thanks.Keepgoing.(183)• -WerunOpenFOAMsimulationsthatproduceafewthousandsoffiles.Unfortunatelythisisa

problemofOpenFOAM.HoweverwhenafewresearchersonaprojectareusingARCHERintensively,wefindthatthelimitofallowablefiles(3million)onARCHERissometimesexceededandbringsahalttoalloursimulations.Thiscouldbeimproved,bysayallowingthesespecialcasesanincreaseinthislimit.(184)

• Myresearchfieldisdataassimilation,inwhichhugegeosciencemodelsofe.g.theatmosphere,withupto10^9variables,arecoupledtohugedatasets,upto10^8every6hours.SincetheseproblemsarenonlinearMonte-Carlomethodsareneededforthis.Present-dayARCHERisnotfitforpurpose,andonlycentresliketheMetOfficecandothisatthemoment.However,theyneedtheacademiccommunitytopushthequalityoftheirmethodsforward,bothintermsofaccuracyandefficiency.IntheUSAtheacademiccommunitydoeshaveaccesstomachinesthatcandothis,weintheUKarebehind,whileatthesametimethenewestandmostexcitingnewideasarecomingfromtheUK,throughclosecollaborationofmathematicians,statisticians,andgeoscientists.Thisisnotonlyforweatherpredictionandbetterunderstandingoftheweather,butalsoforoceanography,biology,ecology,land-surfacescience,climatemodelling,etc.Thereareveryexcitingandinnovativethingswecoulddowithabiggersystem.(187)

• Changethewaythesharednodeswork,oratleastmakesomenon-sharedserialnodesthatcanbeusedforcompiling.Itisverydifficulttocompilewhenyoudon'tknowhowlongthingswilltaketocomplete.ThisisparticularlyproblematicwhenrunningtrainingcoursesonARCHER,whichIdoeveryyear(UKCA).Whilewecanreserveparallelnodes,Ifirstgetthestudentstocompile.Whilethisshouldtake10minutes,theyallthentime-outafteranhourasthereare20+peoplehittingthesharednodes.Itdoesn'tjustwastetheirtimeandmine,butalsowe'repayingforparallelnodesthatwecan'tuse.Thisismybiggestproblemwiththeserviceasitiscurrentlyconfigured.IfIcouldcompileintheparallelqueue,orreserveserialnodes,thiswouldimprovethingsgreatly.WhenIcalledthehelpdeskaboutthistherewasnothingtheycoulddo.(188)

• TheUKNationalSupercomputerserviceisanessentialtoolinsupportofscientificresearchwithintheUK.(201)

• Pleaseprovidedataanalysistutorials(foratmosphericmodeloutputse.gUKCA)(207)• Goodexperience.(209)• Pleaseincreaseuser'squotaandgiveusmorespace.Muchtimeiswastedinbackups(229)• Standardqueuewaitingtimescanbeexcessive:Ihaverecentlybeensubmittingjobsofabout11

nodesfor12hours,andthesehavequiteregularlyspent2-3daysinthequeue.Irealiselargerjobsareprioritised,butafasterturnaroundonthesesmallerrunswouldbeuseful.(230)