page 3

news from the EGI communityISSUE 30MARCH 2018

TOP STORIES

page 5

MORE

Advanced Computingfor Research

www.egi.eu

The new EGI Operations Managerpage 1

Inspired

Pollution forecasts in Bulgaria

EXTraS portal: hidden treasures in the sky

02 EGI at the first EOSC-hub week

page 4

EOSCpilot Science Demonstrators

07 EGI & EUDAT close collaboration

08 eInfraCentral introduces their service catalogue

The eXtreme Data Cloud projectpage 6

Happy Spring! This edition of the newsletter is focused onnew EGI recruits, use cases and contribution to projects.

Your feedback and suggestions are always welcome!

Send an email to Sara & Iulia at:

press@egi.eu

Welcome to issue 30!

Introducing the new EGI Operations Manager

In March 2018, Matthew Viljoen took over as EGI Operations Manager, leading theOperations team. In this article he introduces himself and outlines the team’sstrategic priorities for 2018.

1Inspired // Issue #30, March 2018

As someone with a backgroundin Service Delivery and Operations,I was delighted to be offered theposition of Operations Managerwithin the EGI Foundation, andlook forward to the challengesahead as EGI continues to buildon its strength into the future.

Prior to working with EGI, I hadbeen working at the UK Ruther-ford Appleton Laboratory (RAL)for twelve years. Here I wasinitially managing and servicesfor the UK National Grid Serviceand for the EGEE project. I thenoperated the petabyte scale datastorage service for the UK WLCGTier 1, a principal site for makingCERN data available to UK highenergy physicists. At RAL I alsodeveloped and moved intoproduction a comprehensivenew data storage and accessservice for the Diamond LightSource UK national synchrotron

which now serves multiplepetabytes of storage.

In 2015, I started working at EGIand led the Platform Develop-ment work package in the EGI-Engage project, which identifiedmissing functionality in the HTCand cloud computing as well asfederated data EGI services.

A new challengeAs EGI Operations Manager, Iwill be leading the coordinationand evolution of the federatedoperations team to ensure thatit develops an ever stronger and

consistent high quality globalservice offering, meeting theneeds of researchers regardlessof their discipline.

This will require forging stronglinks with national e-Infrastruc-tures and thematic researchinfrastructures, and streamliningall operations.

Leading the Operations Team, Iwill endeavour to continue toimprove the quality of servicedelivery. This follows on fromthe work done in 2017 withinEGI Foundation to obtain ISO/IEC

More information

Matthew Viljoen is the newEGI Operations Manager.

Bruce Becker is the new EGISenior Operations Officer.

2Inspired // Issue #30, March 2018

9001 and ISO/IEC 20000certification of EGI ServiceManagement processes.

This will ensure that we deliverthe best possible services,working alongside the significantcontribution and hard workfrom service and resourceproviders within the EGI Fede-ration. We have always takendata privacy seriously andinformation security remains apriority for the Operations Team.

We are working to ensure thatservice delivery will continue inthe light of the General DataProtection Regulation (GDPR)coming into force in May 2018,by making sure all requirementsof the regulation are met andreflected in service deliveryagreements. In preparation forthis, all the EGI Operations Teammembers have obtained ISO/IEC27001 certification in Informa-tion Security Management. Moregenerally, during 2018 we will beworking to improve the security,general quality and usability ofthe EGI Federated Cloud service.

Looking ahead2018 also brings us into anexciting new phase of EGI as it isleading the development ofEOSC-hub, the first steps of theEuropean Open Science Cloud.This aims at harmonizing majorEuropean e-infrastructures suchas EGI and EUDAT to oneconsistent service offering for allEuropean users of publiclyfunded e-Infrastructures for re-search and academia.

The project is also integratingservices developed in theINDIGO-DataCloud project andservices operated by membersof the 18 Research Infrastruc-tures part of the project.

Within EOSC-hub, I am leadingthe work of coordinated opera-tions and federated servicemanagement, creating an ITservice management frameworkcovering all aspects of servicedelivery within the EOSC-hub.

The EOSC-hub project itself iscoordinated by the EGI Founda-tion and we will draw from our

extensive experience in runningthe largest academic e-Infras-tructure to make it a success.

Achieving these goals will bechallenging but the EGIFoundation Operations teamincludes people with extensivetechnical and skills, working inmany domains.Finally, I'd like to welcome theOperation’s team latest recruit,Bruce Becker, who joined theteam after a long history of closecollaboration with EGI and haspreviously worked at CSIRMeraka in South Africa.

EGI highlights at the first EOSC-hub week

The first EOSC-hub week is justaround the corner, taking placefrom 16 to 18 April in Málaga,Spain. The EOSC-hub week willbe made of two events: twopublic days (16-17 April, open toeveryone) and one all-handsmeeting (18-20 April, open onlyto EOSC-hub partners).

The public days welcome theparticipation of service pro-viders, representatives of theresearch communities andpolicy makers engaged in theestablishment of the EuropeanOpen Science Cloud.

EGI will participate to theprogramme with the followingsessions and workshops:

• EGI in the EOSC ERA: status,impact and future steps - thissession will outline EGI’s roleand next steps within theEuropean Open Science Cloud

• National e-Infrastructures:status reports and future plans- this session will allow nationalservice providers and Europeanresearch organisations tohighlight their success storiesand future plans.

• Federated Cloud status andevolution: the session will be adiscussion around the EGIFederated Cloud model andlatest technical developments.

The draft programme of theevent and instructions forregistration are available online.

More information

EOSC-hub project

http://www.eosc-hub.eu/

Air pollution has become a realconcern in the past few years inEurope and it poses a growinghealth risk in many of our cities.According to a 2016 study by theWorld Health Organisation, alot of respiratory diseases suchas lung cancer and chronicpulmonary disease have beenassociated with air pollution.

Knowing more about air pollutionand how it affects our day-to-daylives is key to improving our well-being and quality of living.

Ivelina Georgieva and the teamof researchers at the BulgarianAcademy of Sciences wanted toknow more about this issue andstudied the atmospheric struc-ture of Bulgaria and its capitalSofia, for a period of sevenyears, from 2008 to 2014. Theywanted to determine the AirQuality Index (AQI) of Sofia andwhat factors contribute to theair pollution of the city.

Air Quality Index is a trait of theatmosphere that directly reflectsthe impact of the air pollutantson the health and quality of lifeof a given population.

To study the air quality of Sofia,Georgieva and her colleaguesperformed extensive numericalsimulations of the atmosphericcomposition of the city, usingup-to-date modelling tools.

The calculations had largecomputing requirements andGeorgieva decided to use theresources of the BulgarianInstitute of Information andCommunication Technologies.

“For example, 1 day simulationwith meteorological model at 16CPU needs a runtime of 3 hoursand 530 MB output storage space,and 7 years simulations require 1year runtime“, Georgievaexplains. “Without the HPC clusterand the High-ThroughputCompute environment we wouldnot be able to do all this – installand run the models, organise thecalculations in jobs, the neededtime for the results and of coursethe needed storage capacity.”

The results of the simulations,published in the Cyberneticsand Information Technologiesjournal, give an extensivedescription of the atmosphericcomposition – its behaviour,origin and health impact.

Georgieva and her teamdiscovered that the pollutantsaffecting air quality the most aresulphur dioxide and fineparticulate matter, with road

transportation being one ofmain factors that contribute tothe development of air pollution.

Part of this work was conductedduring the EGI-InSPIRE projectand was based on the BulgarianChemical Weather ForecastSystem: a system of forecastingair quality over Bulgaria.

3Inspired // Issue #30, March 2018

Pollution forecasts in Bulgaria

A recent study powered by computing resources of the Bulgarian Academy ofSciences shows the atmospheric structure and air quality index in Bulgaria.

More information

The study was powered byHigh-Throughput Computeservices provided by theBulgarian Academy of Sciences.

BAS is the representative ofBulgaria in the EGI Council.

4

Almost all astrophysical objects,from stars in the surroundingsof the solar system to super-massive black holes in the nucleiof very distant galaxies, display adistinctive variability in terms offlux and spectral shape.

Most of the fluctuating pheno-mena have been discovered withmodern soft X-ray observatories,but a lot of information availablein data archives is still unexploited.

This is where the EXTraS projectstands out. Like a treasure-hunt,EXTraS has been harvesting theunexplored information buriedin the data collected by ESA’s X-ray space observatory XMM-Newton, during 16 years ofobservation. The results of theproject, together with the newanalysis software, are now re-leased to the scientific commu-nity. The software is importantfor enhancing the potential ofdiscovery of the XMM-Newtonmission, especially because itcollects new data every day.

Another success of the EXTraSproject is the resulting sciencegateway – or the EXTraS portal –a web platform that givesscientists the possibility to runanalysis on the whole XMM-Newton science archive (XSA).

The project’s ICT community washeavily involved in thedevelopment of this portal: theteam focused on closing the gapbetween the community-specific

The EXTraS portal: hidden treasures in the sky

Daniele D’Agostino and Giuseppe La Rocca on the successes of the EXTraS projectand how EGI is supporting it

presentation layer, the general-purpose middleware and “fabric”layers of the science gateway. Asa result, the frontend of theportal was set up with PortalTS,a modern web that is based onup-to-date technologies andarchitectural approaches such asthe use of microservices.

With regards to the computecapabilities, the EXTraS portalrelies on the EGI Cloud Computeservice within the extras-fp7.euvirtual organisation and is freelyavailable to all scientists.

The subsequent computationalinfrastructure is made of fourcomponents: the EGI Appli-cations Database (AppDB), theeToken Server, the CERN VirtualMachine File System infras-tructure (CVMFS) and the cloudproviders of the EGI Federation.

All the technical details of theEXTras portal architecture arepresented in the project’s latestpaper: A science gateway forExploring the X-ray Transient andvariable sky using EGI FederatedCloud, published in Future Gene-ration Computer Systems journal.

Inspired // Issue #30, March 2018

Thanks to the EXTraS project,scientists now have access to amassive discovery space whichallows them to explore newquestions in the astronomy andastrophysics research fields.

More information

Daniele D’Agostino isPartner & Science GatewayManager at EXTraS.

Giuseppe La Rocca iseToken Server Manager ofthe project & responsiblefor liaising with the EGIFederation cloud providers.

The EXTraS project is led byAndrea De Luca.

EXTraS project

http://www.extras-fp7.eu/

5Inspired // Issue #30, March 2018

The EOSCpilot project hasrecently passed its mid-termreview in Brussels, receivingpositive feedback from the panelof experts. One of the keyactivities of the project was thesupport of science demonstratorsthat act as early adopters of ‘to-be EOSC’ services.

By the end of the first projectyear, 15 science demonstratorswere selected from diversescientific domains. Each receivedadvice on available services andwas provided with technicalsupport and training to integratescientific applications withservices that fit their purpose.The demonstrators providedfeedback to the project andpromoted their work to variousaudiences beyond the consortium.

During this period, the EGIFoundation and several mem-bers of the EGI federation wereinvolved in the selection andsupport of science demon-strators. Liaison with service andtechnology providers happenedboth within Europe and over-seas, e.g. with Compute Canada.

Here is the status of the first 10science demonstrators:

• TextCrowd: a virtual researchenvironment for implementingthe Natural Language Processing(NLP) encoding/metadataenrichment of textual archaeolo-gical reports has been set up.

The service allows researchersto store textual documents in acloud folder, perform NLPoperations, trigger the semantic

enrichment of the text and getinformation in RDF format.

• PanCancer: ported the Butlerapplication for large scaleprocessing of cancer genomesonto Compute Canada and sitesof the EGI Federated Cloud.

• Photon/Neutron: two applica-tions (OnDA and Crystfel) werecontainerized & tested on HPC &cloud platforms at DESY.

• DPHEP: assessed whetherCERN’s own preservation systemcould be replaced with ‘out-of-the-box’ services combined ofCVMFS, B2SAFE and TrustworthyDigital Repository. Data inges-tion/replication of small datasetsfrom CERN to CINES was put inoperation with an average speedof 800Mbits/sec. Data retrievedfrom CERN was ingested intoCINES & replicated to CINECA foropen access.

• ERFI: Used the EGI Open DataPlatform to develop a data inte-gration framework for sharingdatasets between the ICOS andthe IS-ENES RIs.

• EPOS/VERCE: Integrated threecloud providers of the EGI Fede-ration with a Virtual ResearchEnvironment to compute arealistic scenario of earthquakesshaking using misfit calculations.

• PROMINENCE: Deployed SLURMclusters on the EGI FederatedCloud to run containerized MPI-based applications on a hybridEGI-commercial cloud platform.

• LOFAR: Enhanced three pipe-lines using container technology.

Used Common WorkflowLanguage as workflow engine tolink the different pipelines.Provided access to FZJ/PSNCHPC resources to increase thevariety of platforms where toexecute the different pipelines.

• CryoEM: Extended a workfloweditor environment to enableworkflow import-export capa-bilities, which allows researchersto work more according to theFAIR principles.

• EGA Datasets: Gave access tothe B2FIND instance to depositmetadata produced by genomicspipelines.

EOSCpilot Science Demonstrators: one year later

Gergely Sipos outlines the status of the science demonstrators at the end of thefirst project year

More information

Gergely Sipos is Customer &Technical Outreach Managerat the EGI Foundation.

Gergely is involved in theWP4 (Science Demonstrators),WP5 (Services) and WP7(Skills) of EOSCpilot.

6

The eXtreme Data Cloudproject started in November2017 as a follow-up of thesuccessful INDIGO-DataCloudproject and will last until January2020. XDC develops scalabletechnologies for federatingstorage resources and managingdata in highly distributedcomputing environments. Theinitiative is funded by H2020under the EINFRA-21-2017 call“Computing e-infrastructure withextreme large datasets”, with theconsortium made of 8 partners -University of Cantabria, DESY,CERN, CNRS, AGH, ECRIN ERIC,EGI Foundation and led by INFN.

XDC brings together technologyproviders with long-standingexperience in software develop-ment and research communitiesbelonging to a broad spectrumof data-intensive scientificdisciplines, such as life sciences,biodiversity, clinical research,astrophysics, high-energyphysics and photon science.

The different communitiesprovide concrete use cases andheterogeneous requirements inthe field of access and mana-gement of data volumes at anunprecedented, 'extreme' scale.XDC will integrate some intelli-gence on top of data mana-gement functionalities to meetthe specific needs of scientificcommunities and implement amore flexible computing infras-tructure in Europe. The project’sservices are developed throughthe use of standards andprotocols available on state-of-the-art distributed computing

Introducing the eXtreme Data Cloud project

Daniele Cesini on the follow-up project of INDIGO-Data Cloud

ecosystems. They can be easilyplugged into European e-Infrastructures and in general oncloud based computingenvironments such as EGI, theEuropean Open Science Cloud(EOSC) and the Worldwide LHCComputing Grid (WLCG).

XDC will release open sourcesoftware based on alreadyexisting components enrichedwith new functionalities andplugins that can be adopted byas many user communities aspossible. Production qualityservices such as EOS, dCache,Onedata, the Indigo-PaaSOrchestrator will all beintegrated into the XDC servicecatalogue with improvedfunctionalities concerning policy-driven data management,quality-of-service based datamovement, smart caching,metadata management, securestorage and encryption.

Services provided by XDC will bescalable to cope with extremescale scientific experiments likethose run at the Large HadronCollider at CERN and theCherenkov Telescope Array.Given the XDC software will bereleased as Open Sourceplatforms available for generalexploitation, we foresee a wideadoption of XDC functionalitiesby less experienced and smalleruser communities as well.

Inspired // Issue #30, March 2018

More information

Daniele Cesini is the ProjectCoordinator of XDC.

eXtreme Data Cloud

www.extreme-datacloud.eu

EGI & XDC

EGI is contributing to theeXtreme Data Cloud projectwith dissemination, training &technical exploitation tasks.

EGI is also involved in thequality assurance of the projectand ensures that the softwareoutputs of XDC can be easilyused on e-Infrastructures.

7

From 22 to 25 January 2018, Portohosted over 230 particiants ofEUDAT’s conference “Putting theEOSC vision into practice”.Attendees included policy makers,service providers and researchcommunities representativesfrom 25 countries working onvarious data challenges.

The conference was opened byAugusto Burgueño Arjona,Head of the eInfrastructure Unitof the Directorate General forCommunications Networks,Content and Technology (DGCONNECT), who presented EOSCas an open science instrumentsupporting the collaborationbetween e-Infrastructures &research infrastructures: EOSChas to be an inclusive ecosystemwhere horizontal and thematicservice providers work together tomeet the user needs.

The discussion on how to putthe EOSC vision into practicewas addressed with breakoutsessions themed on the creationof a thriving data economy.

The topics approached were inthe range of: interoperability ofservices, the role of researchinfrastructures as thematicservice providers, businessmodels and sustainability ofdata infrastructures, legal issues.

One of the sessions was dedicatedto the results accomplished bythe collaboration of EGI andEUDAT for the implementation of aproduction cross-infrastructureoffering access to data and high-throughput computing resources.

Making data and cloud resources interoperable usingEUDAT & EGI services

Sara Garavelli on the close collaboration between EUDAT an EGI

The work of the collaborationinvolved concrete user commu-nities in the design process. Thishelped both EGI and EUDAT tobetter shape their services tomatch real needs of their users.

Two major use cases werebrought in by the ICOS and ENESresearch communities:

The ICOS use case focused onthe new web-based serviceoffered on the ICOS CarbonPortal to perform 3-dimensionalStochastic Time-InvertedLagrangian Transport (STILT)atmospheric transport modelcalculations. The input dataconsists of meteorological airtransport data (from ECMWF),data on greenhouse gasemissions (from EDGAR), andatmospheric observations (fromICOS and other sources). Theoutput data shows time series ofconcentrations of greenhousegases and their resulting foot-prints at selected locations. Thedata was successfully handledvia a combination of EUDATB2STAGE and B2SAFE servicesand other network file mana-gement systems, while theproduction model was visualisedusing EGI computing services.

The ENES use case addressedthe volume increase of theclimate data archive byemploying the EUDAT GeneralExecution Framework (GEF)Workflow API, in combinationwith EUDAT B2 services, andinterfacing with the EGI FederatedCloud. Post-processing results

Inspired // Issue #30, March 2018

(e.g. data on carbon gasemissions) are sent back to bedisplayed and further processedat the IS-ENES platform. Theycan also be downloaded indifferent common data formatsas tailored products via a simplewebsite interface. The input data- typically Coupled ModelIntercomparison Project phase5/6 (CMIP5/CMIP6) data – is nowbeing downloaded locally byclimate impact researchers andmakes room for a moresustainable data workflow.

To conclude, the session at theEUDAT conference was a greatopportunity to present examplesof the collaboration between EGI& EUDAT and between genericand thematic service providersin their effort to supportresearchers’ needs.

More information

Sara Garavelli is OutreachManager at the EUDAT CDI.

8

eInfraCentral was set up to ensurethat by 2020 a broader and morevaried set of users (includingindustry) discovers and accessesthe existing and developing e-infrastructure capacity. This isdone via the creation of acommon service catalogue incollaboration with Europe’s leadinge-infrastructures, and thedevelopment of a portal throughwhich to access it.

Why should researchers usethe eInfraCentral Portal?The eInfraCentral Portal makes iteasier for researchers to find theinformation about a broad rangeof e-infrastructure services theymay be interested in, discovernew ones, compare variousservice offerings in one place,and assess the relevance of theoffered services by rating themand leaving feedback.

How does eInfraCentralsupport the development ofprofessional servicecatalogues?eInfraCentral is working to aggre-gate the information available onthe service catalogues of a rangeof e-infrastructure and serviceproviders. We have developed aservice catalogue template,building on the experience ofthe five partner e-Infrastructures.The template is now being rolledout to another 30+ e-infrastructureprojects and is helping to fosteran alignment of service descrip-tions. Managers of service cata-logues can use our results toimprove their service presen-tation and management and weare organising webinars tosupport this process.

eInfraCentral introduces their service catalogue

Jelena Angelis answers frequently asked questions

How does eInfraCentralportal complement existingplatforms, for example, theEGI Marketplace or the futureEOSC-hub marketplace?Our mission is to enhance thevisibility of such marketplaces,not replace them. eInfraCentralis a portal that gives an oppor-tunity for researchers to com-pare information on e-infras-tructure services from variousproviders and marketplaces.Once a service is selected, theportal redirects the researcher tothe service provider or mar-ketplace where the service canbe accessed and ordered.

Is the eInfraCentral servicecatalogue a basis for theEOSC service catalogue?Our long-run aim is to ensurethe broadest coverage possibleof services from European-leveland national e-infrastructures.Along with EOSC-hub andOpenAIRE-Advance, eInfraCentralis one of the key building blocksof the future EOSC Portal asrecognised in the EuropeanCommission’s ImplementationRoadmap for the EuropeanOpen Science Cloud.

Inspired // Issue #30, March 2018

We will be working withcolleagues from these relatedprojects in the coming year tosetup the EOSC service catalogue.

How does the projectguarantee the quality of theservice catalogue?First of all, the quality is ensuredvia the neutrality of the eInfra-Central consortium partners – byavoiding conflict of interest, theproject team guarantee theobjectivity and trustworthinessof the service catalogue.Secondly, selected metrics onservice level targets will also bedisplayed to inform users aboutthe quality of the servicesavailable. Thirdly, metrics fromthe use of the portal will be fedback to the service providers tohelp them improve theirservices.

How can you register on theeInfraCentral Portal?It will shortly be possible toregister services and create youruser profile via the eInfraCentralPortal. In the meantime, pleasecontact us directly at:contact@eInfraCentral.eu forfurther information.

More information

Jelena Angelis is the eInfraCentral project manager.Browse the beta version of the catalogue of services:

http://beta.einfracentral.eu/home