Embed Size (px)
DESCRIPTION
Building an E uropean R esearch Community through Interoperable Work flow s and Data ER-flow project Gabor Terstyanszky, ER-flow Project Coordinator 05 July 2013. ER-flow is supported by the FP7 Capacities Programme under contract No. RI-261585. ER-flow Project. Partners:. - PowerPoint PPT Presentation
Citation preview
Building an European Research Community through Interoperable Workflows and Data
ER-flow projectGabor Terstyanszky,
ER-flow Project Coordinator05 July 2013
ER-flow is supported by the FP7 Capacities Programme under contract No. RI-261585
22
University of Westminster UoW United KingdomMagyar Tudomanyos Akademia Szamitastechnikai es Automatizalasi Kutato Intezete
MTA-SZTAKI Hungary
Centre National de la Recherche Scientifique CNRS FranceStichting European Grid Initiative EGI.eu The NetherlandsAcademic Medical Center of the University of Amsterdam
AMC The Netherlands
Technische Universität Dresden TUD GermanyLudwig-Maximilians-Universität München LMU GermanyUniversity College London UCL United KIngdomTrinity College Daudublin TCD IrelandIstituto Nazionale di Astrofisica INAF Italy
Partners:
Technology providers:CNRS, EGI.eu, MTA-SZTAKI, UoW
Research Communities:Astro-Physics INAFComputational Chemistry LMU + TUDHelio-Physics TCD + UCLLife Science AMC
Duration:September 2012 – August 2014
ER-flow ProjectER-flow Project
3
Project Aim and ServicesProject Aim and ServicesAim:• To provide a simulation platform for research communities to enable
seamless execution of workflows of different workflow systems through workflow interoperability
• To investigate data interoperability issues in the workflow domain and propose solutions
Services:• To support the whole workflow lifecycle: editing, uploading, browsing
downloading and executing workflows
• To provide coarse-grained workflow interoperability solution
• To provide GUIs to manage workflows
Key actors:• researchers workflow engine developers
workflow developers
44
ER-flow & Research CommunitiesER-flow & Research Communities
Supported research communities:• Astro-Physics 14 workflows
• Computational Chemistry 20 workflows
• Helio-Physics 14 workflows
• Life Science 20 workflows
Supported research communities:• number of users: minimum: 250
• Number of executed workflow: 3000
Further research communities:• at least four more research communities Hydrometeorology, Seizmology ???
5
abstract workflow
concrete workflow
workflow engine
workflow configuration
concrete workflow
engine
abstract workflow
engine
workflow engine configuration
workflow formal descriptionworkflow engine formal description
Coarse-Grained Interoperability:Coarse-Grained Interoperability:CGI conceptCGI concept
666
WF Submission Service
Workflow Engine B
Workflow of Workflow Engine B
Workflow Engine A
Coarse-Grained Interoperability:Coarse-Grained Interoperability:submitting non-native workflowsubmitting non-native workflow
DCI
• non-native workflow: WF
- non-native workflows are black boxes which are managed as legacy code applications
Workflow Engine A
Coarse-Grained Interoperability: Coarse-Grained Interoperability: meta-workflow submissionmeta-workflow submission
• native workflows: J1, J3 and WF2• non-native workflows: WF4
- black boxes which are managed as legacy code applications
WF2 WF3
J1
WF4
Submission Service
Workflow Engine B
Workflow of Workflow Engine B
Workflows of Workflow Engine A
DCI
Workflow Engine A
Workflow Repository
8
gLite DCI Globus DCI
DCIs
ARC DCI Unicore DCI
Kepler WEGWES WE
Taverna WE
Triana WE
MOTEUR WE
ASKALON WE
PGRADE WE
SHIWA Portal
GEMLCA admin
SHIWA Science Gateway
WE1 WEp
GEMLCA Repository
WF1 WFm
SHIWA Repository
WF1 WFn
WS-PGRADE Workflow
engine
WS-PGRADE Workflow
editor
GEMLCA Service
GEMLCA with GIB
SHIWA Proxy Server
Proxy Server
Galaxy WE
Pegasus WE
ProActive WE
Workflow Engines
ER-flow Development Environment:ER-flow Development Environment:SHIWA Simulation Platform (SSP)SHIWA Simulation Platform (SSP)
999
SHIWA Portal: Editing Workflow SHIWA Portal: Editing Workflow
1010
SHIWA Portal: Configuring WorkflowSHIWA Portal: Configuring Workflow
1111
SHIWA Portal: Executing WorkflowSHIWA Portal: Executing Workflow
1212
Exporting Workflow (1):Exporting Workflow (1):Select the SHIWA RepositorySelect the SHIWA Repository
1313
Exporting Workflow (2):Exporting Workflow (2):AuthenticationAuthentication
1414
Exporting Workflow (3):Exporting Workflow (3):Display Workflow Data and Submit itDisplay Workflow Data and Submit it
1515
Importing Workflow (1):Importing Workflow (1):Select the SHIWA RepositorySelect the SHIWA Repository
1616
Importing Workflow (2):Importing Workflow (2):Select the Workflow and Import itSelect the Workflow and Import it
1717
Importing Taverna WorkflowsImporting Taverna Workflows
18
Robot Certificate ManagementRobot Certificate Management
19
Workflow Execution StatisticsWorkflow Execution Statistics
202020
SHIWA Repository: Table ViewSHIWA Repository: Table View
21
SHIWA Repository: Table ViewSHIWA Repository: Table View
222222
SHIWA Repository: Browse ViewSHIWA Repository: Browse View
23
SHIWA Repository: Browse ViewSHIWA Repository: Browse View
2424
CGI Developer Scenario: CGI Developer Scenario: Specifying Workflow EngineSpecifying Workflow Engine
SHIWA Portal
GEMLCA admin
SHIWA Science Gateway
WE1 WEp
GEMLCA Repository
WF1 WFm
SHIWA Repository
WF1 WFn
WS-PGRADE Workflow
engine
WS-PGRADE Workflow
editor
GEMLCA Service
GEMLCA with GIB
SHIWA Proxy Server
Proxy Server
step 1specify WE
data
step 2upload WE
binary, dependencies
step 3deploy WE
workflow engine
developer
252525
CGI Developer Scenario: CGI Developer Scenario: Specifying Non-Native WorkflowsSpecifying Non-Native Workflows
SHIWA Portal
SHIWA Science Gateway
WE1 WEp
GEMLCA Repository
WF1 WFm
SHIWA Repository
WF1 WFn
WS-PGRADE Workflow
engine
WS-PGRADE Workflow
editor
GEMLCA Service
GEMLCA with GIB
SHIWA Proxy Server
Proxy Server
step 1specify WF
data
step 2upload WF
step 3deploy WF
workflow developer
26
CGI User Scenario: CGI User Scenario: PGRADE as Native WE PGRADE as Native WE
SHIWA Repository
SHIWA Portal
WF1
SHIWA Science Gateway
GEMLCA Service
WFn
WE1 WEp
GEMLCA Repository
WF1 WFm
GEMLCA with GIB
WS-PGRADE Workflow
engine
WS-PGRADE Workflow
editor
WF liste-scientists
SHIWA Proxy Server
Proxy Server
step 1search WF
step 2 edit WF
Kepler WEGWES WE
Taverna WE
Triana WE
MOTEUR WE
ASKALON WE
PGRADE WE
Galaxy WE
Pegasus WE
ProActive WE
Workflow Engines
gLite DCI Globus DCI
DCIs
ARC DCI Unicore DCI
step 7 run WFstep 6
retrieve proxy
WE + WF step 5retrieve WF
step 4submit WF
step 3retrieve WF data
2727
SHIWA Science Gateway
GEMLCA Service
SHIWA Repository
Submission service
WE1 WEn
WF1 WFm
user
SHIWA Proxy Server
Proxy Server
step 1search WF
step 2 edit WF
Kepler WEGWES WE
Taverna WE
Triana WE
MOTEUR WE
ASKALON WE
PGRADE WE
Galaxy WE
Pegasus WE
ProActive WE
Workflow Engines
gLite DCI Globus DCI
DCIs
ARC DCI Unicore DCI
step 6 run WF
step 5retrieve proxy
WE + WF
step 4retrieve WF
step 3submit WF
GEMLCA Client
MOTEURWorkflow
Engine
MOTEUR Workflow
editor
GEMLCA UI
CGI User Scenario: CGI User Scenario: MOTEUR as Native WE MOTEUR as Native WE
2828
SSP: Execution InfrastructureSSP: Execution Infrastructure
gLite DCI Globus DCI
DCIs
ARC DCI Unicore DCI
Kepler WEGWES WE
Taverna WE
Triana WE
MOTEUR WE
ASKALON WE
PGRADE WE
Galaxy WE
Pegasus WE
ProActiveWE
Workflow Engines
Astrophysics Science Gateway
Computational Chemistry Science Gateway
Heliophysics Science Gateway
Life sciences Science Gateway
SHIWA RepositoryWFn
SHIWA Submission
Service
WFn
submitting native
workflows
submitting non-native workflows
29
name short description WE middleware VOASTRO PHYSICS COMPCAT capture comets from the interstellar space WS-PGRADE gLITE Astro + VOCE VOFRANEC/BASTI evolutionary stellar models WS-PGRADE gLITE A&A VOLasMoG modified gravity models WS-PGRADE gLITE Astro VOMESSTREAM modelling the dynamic evolution of meteoroid stream WS-PGRADE gLITE Astro + VOCE VOPLANCK simulation of the ESA Planck mission WS-PGRADE gLITE Astro + Planck VOVisIVO Computational Cosmology WS-PGRADE gLITE A&A VO
COMPUTATIONAL CHEMISTRY GROMACS Molecular Dynamic application Energy Minimisation WS-PGRADE gLiTE/UniCore MoSGrid VO Equilibration WS-PGRADE gLiTE/UniCore MoSGrid VO Single TPR WS-PGRADE gLiTE/UniCore MoSGrid VOCADDSuite docking application Docking with ligand generation WS-PGRADE gLiTE/UniCore MoSGrid VO Docking without ligand generation WS-PGRADE gLiTE/UniCore MoSGrid VONWChem Quantum Chemistry/Molecular Dynamics application Geometry optimisation = basic WF WS-PGRADE gLiTE/UniCore MoSGrid VO Opt+freq WS-PGRADE gLiTE/UniCore MoSGrid VO Freq WF WS-PGRADE gLiTE/UniCore MoSGrid VO TD-DFT WF WS-PGRADE gLiTE/UniCore MoSGrid VO Mulliken WF WS-PGRADE gLiTE/UniCore MoSGrid VO Solvation WF WS-PGRADE gLiTE/UniCore MoSGrid VO Spectroscopic analysis = Metaworkflow WS-PGRADE gLiTE/UniCore MoSGrid VO Parameter sweep = Parameter Workflow WS-PGRADE gLiTE/UniCore MoSGrid VO Transition state seach WS-PGRADE gLiTE/UniCore MoSGrid VO Transition state analysis = MetaWF of basic, transition state search and freq WF WS-PGRADE gLiTE/UniCore MoSGrid VO High throughput WS-PGRADE gLiTE/UniCore MoSGrid VOHELIO PHYSICS
Monthly Event counts Counts the number of events for a period by querying the HEC catalogue Taverna 2 web service noneDetection of fastest type II CME identifying the fastest type II CME in time Taverna 2 web service none
finding events in Xray and radio Taverna 2 web service noneRetrieve images for an event retrieving solar monitor urls for votable events Taverna 2 web service none
Retrieve data for HEC event dataRetrieves all instrument data for an event defined in the HEC catalogue and instrument defined in ICS () Taverna 2 web service none
Associate flares with active regions associating flares with active regions Taverna 2 web service none
Origin of solar wind events on Earth Finds the origins on the Sun of the Solar Wind detected on the Earth Taverna 2 web service noneLIFE SICENCE free surfer medical imaging WS-PGRADE gLITE vlemed VOdouble cross validation pattern classification WS-PGRADE/ gLITE vlemed VO /MOTEUR gLITE vlemed VOprotein docking autodock vina with filtering WS-PGRADE gLITE vlemed VOSNP annotation DNA sequencing with Annovar WS-PGRADE/ gLITE, PBS vlemed VO
/MOTEUR gLITE vlemed VOSNP calling genome data re-sequencing WS-PGRADE/ gLITE, PBS vlemed VO
/MOTEUR gLITE vlemed VOsequence alignment aligning sequence fragments to a reference database WS-PGRADE/ gLITE, PBS vlemed VO
/MOTEUR gLITE vlemed VOindel calling call insertions and deletions from sequence alignments WS-PGRADE gLITE, PBS vlemed VOsequence assembly of small genomes reconstruction of genomes by building contig sequences WS-PGRADE gLITE, PBS vlemed VO
simulation of low coverage sequence experiments random samples alignments from a sequence alignment file WS-PGRADE gLITE, PBS vlemed VOexome sequencing WS-PGRADE gLITE, PBS vlemed VOdti processing WS-PGRADE gLITE vlemed VO
/MOTEUR gLITE, PBS vlemed VO
ER-flow Research CommunitiesER-flow Research Communities
303030
End User ViewEnd User View
3131
End User ViewEnd User View
3232
End User ViewEnd User View
3333
End User ViewEnd User View
3434
User Tailored View (ASM)User Tailored View (ASM)
3535
User Tailored View (ASM)User Tailored View (ASM)
