View
226
Download
0
Tags:
Embed Size (px)
Citation preview
WS-VLAM Introduction presentation
WS-VLAM Introduction
Systems and Network Engineering group
Institute of informatics
University of Amsterdam
WS-VLAM Introduction presentation
Introduction
• This presentation gives a quick introduction to the WS-VLAM workflow management system
• It helps the reader understand the technology push which motivated our design, describes the current state and present the ongoing activities and planned ones.
• NOTE: We are currently still working on the User’s and developer’s
Guide, they are not available at this time. The presentations do not replace the documentation it just give a quick introduction.
Do not hesitate to contact us for further information
Thanks for taking some of your time to read about the WS-VLAM work, any comments are more than welcome
WS-VLAM developers [email protected]
WS-VLAM Introduction presentation
Needed background …
• To be able to follow the content of this presentation you will need to know about:
WSRF= Web Service Reference Framework
GT4= Globus Toolkit version 4
• More detailed description of the presented work can be found in the section of publication of the WS-VLAM home page
WS-VLAM Introduction presentation
Objective of this presentation…
• Why you should look at this presentations You want to know how WS-VLAM works and
interacts with the Grid middleware.
• Why you don’t have to worry if you do not understand the content on this presentation: You are an end-user: it will be enough for you to
read the User’s guide or the WS-VLAM composer presentation
You are an application developer: it will be enough for you to read the Developer’s guide & the WS-VLAM developers presentation
WS-VLAM Introduction presentation
Outline
• Introduction • Architecture of the WS-VLAM• Hierarchical workflow compositions &
execution • Interoperability with other WMS• Conclusions
UvA, Amsterdam June 2007 WS-VLAM Introduction presentation
GVLAM:
• Enable end-users (scientists) to define, execute, and monitor e-science applications. Location independent experimentation. Application Domain portal.
• Provide application developers: Abstract view of the Grid (shields the Grid
details). Easy way to port applications to the Grid.
WS-VLAM Introduction presentation
The Virtual Laboratory for e-Science …
Dutc
h te
lesc
ienc
e
Data
inte
nsive
scie
nce
Med
ical
diag
nosis
Generic e-science framework layer
Application layer
Bio
info
rmat
ics
ASP
Bio
dive
rsity
Food
Info
rmat
ics
Grid layer
WS-VLAM Introduction presentation
Process Management Stack
Before and after OGSA/WSRF
Network & Computing Resources
Grid Middleware: Process & resource management
Application Templates
User Applications
Workflow Management system
Web Service Interface
Data Management Stack
Network & storage Resources
Grid Middleware: Data management
Web Service Interface
WS-VLAM Introduction presentation
Process Management StackData Management Stack
Workflow Management Systems
Network & Computing Resources
Grid Middleware: Process & resource management
Application Templates
User Applications
Workflow execution
Workflow composition
Network & storage Resources
Grid Middleware: Data management
Workflow Management system
Web Service Interface Web Service Interface
WS-VLAM Introduction presentation
Data Management Stack Process Management Stack
Grid Service & OGSA-DAI
Network & Computing ResourcesNetwork & storage Resources
Grid Middleware: Process & resource management
Grid Middleware: Data management
Application Users
Workflow execution
GridServices
GridServices
GridServices
OGSA DAI
Application Templates
User Applications
Workflow composition
Workflow Management system
Web Service Interface Web Service Interface
WS-VLAM Introduction presentation
Data Management Stack Process Management Stack
First Application developers …
Network & Computing ResourcesNetwork & storage Resources
Grid Middleware: Process & resource management
Grid Middleware: Data management
Workflow execution
GridServicesApplication
web service
GridServices
GridServices
Application Templates
User Applications
Application TemplatesWeb service
Workflow composition
Web Service Interface Web Service Interface
OGSA DAI
Workflow Management system
WS-VLAM Introduction presentation
Process Management Stack
Second the Workflow developers …
Data Management Stack
Network & Computing ResourcesNetwork & storage Resources
Grid Middleware: Process & resource management
Grid Middleware: Data management
Workflow execution
Workflow composition
Workflow Management system
GridServices
GridServices
GridServicesApplication Templates
Web service
Web Service Interface
OGSA DAI
WorkflowEngine
Web service
WorkflowEngine
Web service
Web Service Interface
Application web service
Distributed Workflow
WS-VLAM Introduction presentation
WS-VLAM WMS…
• Goals Create a WMS which adopt a Service Oriented
Paradigm
• Approach Workflow Engine (RTSM)
o Implemented as a WSRF service using Globus 4 (GT4)
Workflow Composer (GUI)o Thin client
Repository for workflow componentso Implemented also as WSRF service.
WS-VLAM Introduction presentation
Conclusions
• WSRF compliant middleware such as GT4 offers useful basic services to develop a truly grid enabled workflow engine:
Delegation service and ws-notification mechanisms proved to be very useful
• Separating the workflow composition from execution suits better the Grid environments and improve interoperability
• OGSA-WG is collecting cases and input to decide which services should be standardized
WS-VLAM Introduction presentation
New workflow Components
• The workflow components developed the last year, can be still used and will be deployed on the shared workflow component repository on the rapid prototyping environment Ridge-O-Grammar modules from (SP 1.5) Kansk Toolbox from (SP 1.4) AID web services (SP 2.2) Virtual file system Web service (SP 2.4) Generic workflow components (SP 2.5)
o File Reader/Writer, o universal File Reader/Writer (SRB/GFTP/...)o Demos: MRIscanner, Histogram, Floatingball
• We will contact other SPs to integrate their components o Legacy code, or fully integrated code (java, C++, Python)
WS-VLAM Introduction presentation
other tasks
• Access to the ws-vlam engine from Taverna
• Access to the ws-vlam engine from Kepler
• Parameter sharing among workflow components
• Job farming
• Support for more than one job submission mechanisms (GT5-GRAM, Direct, etc)
• Fault tolerance
• Control flow
• Data provenance
• Semantic Service discovery
UvA, Amsterdam June 2010
WS-VLAM Introduction presentation
References and download
• VLAM client can be downloaded from www.science.uva.nl/~gvlam/wsvlam/
• Documentation can be downloaded www.science.uva.nl/~gvlam/wsvlam/doc/
UvA, Amsterdam June 2010
UvA, Amsterdam June 2010 WS-VLAM Introduction presentation
Workflow interoperability: Execute a VLAM workflow from Kepler/Taverna
• A predefined Application workflow developed in VLAM can be executed as a single step in Kepler/Taverna (no need to recompose graphically the whole workflow).
• The predefined Application workflow will be executed on any remote computing resource where the VLAM-RTSM GT4 Web service is installed.
• Advantages: Compose workflow where sub-workflows (which require grid
resources) are executed on grid-enabled resources, while the rest of the workflow is either executed using other Kepler actors or taverna processors
It is also more efficient, since it avoid the overhead which will result by wrapping every workflow component as a separated web service or a separate remote grid-execution.
UvA, Amsterdam June 2010 WS-VLAM Introduction presentation
Workflow interoperability: Execute a VLAM workflow from Kepler/Taverna
• A predefined Application workflow developed in VLAM can be executed as a single step in Kepler/Taverna (no need to recompose graphically the whole workflow).
• The predefined Application workflow will be executed on any remote computing resource where the VLAM-RTSM GT4 Web service is installed.
• Advantages: Compose workflow where sub-workflows (which require grid
resources) are executed on grid-enabled resources, while the rest of the workflow is either executed using other Kepler actors or taverna processors
It is also more efficient, since it avoid the overhead which will result by wrapping every workflow component as a separated web service or a separate remote grid-execution.
UvA, Amsterdam June 2010 WS-VLAM Introduction presentation
Workflow interoperability: Execute a VLAM workflow from Kepler/Taverna
GRAMservices
GT4 Java Container
RTSMFactory
Delegationservice
Worker nodes
pre-ws-GRAM
RTSM Instance
Workflowcomponents
Kepler/Taverna workbench
RTSM Client
WorkfowDescription
(XML)
Das2 or PoC facilities.
1. Kepler/Taverna users can have access to some of the parameters of the Application workflow to change the default values
2. Kepler/Taverna users have to specify the location of the input data file as URL and will get back a URL if the Sigwin generate data files
3. Graphical output of the Application workflow are handled automatically by the VLAM Taverna processor /Kepler actor.
RTSM-GT4 Web serviceVLAM Actor
or
Taverna processor
(1) Proxy Delegate
(2) Service
Invocation
Some References
1. Science Z Zhao, A Belloum, M Bubak Editorial: Special section on workflow systems and applications in e-Future Generation Computer Systems 25 (5), 525-527
2. A. Wibisono, D. Vasyunin, V. Korkhov, AS.Z. Belloum WS-VLAM: a GT4 based workflow management system , The 2nd International Workshop on Scientific Workflows, In conjunction with ICCS 2007, Beijing, China, 2007
3. A.S.Z. Belloum, V. Korkhov, S. Koulouzis, M. A Inda, and M. Bubak Collaborative e-Science experiments: from scientific workflow to knowledge sharing JULY/AUGUST, IEEE Internet Computing, 2011
4. Ilkay Altintas, Manish Kumar Anand, Daniel Crawl, Shawn Bowers, Adam Belloum, Paolo Missier, Bertram Ludascher, Carole A. Goble, Peter M.A. Sloot, Understanding Collaborative Studies Through Interoperable Workflow Provenance, IPAW2010, Troy, NY, USA
UvA, Amsterdam June 2010 WS-VLAM Introduction presentation
Some References
5. A. Belloum, Z. Zhao, and M. Bubak Workflow systems and applications , Future Generation Comp. Syst. 25 (5): 525-527 (2009)
6. Z. Zhao, A.S.Z. Belloum, et al., Distributed execution of aggregated multi domain workflows using an agent framework The 1st IEEE International Workshop on Scientific Workflows, Salt Lake City, U.SA, 2007
7. Zhiming Zhao, Adam Belloum, Cees De Laat, Pieter Adriaans, Bob Hertzberger Using Jade agent framework to prototype an e-Science workflow bus Authors Cluster Computing and the Grid, 2007. CCGRID 2007
UvA, Amsterdam June 2010 WS-VLAM Introduction presentation