Adoption Status of the OGSA-BES interface Morris Riedel, Mohammad Shahbaz Memon … and many others from the UNICORE community Jülich

Adoption Status of theAdoption Status of theOGSA-BES interfaceOGSA-BES interface

http://www.unicore.euhttp://www.unicore.eu

Morris Riedel, Mohammad Shahbaz Memon

… and many others from the UNICORE community

Jülich Supercomputing Centre (JSC)

2http://www.unicore.eu

Architecture Overview OGSA-BES Adoption Overview

Job Submission Description LanguageHigh Performance Computing Basic Profile

OGSA-BES Adoption in Detail OGSA-BES Limitations/Additions Future Work for OGSA-BES

Link to other GIN session: Security Profile References

OutlineOutline


Architecture OverviewArchitecture Overview


UNICOREWS-RF

hostingenvironment

XNJS

ArchitectureArchitecture

IDB

UNICORE Atomic Services

OGSA-*

XACML entity

emerging standardinterfaces

Grid services hosting

job incarnation &authorization

authentication

scientific clientsand applications

parallel scientific jobs of multiple end-users

on target systems

UNICOREWS-RF

hostingenvironment

XNJSIDB

UNICORE Atomic Services

OGSA-*

XACML entity

Gateway

UNICOREWS-RF

hostingenvironment

ServiceRegistry

Gateway

ByteIO

BES

RUS

ByteIO

BES

RUS

XACML XACML

HPC-P

GPEapplication client

command-line client

Eclipse-basedclient

Portal client, e.g. GridSphere

WS-ISOAP JSDL

XUUDB

SAML-VOMS

X.509

UR

JSDL

HPC-P

UR

JSDL

WS-RF

SAML

X.509

Local RMS (e.g. Torque, LL, LSF, etc.)

Target System Interface

Local RMS (e.g. Torque, LL, LSF, etc.)

Target System Interface

DRMAA DRMAA

M. Riedel et al. [1]

R. Ratering et al. [6]


OGSA-BES Adoption OverviewOGSA-BES Adoption Overview


Three Services basicallysubstituting the proprietary services: TargetSystemFactory (TSF) TargetSystemService (TSS) JobManagementService (JMS)

OGSA-BES Factory Service CreateActivity( JSDL )

OGSA-BES Management Service E.g. Allow new submissions or not

Use of UNICORE 6 Backend XNJS Forwarding to TSI on resource

Developed by

OGSA – Basic Execution Services (BES)OGSA – Basic Execution Services (BES)

M.S. Memon et al. [2]


UNICORE 5 Proprietary description:

Abstract Job Object (AJOs) UNICORE 6 Services

Target System Service (TSS)accepts JSDLs

OGSA – Basic Execution Services accepts JSDLs

UNICORE 6 Execution Backend XNJS backend component

natively accept JSDLs JSDLs are transformed

into a proprietary simple protocolto submit job to RMSs (e.g. Torque)

Job Submission Description Language (JSDL)Job Submission Description Language (JSDL)

M. Riedel et al. [1]


(Participated in the Supercomputing 2007 Interoperation demonstrations)

XNJS accept JSDL extensions compliant to ones defineswithin the HPC-BP profile

Service Container provides asecurity handler to accept credentials according toWS-Security Username TokenProfile (part of HPC-BP)

HPC – Basic Profile (HPC-BP)HPC – Basic Profile (HPC-BP)

M.S. Memon et al. [2]


OGSA-BES Adoption in DetailOGSA-BES Adoption in Detail


Support of all three defined portTypes with defined faults Configurations in uas.config configuration file of UNICORE

E.g. initial value whether accept new activities or not(property of BES-Management portType)

Support of BES State Model (and UNICORE state model) Additional States:

BES interface implementation (developer view) Use the xFire SOAP engine Use XMLBeans for Java-XML bindings No problems in creating XMLBeans from BES schemas

(no schema for BES-Activity provided) Sourcecode as open source on UNICORE@SourceForge Execution Management provided by XNJS of UNICORE

OGSA – BES Adoption in Detail (1)OGSA – BES Adoption in Detail (1)


Supports filtered activities during BES-Factory operations… (Filtered activities are activities, which are profiled or associated

with users credentials, and its visibility will be exposed to only the authorized users who created the activity)

Own filter created to support this feature Supported Operations: GetActivityDocuments,

GetActivityStatuses, TerminateActivities UNICORE 6 supports the optional WS-RF Rendering by

adding all WS-RF compliant message exchanges Enables GetResourceProperties, UpdateResourceProperties,… Activity resources are modeled as WS-RF resources, each BES

Activity instance maintains activity document as state Consists of JSDL sent by user, reference to BES-Factory which created

that activity instance, and the current activity status



Client support in UNICORE UNICORE Rich Client support as additional plug-in

demonstrator UNICORE Command-Line Client (UCC) Support

Interoperability Tests with othe OGSA-BES implementations in the scientific Grid landscape Production Level for e-Infrastructure integration

Only tested with CREAM-BES (SAML-based Security), planned for gLite release – CREAM in certification process

Initial tests with Globus-BES beta (provided by KTH) Interoperablity Testings with other HPC-BP adopters

E.g. NorduGrid ARC implementations, University of Virginia implementations,…



OGSA-BES Limitations/AdditionsOGSA-BES Limitations/Additions


Additions to the BES-Factory Attributes Document INFOS! GLUE resource property (for specifying admin domain info,

application info, computing resource info) for more information Additions to the BES-Activity portType

(operations are not normatively defined in specification) To support an easy access to single resources (instead of

vectors/arrays as in BES-Factory) operations added In addition, also WS-RF compliant message exchanges work at

this portType – to work with the resource model in UNICORE No aligned security model strong enough for production

High Performance Computing Basic Profile good first step into right direction, but not for production usage in e.g.DEISA

Precise profiled attribute-based AuthZ support required today

OGSA – BES Limitations / Additions (1)OGSA – BES Limitations / Additions (1)


Link to storage added in BES-Activity (out of scope of OGSA-BES – but required in UNICORE) Storage Endpoint to a UNICORE proprietary Storage

Management Service (SMS) – Req. for BES-related activity! SMS Endpoint added to the BES Activity resource properties

Property manages a user space mapping with individual activities SMS can be used to access outcome of computational jobs

File Transfer for data staging issues (not precise enough) SMS link in turn is a factory for managing individual file transfer

functions such as stage-in/out of files for individual jobs File Transfer realized via File Transfer Service (i.e. ByteIO) Data Staging Profile might be interesting but is not complete in-

line with production deployment



UNICORE 6 basic infrastructure relies on WS-RF Any activity is an instance of the WS-RF resource model

‘Optional WS-RF support’ in specification makes it difficult for UNICORE clients to work with other BES implementations E.g. GetResourceProperties of CREAM-BES fails since it is not

WS-RF compliant ‘Optional WS-RF support’ requires a factory for the factory

WS-RF factory pattern difficult to implement Because factory is not end-user specific instance



Bottom lines BES is not isolated – it must be well embedded with other

middleware services – but how to standardize/specify Storage is tightly connected with BES File transfer is tightly connected with BES Security Setups are tightly connected with BES

For all these topics are not normative standards defined – only BES/JSDL

More Profiling is required



Future Work for OGSA-BESFuture Work for OGSA-BES


Enable more functionalities in terms of … data-staging profile, information model GLUE hooks, …

Delegation Scenarios How are third party transfer credentials transferred through the

OGSA-BES interface E.g. data stage-in is performed using GridFTP (requiring

credentials – is not an implementation issue) SAML delegation vs. Proxy delegation Link to GIN Security

Profile Session More production-oriented security profiles aligned with BES

Attribute-based Authorization, where credentials transported? E.g. WS-Security Message exchanges in SOAP header

Link to GIN Security Profile Session

Future Work for OGSA-BESFuture Work for OGSA-BES


ReferencesReferences


[1] M. Riedel, B. Schuller, D. Mallmann, R. Menday, A. Streit, B. Tweddell, M.S. Memon, A.S. Memon, B. Demuth, Th. Lippert, D. Snelling, S. van den Berghe, V. Li, M. Drescher, A. Geiger, G. Ohme, K. Benedyczak, P. Bala, R. Ratering, A. Lukichev Web Services Interfaces and Open Standards Integration into the European UNICORE 6 Grid Middleware Proceedings of 2007 Middleware for Web Services (MWS 2007) Workshop at 11th International IEEE EDOC Conference "The Enterprise Computing Conference", 2007, Annapolis, Maryland, USA, to appear

[2] M.S. Memon, A.S. Memon, M. Riedel, B. Schuller, D. Mallmann, B. Tweddell, A. Streit, S. van den Berghe, D. Snelling, V. Li, M. Marzolla, P. Andreetto Enhanced Resource Management Capabilities using Standardized Job Management and Data Access Interfaces within UNICORE Grids Proceedings of 3rd Workshop on Scheduling and Resource Management for Parallel and Distributed Systems SRMPDS 2007, ICPADS'07 - The 13th International Conference on Parallel and Distributed Systems Hsinchu, Taiwan, December, 2007, to appear

[3] M. Riedel, R. Menday, A. Streit, and P. BalaA DRMAA-based Target System Interface Framework for UNICOREProceedings of Second International Workshop on Scheduling and Resource Management for Parallel and Distributed Systems (SRMPDS’06) at Twelfth International Conference on Parallel and Distributed Systems (ICPADS’06), Minneapolis, USA, IEEE Computer Society Press, pages 133 - 138

References (I)References (I)


[4] W. Frings, M. Riedel, A. Streit, D. Mallmann, S. van den Berghe, D. Snelling, and V. LiLLview: User-Level Monitoring in Computational Grids and e-Science Infrastructures.In Proc. of German e-Science Conference 2007, Baden-Baden, Germany, Online-Publication

[5] V. Venturi, M. Riedel, A.S. Memon, M.S. Memon, F. Stagni, B. Schuller, D. Mallmann, B. Tweddell, A. Gianoli, V. Ciaschini, S. van de Berghe, D. Snelling, A. StreitUsing SAML-based VOMS for Authorization within Web Services-based UNICORE Grids.Proceedings of 3rd UNICORE Summit 2007 in conjunction with EuroPar 2007, Rennes, France, to appear

[6] R. Ratering, A. Lukichev, M. Riedel, D. Mallmann, A. Vanni, C. Cacciari, S. Lanzarini, P. Bala, K. Benedyczak, M. Borcz, R. Kluszcynski, and G. Ohme,GridBeans: Supporting e-Science and Grid Applications.In Proc. of the 2nd IEEE International Conference on e-Science and Grid-Computing (e-Science 2006), IEEE Computer Society Press, Amsterdam, NL, December 4-6, 2006, ISBN: 0-7695-2734-5, proceedings on CD

References (II)References (II)