GridLab Enabling Applications on the Grid

GridLabEnabling Applications on the Grid

www.gridlab.org

Jarek Nabrzyski et al.

naber@man.poznan.pl

office@gridlab.orgPoznań Supercomputing and

Networking Center

Grid 2002, Baltimore, 18 November 2002

GridLab Project

Funded by the EU (5+ M€), January 2002 – December 2004Application and Testbed oriented

Cactus Code, Triana Workflow, all the other applications that want to be Grid-enabled

Main goal: to develop a Grid Application Toolkit (GAT) and set of grid services and tools...:

resource management (GRMS), data management,monitoring, adaptive components, mobile user support, security services,portals,

... and test them on a real testbed with real applications

Grid 2002, Baltimore, 18 November 2002

GridLab Members

PSNC (Poznan) - coordination AEI (Potsdam) ZIB (Berlin) Univ. of Lecce Cardiff University Vrije Univ. (Amsterdam) SZTAKI (Budapest) Masaryk Univ. (Brno) NTUA (Athens)

Sun MicrosystemsCompaq (HP)

ANL (Chicago, I. Foster) ISI (LA, C.Kesselman) UoWisconsin (M. Livny)

collaborating with:Users!

EU Astrophysics Network,

DFN TiKSL/GriKSL

NSF ASC Project

other Grid projectsGlobus, Condor,

GrADS,

PROGRESS,

GriPhyn/iVDGL,

CrossGrid and all the other European Grid Projects (GRIDSTART)

other...

Grid 2002, Baltimore, 18 November 2002

GridLab Applications

Cactus (www.cactuscode.org) Triana (www.triana.co.uk)

Grid 2002, Baltimore, 18 November 2002

GridLab Aims

Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community),

Make it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizations

Dream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.

Grid 2002, Baltimore, 18 November 2002

What GridLab isn’t

Don’t want to develop low level Grid Infrastructure,

Don’t want to repeat work which has already been done (want to incorporate and assimilate it … Globus APIs, OGSA, ASC Portal (GridSphere/Orbiter), GPDK, GridPort, DataGrid, GriPhyn)

Grid 2002, Baltimore, 18 November 2002

GridLab end user requirements

Application oriented environment,Applications running on resources of one or more virtual organisations,Flexible, easy-to-use, simple interfaces to resources, jobs, and data (including compiling, tracking jobs, cataloguing data),Means to make efficient and effective use of resources,Robustness, implying that smart adaptivity, complete control and fail safety are available on all levels,The ability to work in a disconnected environment,Mobile working

Grid 2002, Baltimore, 18 November 2002

GridLab end user requirements

The ability to run in environments as minimalistic as laptops with no grid infrastructure to fully deployed Virtual Organisations,

Complexity should be hidden as much as possible,

Provide a collaborative infrastructure,

The infrastructure must cater for all classes of applications, from lightweight to largescale,

The infrastructure must provide capabilities to customise choice of service implementation (e.g. using efficiency, reliability, first succeeding, all)

Grid 2002, Baltimore, 18 November 2002

Solution...

GAT – a layer between apps and emerging grid technologies

GridLab tesbted/VO

Close cooperation between developers and deployers

End Users

GAT Tool Developers

Grid Infrastructure Developers

GAT-APIDevelopers

Grid 2002, Baltimore, 18 November 2002

The Grid is complex …

Monitoring

Resource Management

InformationSecurity

DataManagement

GLOBUS

ApplicationManager

Logging

NotificationMigration

Profiling

SOAP WSDL Corba OGSA Other

Other GridInfrastructure?

Cactus

“Is there a better resource I could be using?”

Grid 2002, Baltimore, 18 November 2002

…need to make it easier to use

GAT

Cactus

“Is there a better resource I could be using?”

GAT_FindResource( )

The Grid

Grid 2002, Baltimore, 18 November 2002

GAT: What is It?

GAT: Grid Application ToolkitImplements the GAT-API

Used by applications (different languages)

GAT Adaptors Connect to capabilities/services

GAT EngineProvides the function

bindings for the GAT-API

Grid 2002, Baltimore, 18 November 2002

Grid Application Toolkit

The GAT provides functionality through a carefully constructed set of generic high-level APIs, through which an application will be able to call the underlying grid services,

Set of application developer APIs for Grid tools, services and software libraries, (and example implementations) that support the development of grid-enabled applications (open source!)

Usable from any high level “application” (any generic code, Cactus, Triana, Portals, Scripts, …)

Grid 2002, Baltimore, 18 November 2002

GAT

More or less …Set of calls GAT_ToolOrService(arguments)

Your chosen tools/services: resource broker, information server, application manager, grid monitoring, data manager, notification, etc.

Set of APIs for dealing with the GAT (registration, information, errors, fault tolerance)

Grid 2002, Baltimore, 18 November 2002

GAT Engine

When an application makes a GAT-API call, the engine searches through an internal database of adaptors for the requested capability and calls it

Grid 2002, Baltimore, 18 November 2002

GAT Adaptor

Interface between GAT Engine and one or more capabilitiesTranslates user requests to appropriate interface syntax for a capability providerActive adaptors change dynamicallyIncludes “security context”Return appropriate error codes

ExamplesOGSA adaptor (provides many capabilities)Globus adaptor (directly talk to gatekeepers)Adaptors for each GridLab service provider“Local” adaptors (GAT_MoveFile => “cp”, GATFindResource => “localhost”)

Grid 2002, Baltimore, 18 November 2002

GAT Adaptor

Grid 2002, Baltimore, 18 November 2002

GAT: AIM

Abstract Grid capabilities (services) from the application developer.

Application developer concentrates on the functionality as needed by the application.

Hide complexity.

Provides a layer (buffer zone) between applications and the Grid.

Grid 2002, Baltimore, 18 November 2002

The Same Application …

Application

GAT

Application

GAT

Application

GAT

Laptop The GridSuper Computer

No network! Firewall issues!

Grid 2002, Baltimore, 18 November 2002

GAT Architecture

Grid 2002, Baltimore, 18 November 2002

GridLab services

Software environment for Grid-enabling scientific applications

GridLab services, third party services and various core-grid services will be supported by GAT

In the advent of the Open Grid Service Architecture (OGSA), GridLab's architecture will revolve around the notion of services,

all the GridLab services will be OGSA compliant

currently all the services are Web Services based

roadmap for Web Services to OGSA transformation is being prepared (6-8 months from now)

Grid 2002, Baltimore, 18 November 2002

GridLab Testbed

Managed by Masaryk University, Czech Republic

Operational since March 2002

Most of the machines are not dedicated, except one cluster at PSNC

Built on existing middleware:Globus Toolkit 2.0

MDS 2

PKI/GSI for secure access

Provide functional Grid environment for development and testing of GATs, services and their components

Production stability a goal

No long production jobs

Grid 2002, Baltimore, 18 November 2002

More info / summary

www.GridLab.orggridlab@gridlab.org, news@gridlab.orgoffice@gridlab.orgYou’re welcome to join our testbed and test our software, Bring your application and test it with the GAT and our services.We are open for collaborations!Visit us on Tuesday and see the demos at:

Sun Booth, 12.00-1.00 p.m.Argonne Booth, 1.00 p.m – 2.30 p.m.