1Forschungszentrum Jülich 11:00 – 11:20UNICORE – A European Grid Middleware (20 min) Achim Streit (FZJ) 11:20 – 11:30Demonstration of UNICORE in DEISA

1 Forschungszentrum Jülich

11:00 – 11:20 UNICORE – A European Grid Middleware(20 min) Achim Streit (FZJ)

11:20 – 11:30 Demonstration of UNICORE in DEISA(10 min) Andrea Vanni (CINECA)

11:30 – 11:50 GPE4UNICORE(20 min) Hans-Christian Hoppe (INTEL)

11:50 – 12:10 COVS(20 min) Morris Riedel (FZJ)

12:10 – 12:30 Standards in UNICORE(20 min) Dave Snelling (FLE)

OGF19OGF19Software Providers ForumSoftware Providers Forum

Forschungszentrum Jülichin der Helmholtz-Gesellschaft

January 2007

A European Grid MiddlewareA European Grid Middleware

Achim [email protected]


UNiform Interface to COmputing Resources seamless, secure, and intuitive

Initial development started in two German projects funded by the German ministry of education and research (BMBF) 08/1997 – 12/1999: UNICORE project

Results: well defined security architecture with X.509 certificates, intuitive GUI, central job supervisor based on Codine from Genias

1/2000 – 12/2002: UNICORE Plus project Results: implementation enhancements (e.g. replacement of Codine

by custom NJS), extended job control (workflows), application specific interfaces (plugins)

Continuous development since 2002 in several European projects Open Source under BSD license since Summer 2004 Core developers today from Europe: CINECA, ICM, Intel, FLE, FZJ

History lessonHistory lesson


A vertically integrated Grid middleware system since 1997 Provides seamless, secure, and intuitive access to distributed

resources and data Used in production and projects worldwide Open Source under BSD license Features

intuitive GUI with single sign-on X.509 certificates for AA and

job/data signing workflow engine for complex

workflows extensible application support

with plug-ins interactive access with

UNICORE-SSH

Key featuresKey features


EUROGRID

UNICORE Plus

GRIP

GRIDSTART

NextGRID

D-Grid IP

OMII-Europe

EGEE-II

A-WARE

CoreGRID

Chemomentum

eDEISA

DEISA

2008200720062005200420032002200120001999 2009

PHOSPHORUS

UNICORE

UniGrids

VIOLA

OpenMolGRID

ProjectsProjects

More than a decade of German and European research & development and infrastructure projects

BEinGRID


Interactive access (UNICORE-SSH) Improved workflow capabilities

(MetaPlugin for Workflows) High-level API for programming Grids

(Roctopus) DRMAA-based TSI Collaborative Online Visualization

and Steering (COVS) Comfortable configuration tool Site Functionality Monitoring Tool (SIMON)

Recent DevelopmentsRecent Developments


Website: http://www.unicore.euWebsite: http://www.unicore.eu


About 450 users in 200 research projects ¼ of them uses UNICORE

Access via UNICORE to IBM p690 eSeries Cluster (1312 CPUs, 8.9 TFlops)

SoftComp Cluster (264 CPUs, 1 TFlop) JUGGLE (176 cores, 845 GFLops) Cray XD1 (120 CPUs + FPGAs, 528 GFlops)

Usage at the National German Usage at the National German HPC center NICHPC center NIC


Consortium of leading national HPC centers in Europe Deploy and operate a persistent, production quality, distributed,

heterogeneous HPC environment

IDRIS – CNRS, France

FZJ, Jülich, Germany

RZG, Garching, Germany

CINECA, Bologna, Italy

EPCC, Edinburgh, UK

CSC, Helsinki, Finland

SARA, Amsterdam, NL

HLRS, Stuttgart, Germany

BSC, Barcelona, Spain

LRZ, Munich, Germany

ECMWF, Reading, UK

Distributed European Infrastructure Distributed European Infrastructure for Supercomputing Applicationsfor Supercomputing Applications


Dedicated 1 Gb/s network as a basis High performance datagrid via GPFS

Extended to non-AIX Linux like SGI Altix, Mare Nostrum Common Production Environment on all sites Job migration across sites

Used to load balance the global workflow when a huge partition is allocated to a DEISA project in one site

UNICORE as Grid Middleware for workflow applications Co-allocation for applications running on multiple sites at the same time Global data management to include tertiary storage and hierarchical

data management system Science Gateways and Portals to facilitate the access of new, non

traditional users communities

ServicesServices


Usage in DEISAUsage in DEISA

fully-meshed UNICORE infrastructure complex multi-site workflows easily possible heavily used by scientific DECI

(DEISA Extreme Computing Initiative) projects/jobs


Usage in DUsage in D-Grid-Grid

Core D-Grid sites committing parts of their existing resources to D-Grid Approx. 700 CPUs Approx. 1 PByte of storage UNICORE is installed and used

Additional Sites receiving extra money from the BMBF for buying compute clusters and data storage Approx. 2000 CPUs Approx. 2 PByte of storage UNICORE (as well as Globus and

gLite) will be installed as soon as systems are in place

LRZDLR-DFD


Usage in Industry and Usage in Industry and Commercial SupportCommercial Support


New infrastructure based on web services Preserved traditional “User-level” features

Atomic: simple tasks, such as „Execute script“ Client: workstation GUI Workflow: edit, run and monitor graphs of atomic tasks

Additional “User-level” features Portal: web based portal client Streaming: client-server streaming support (for visualization or

media applications) Application development features

Software license management Simplified application deployment

Deployment features User and virtual organization (VO) management

Roadmap to UNICORE 6.0Roadmap to UNICORE 6.0


Architecture of Version 6.0Architecture of Version 6.0


Goal is to provide key software components for building e-infrastructures

Project will demonstrate “proof of concept” with expectation for a follow-on project in FP7

Initial focus on providing common interfaces and integration of major Grid software infrastructures

Committed to standards process Implementing agreed open standards and working with

standards process (OGF/Oasis)University of Southampton, UK (coordinator) University of Chicago, USA

Fujitsu Laboratories Europe, UK NCSA, University of Illinois, USA

Forschungszentrum Jülich, Germany University of Southern California, Los Angeles, USA

Kungl Tekniska Högskolan, Sweden University of Wisconsin-Madison, USA

Istituto Nazionale di Fisica Nucleare, Italy Beihang University, China

Poznan Supercomputing & Networking Center, Poland China Institute of Computing Technology, Beijing, China

University of Edinburgh, UK Computer Network Information Centre, Beijing, China

CERN, Switzerland Tsinghua University, China

Interoperability and Usability of Interoperability and Usability of Grid InfrastructuresGrid Infrastructures


Common interoperable services Database Access (OGSA-DAI), VO Management (VOMS),

Accounting (GGF-RUS), Job Submission and Monitoring (OGSA-BES, ESI/UAS), Portal (Gridsphere)

Capability to add additional services Emphasis on porting and re-engineering work, not developing from

scratch Infrastructure integration

Interoperable security framework Initial gLite/Globus/UNICORE/CROWNgrid interoperability

Quality Assurance Published methodology and compliance test

Repository of open-source, quality assured software services for gLite, Globus, UNICORE and CROWNgrid

Evaluation infrastructure to “test” services

inin


Filling an existing gap for science and business

Portals Integration of a established

commercial portal solution, EnginFrame and an open source solution

A-WARE Workflow Orchestrator Service (WOS) A flexible and powerful workflow

engine Maturing and support of UNICORE 6

technology 5 partners (from IT, UK, FR, DE),

coordinated by CINECA

Provide an integrated Grid solution for workflow-centric, complex applications with a focus on data, semantics and knowledge Based on UNICORE 6 End user focus: ease of use,

domain specific tools, “hidden Grid”

Focus on data services and knowledge management solutions

Provide decision support services for risk assessment, toxicity prediction, and drug design

9 Partners (from PL, DE, EE, NI, IT, FR, CH), coordinated by ICM

inin


Collaborative Online Visualization and Steering (COVS) A framework for scientific simulations and visualizations

In addition to usual post-processing (offline) techniques Enables to view the actual status (online) of

parallel simulations Based on the communication library VISIT Works with all VISIT-enabled scientific

visualizations Implemented as a higher level

service of UNICORE Collaboration server + multiplexer

for geographically dispersed clients Usage of UNICORE security

infrastructure for single sign-on Successfully demonstrated at

OGF18, Europar’06, SC’06, …

A use case of UNICORE 6A use case of UNICORE 6

Documents

1Forschungszentrum Jülich 11:00 – 11:20UNICORE – A European Grid Middleware (20 min) Achim Streit (FZJ) 11:20 – 11:30Demonstration of UNICORE in DEISA