The EDGeS project receives Community research funding XtremWeb-HEP & EGEE CSST-HUST-Wuhan Octobre 11-15th, 2010 Oleg Lodygensky - LAL - [email protected]

The EDGeS project receives Community research funding

XtremWeb-HEP & EGEEXtremWeb-HEP & EGEE

CSST-HUST-WuhanCSST-HUST-Wuhan

Octobre 11-15th, 2010Octobre 11-15th, 2010

Oleg Lodygensky - LAL - Oleg Lodygensky - LAL - [email protected]@lal.in2p3.fr

2Authors : O. Lodygensky HUST : XtremWeb-HEP 2

EDGeSEDGeSBOINC-based DGs

XtremWeb/XWHEP-based DGs

localXtremWeb DG

IN2P3200 PCs

300 (150%)

new public XtremWeb DGEDGeS@home

1.000 PCs

EGEE VO

publicXtremWeb DG

IN2P3

300 PCs6 (2%)

publicXtremWeb DGAlmereGrid

3.000 PCs?

new public BOINC DG

EDGeS@home

10.000 PCs

publicBOINC DG

SZDG

30.000 PCs77.000 (256%)

publicBOINC DG

Extremadura

70.000 PCs22.500 (32%)

localBOINC DG

Westminster1.500 PCs

1881 (125%)

localBOINC DGCorrelationSystems

publicBOINC DGAlmereGrid

0 PCs1.700

publicBOINC DGIbervicis

0 PCs24.000

mailto:EDGeS@home

mailto:EDGeS@home

3HUST : XtremWeb-HEPAuthors : O. Lodygensky 3

EDGeSEDGeS

desktopgrid.vo

voms

lb wmsbdii

an EGEE VO

BOINC-EGEEbridge

EGEE>DGbridge CE

UoW queueSDZGr queueAlmere queue

Fundecyt queue

BOINC-based DGs

UoW, Correlation

Systems local DGs

SZDGr,

Almere,

Fundecytpublic DGs

AR

UICoreService Resource

lfc

bridge

bridge

bridge

bridge

myproxy

ce

wms

ce

4

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky

Bridging EGEE and Desktop Grids

Grid User

X509 proxy

VOMS Proxy

Submits Job with VOMS proxy

Publishes available Resources

Pushes Job

Log

Log

VOMS Admin

Manages VO

Site Admin

Manages Site

Accesses Data with VOMS proxy


Gives Job Status

Gives Accounting and Auditing

VOMS Server

AccountingLogging & Bookkeeping

Meta-scheduler(WMS)

Site Computing Resource

Site Storage

Resource

Grid Admin

Sends back Output Sandbox Sends back

Output Sandbox

VOMS Proxy =X509 proxy with

VOMS extensions

EGEE (Service Grid)Well defined infrastructure:- authentication- authorization- logging and bookkeeping- quality of service (QoS)

A strong effort of standardization

Volunteer PCs

Volunteer PCs

Desktop Grids : Boinc, XWHEPUndefined infrastructure:- anonymity- volatility

How to bridge ? - security - monitoring - logging

DG User

?

Server(scheduler)

5

V 5.7.5-1


XWHEP->EGEEXWHEP->EGEEresource sharingresource sharing

9

V 5.7.5-1


XWHEP :XWHEP :XtremWeb by High Energy PhysicsXtremWeb by High Energy Physics

LAL :•LCG Tiers-2•EGEE seed resource• XtremWeb by High Energy Physics (XWHEP) developer & maintainer

XWHEP is based on XtremWeb 1.8.0. by INRIA.

http://www.xtremweb-hep.org/spip.php?rubrique16

XWHEP ensures security at different levels:

• authentication :✓ SSL and X509 certificates

• authorization and confidentiality :✓ user rights management✓ user groups✓ access rights

• volunteer nodes integrity :✓ application repository✓ sandboxing

• monitoring and bookeeping :✓all messages are logged

XWHEP environment➡ multi users➡ multi applications

http://dghep.lal.in2p3.fr/spip.php?rubrique16

http://dghep.lal.in2p3.fr/spip.php?rubrique16

10

V 5.7.5-1


XWUser Job

Execution Sandbox

Userdata andapplications

Dynamically downloadedXW user dataandapplication

Externaldata

servers

Server certificate

Server public key

Job Mgt

DeployedXW-Client UI

DeployedXW-Computing

Service

PC

Volunteer PC

Data Sandbox

All communicationsare encrypted

Data Mgt

XW Services

Local I/O

XWHEP : Architecture, authenticationXWHEP : Architecture, authenticationand resource integrityand resource integrity

Authentication: - login/password - X509 cert. Node integrity:

- sandboxing

Authentication: - login/password - X509 cert.

XW Coordinator(scheduler)

XW Data Repository

24

V 5.7.5-1


VOMS Server

Meta-scheduler(WMS)

DG User

jLite : proxy init

XW Coordinator(scheduler)

jLiteby Oleg Sukhoroslovhttp://code.google.com/p/jlite/

Public worker

Group worker

Private worker

User X.509 Cert

User X.509 proxy

Server certificate

Server public key

Security, monitoring and logging are the main issues in Pilot Jobs. (http://edms.cern.ch/document/855383)

XWHEP->EGEEXWHEP->EGEEbridgingbridging

XW Bridge

Submit Pilot jobw/ user X509 proxy

Retreived signedDG user job

DG user job submissionwith X509 proxy

DownloadDG userX509 proxy

Pilot Job

Private worker

WN


Pilot Job submitted as EGEE Job

DG user job deployment, status, results

http://code.google.com/p/jlite/

http://edms.cern.ch/document/855383

27

V 5.7.5-1


EDGeS MonitoringEDGeS Monitoring

28

V 5.7.5-1


How to port an application fromHow to port an application fromXWHEP to EGEEXWHEP to EGEE

•EGEE is made of Linux computing resourcesEGEE is made of Linux computing resources•XWHEP applications must have a Linux XWHEP applications must have a Linux

versionversion

•There is no specific need to recompile nor There is no specific need to recompile nor relink binariesrelink binaries

29

V 5.7.5-1


EGEE->XWHEPEGEE->XWHEPresource sharingresource sharing

30

V 5.7.5-1


Bridging EGEE and Desktop Grids

Grid User

X509 proxy

VOMS Proxy

Submits Job with VOMS proxy

Publishes available Resources

Pushes Job

Log

Log

VOMS Admin

Manages VO

Site Admin

Manages Site



Gives Job Status

Gives Accounting and Auditing

VOMS Server

AccountingLogging & Bookkeeping

Meta-scheduler(WMS)


Site Storage

Resource

Grid Admin

Sends back Output Sandbox Sends back

Output Sandbox

VOMS Proxy =X509 proxy with

VOMS extensions

EGEE (Service Grid)Well defined infrastructure:- authentication- authorization- logging and bookkeeping- quality of service (QoS)

A strong effort of standardization

Volunteer PCs

Volunteer PCs

Desktop Grids : Boinc, XWHEPUndefined infrastructure:- anonymity- volatility

How to bridge ? - security - monitoring - logging

DG User

?

Server(scheduler)

31

V 5.7.5-1


gLite ensures security:

• authentication :✓ X509 cert. and VOMS ext.

• authorization and confidentiality :✓ VOMS server

• monitoring and bookeeping

EGEE->XWHEP bridge :EGEE->XWHEP bridge :authentication, authorization, logging, authentication, authorization, logging,

monitoringmonitoring

XWHEP ensures security at different levels:

• authentication :✓ SSL and X509 certificates

• authorization and confidentiality :✓ user rights management✓ user groups✓ access rights

• volunteer nodes integrity :✓ application repository✓ sandboxing

• monitoring and bookeeping :✓all messages are logged

35

Volunteer PCs

Volunteer PCsXtremWeb

User

V 5.7.5-1


QM WS client

QM

XW

plu

gin

WS

Int

erfa

ce

P-GRADE Portal

EDGeSEDGeSARAR

EGEE -> XtremWebEGEE -> XtremWebresource sharingresource sharing

VOMS Server

LCG CE

Meta-scheduler(WMS)

EDGeS CE XW Coordinator(scheduler)

V 5.7.5-1

HUST : XtremWeb-HEPAuthors : O. Lodygensky 8

V 5.7.5-1


Standard gLite tools usage

V 5.7.5-1


Documents

The EDGeS project receives Community research funding XtremWeb-HEP & EGEE CSST-HUST-Wuhan Octobre 11-15th, 2010 Oleg Lodygensky - LAL - [email protected]