Grid.it Enabling Platforms for High-Performance Computational Grids

1

Grid.it

Enabling Platforms for High-Performance Computational Grids

Oriented to Scalable Virtual Organizations

Marco VanneschiDipartimento di Informatica, Università di Pisa

Meeting on Grids for Complex Problem Solving MIUR, Rome, March 20th, 2003

National Research Programme 2001 – 2003Strategic Projects on “Enabling Technolgies for Information Society”

FIRB

Project coordinated by the National Research Council (CNR)

2

PISA

NAPOLI

COSENZA

PERUGIA

PADOVA

GENOVA

LECCE

MILANO

PALERMO

ROMA

TORINO

MATERA

PAVIA

BARI

BOLOGNA

National Coordination on Large-scale Platforms and Grids

CNR and UniversityHPC, Parallel Programming, Grid computing, Scientific libraries, Data base and knowledge discovery, Earth Observation, Computational chemistry, Image processing, …

ASIApplications of Earth Observation

CNITTechnologies and HW-SW infrastructures for high-performance communication, Optical technologies, …

CAGLIARI

INFN and UniversityGrid (INFN-Grid, DataGrid, DataTag) ,

e-science applications: Astrophysics,

Bioinformatics, Geophysics, …

Grid.it cost = 11.1 M€ (8.1 M€ funded, of which 1.1 M€ for young researchers) + other synergies

3

FIRB / Grid.it

• A Research Project in ICT.

• Advances and innovations in methodologies and technologies for the realization and utilization of Large-scale platforms and Grids.

• Coordination with the European Community strategies for Next Generation Grid.

• Advances and innovations are evaluated in terms of their capability to solve very critical, real problems in the medium-long term.

• Current standards in ICT are assumed as the technological starting point (e.g. Objects, Web services, Globus), over which new technologies are studied and built.

4

Where innovation is required (1)

• Covering the gap between a set of, even powerful, ICT technologies (Web, Databases, Objects, etc) and their effective integration into a system (Grid) for multidisciplinary applications and virtual organizations.

• All the system levels need new solutions or advancements in current solutions.

• Security mechanisms and policies must be applied to all levels.

5

MPP / SMPMPP / SMP Cluster / Cluster / BeowulfBeowulf

CORBA, Java, CORBA, Java, components …components …

GridGrid

Integration of technologies for seemless HPC applications development

e-e-science, science, e-e-business, … applicationsbusiness, … applications

Next Generation Grid Software TechnologyNext Generation Grid Software Technology

2100 2100 2100 2100

2100 2100 2100 2100

2100

6

• Architecture and basic software technology (compute and storage layer):

– Basic middleware services and data management, Programming environments and tools, Capability to exploit advances in high-performance networking

• Information management and control (information layer):

– Seemless access and management of heterogeneous resources and information

• Knowledge management (knowledge layer):

– Knowledge discovery, decision support (for applications and for the system itself)

Where innovation is required (2)

7

Objectives of FIRB / Grid.it project

Grid hardware-software platforms for high-performance large-

scale processing

including high-performance nodes and networks

New Software Technology for Grid application development:

High-performance Component-based Programming

Environment

Middleware Tools and Grid Portals

Information and Knowledge Services

Security Applications of scientific and technological high interest, able

to stress, and to act as feedback to, all the features of the Grid

technology

8

Research activities and system levels

Programming Tools and Environment

GARR Large-bandwidth Optical NetHigh-perf. Networks

• High-perf. Components

• Scientific Libraries

• Cost models

• Resource Management

• ProblemSolving

Environments

Applications forE-Science&Engineering

• Astrophysics• Geophysics

• Bio-informatics• Comp. Chemistry • Earth Observation

Middleware

Data Intensivecore services

Scheduling Monitoring Commmunic.

GridPortals

SecurityKnowledge

ServicesResourceBrokers

9

Structure of FIRB/Grid.it Project

Research Units:

1. CNR, ISTIDomenico Laforenza

2. CNR, ISTMMarzio Rosi

3. CNR, ICARAlmerico Murli

4. INFNMirco Mazzucato

5. CNITGiancarlo Prati

6. ASIGiovanni Milillo

Technical Board:

Coordinated by Domenico LAFORENZA

Research strategies to integrate the results of the various Workpackages (14).

10

Workpackages

WP1. Grid Oriented Optical Switching Paradigms (Castoldi, CNIT)

WP2. High Performance Photonic Testbed (Giordano, CNIT)

WP3. Grid Deployment (Mazzucato, INFN)

WP4. Security (Talamo, Univ. Roma Tor Vergata)

WP5. Data Intensive Core Services (Mazzucato, INFN)

WP6. Knowledge Services (Turini, Univ. Pisa)

WP7. Grid Portals (Aloisio, Univ. Lecce, ISUFI)

WP8. High-performance Component-based Programming Environment

(Danelutto, Univ. Pisa)

WP9. Grid-enabled Scientific Libraries (Murli, Univ. Napoli & ICAR)

WP10. Grid Applications for Astrophysics (Benacchio, INAF)

WP11. Grid Applications for Earth Observation Systems Application (Milillo, ASI)

WP12. Grid Applications for Biology (Apostolico, Univ. Padova)

WP13. Grid Applications for Molecular Virtual Reality

(Laganà, Univ. Perugia & ISTM)

WP14. Grid Applications for Geophysics (Navarra, INGV)

11

Expected Results & Research Products

Applications and Demonstrators

(WP10, WP11, WP12, WP13, WP14) 4100 K€

CNRINFNASI

Component-based Programming Environment (WP8),

e-science Components (WP9)

Knowledge Services (WP6), Grid Portals (WP7),

Security (WP4) 4500 K€CNR

Photonic, high-bandwidth networks

(WP1, WP2)

2000 K€CNIT

Basic Grid Infrastructure

and Data Core Services

(WP3, WP5)

500 K€

INFN

12

FIRB / Grid.it : synergies

• INFN: European projects, Data Grid, …

• CNR: Agenzia 2000, MIUR Strategic Projects 449/97 (“ex-5%”) 1999 and 2000, PQE2000

• CNIT: projects in advanced networking technology

• ASI: ASI-PQE2000, National Space Program (PSN), …

Other national inititives:

• SPACI (PON): Grid infrastructure UniLe ISUFI, ICAR-CNR Na, UniCal

13

Coordination of national competences

FIRB / Grid.it

CNR projects in HPC and Grid

(Agenzia 2000, 449/97, …), PQE2000

Other initiativese.g. PON-SPACI

ASI projects in EOS and

HPC,PSN (National Space

Program), ASI-PQE2000

CNIT projects on networking

INFN Grid projects,

EC projects,DataGrid, …

14

FIRB / Grid.it: international cooperations

• Global Grid Forum (GGF): Domenico Laforenza in GGF Steering Committee, Area Director of Information Systems and Performance

• Several Europen Community projects and US cooperations

• Europen Community proposals for Next Generation Grid:– CoreGrid coordinated proposal for NoE - DG INFSO/F2

– EGEE coordination (CERN) - IP

15






• Cost models


• ProblemSolving

Environments




Middleware



GridPortals

SecurityKnowledge


16

Software Technology of Grid.it

Specialized components and servicesSpecialized components and servicesscientific, image processing, data mining, knowledge services, …scientific, image processing, data mining, knowledge services, …

Applications

High-performance component-based High-performance component-based programming model and toolsprogramming model and tools

Next Generation Grid MiddlewareNext Generation Grid Middleware

Distributed + HPC HW-SW platforms

17

Software Technology of GRID.IT

General Issues

• Improvements in programmability, easy software development, software reuse and performance portability in a heterogeneous, dynamic application context

• Software adaptability

• High-level services and knowledge management

• Fault tolerance and security

• Coordination with standardization entities (GGF, CCA, …)

• Open source technology

18

Software Technology of GRID.IT - CNR

• Programming Environment

• Resource management and Brokers

• Knowledge Services

• Security at all levels

19






• Cost models


• ProblemSolving

Environments




Middleware



GridPortals

SecurityKnowledge


20

High-performance networks - CNIT

• Very large bandwidth optical fibers (DWDM 10Gb x 32 channels)

• MAN ring

• High-performance “Node” of GARR Network

• Cell (IP/ATM) and packet (IP/Gigabit Ethernet) connections

• A collection of LANs/clusters is seen as a same high-performance machine

• Cost models to be exploited by the Grid.it software technology

21






• Cost models


• ProblemSolving

Environments




Middleware



GridPortals

SecurityKnowledge


22

Grid enabled applicationsINFN – CNR – ASI

• Mainly scientific applications in FIRB/Grid.it.

• However, issues of Knowledge Discovery and Data Mining are intensively investigated and applied in Grid contexts; e.g. finance and PA applications.

• Fundamental feedback for the new Grid.it technology.

23

Conclusion

• FIRB/Grid.it: a research project on innovative methodologies and technologies for Grid computing and their effective utilizations and developments

• Large, qualified national participation and synergies

• Creating the conditions for the strong coordination of the national project with the European initiatives for Next Generation Grid

24

Thanks for attention

Documents

Grid.it Enabling Platforms for High-Performance Computational Grids