Embed Size (px)
Citation preview
PIONIER - Polish Optical Internet: The eScience Enabler
Jarek NabrzyskiPoznan Supercomputing and Networking Center
Digital Divide and HEPGRID Workshop
Poland
Population: 38 mln
Area: 312 000 km2
Since 01.05.04 part of EU
Temperature today: 00C
PIONIER - an idea of „All Optical Network”, facts:• 4Q1999 – proposal of program submited to KBN• 2Q2000 – PIONIER testbed (DWDM, TNC 2001)• 3Q2000 – project accepted (tender for co-operation, negotiations with Telcos)• 4Q2001 – I Phase: ~10 mln Euro
– Contracts with Telbank and Szeptel (1434 km)• 4Q2002 – II Phase: ~18.5 mln Euro
– Contracts with Telbank, regional Power Grids Companies (1214 km)– Contract for equipment: 10GE&DWDM and IP router
• 2H2003 – installation of 10GE with DWDM rep./amp.– 16 MANs connected and 2648 km of fibers installed– Contracts with partners (Telbank and HAVE) (1426 km):
I phase ~ 5 mln Euro • 2004/2005 – 21 MANs connected with 5200 km of fiber
PIONIER - fibers deployment, 1Q2004.
ŁÓ DŹ
TO RUŃPO ZNAŃBYDG O SZCZ O LSZTYNBIA ŁYSTOK
G D AŃSKKO SZALINSZCZECIN
ZIELO NAG Ó RA W RO CŁAWCZĘSTO CHO WAKRAK Ó W RZESZÓ W
LUBLINKIELCE PUŁAW YRADO MKATOW ICEG LIW ICE BIELSKO -BIAŁA
O PO LE
G U BIN WARSZAWA
CIESZYN
SIEDLCE
10 G E nodes10G E links
ŁÓDŹ
TORUŃ
POZNAŃ
BYDG OSZCZ
OLSZTYN
BIA ŁYSTOK
GD AŃSK
KO SZALIN
SZCZECIN
ZIELO NAGÓ RA
W RO CŁAW
CZĘSTO CHOWA
KRAK ÓW RZESZÓ W
LUBLIN
KIELCE
PUŁAW Y
RADO M
KATOW ICEGLIW ICE
BIELSKO-BIAŁA
OPO LE
GU BIN WARSZAWA
CIESZYN
SIEDLCE
PIO NIER nodesInstalled fiber
F ibers planned in 2004PIO NIER nodes planned in 2004
Installed fiber
PIONIER nodes
Fibers started in 2003
Fibers planned in 2004/2005
PIONIER nodes planned in 2004/2005
How we build fibers
• Co-investment with telco operators or self-investment (with right of way: power distribution, railways and public roads)
• Average of 16 fibers available (4xG.652 for national backbone, 8xG.652 for regional use, 4xG.655 for long haul transmission) (2001-2002)
• 2 pipes and one cable with 24 fibers available (2003)• Average span length 60km for national backbone (regeneration
possible)• Local loop contruction is sometimes difficult (urban area -
average 6 months waiting time for permissions)• Found on time ...
Link optimization
• a side effect of an urgent demand for DCM module ;-)• replacement of G.652 fiber with G.655 fiber (NZDS) • similar cost of fiber G.652 and G.655• cost reduction via:
• lower # of amp./regenerators, • lower # of DCM
But:• optimisation is valid for selected cases only and is
wavelength/waveset/link dependent.
optical lines: G.652
Lr= 161.1 km
port 2 Sulechów port 1
12.00dB 12.00dB 6.00dB
1310nm -11.50dB -11.50dB -10.30dB -8.10dB 4.00dB -20.10dB -17.10dB -10.80dB -3.00dB -20.10dB -10.10dB -1.10dB 3.90dB 1310nm
DCM40
10 km 57 km 26 km 42 km DCM40 24 km 4 km
2.20dB 12.10dB 6.30dB 7.80dB 5.00dB 2.10dB
2.00dB 11.80dB DCM40 6.30dB 8.90dB 5.20dB 2.10dB
DCM40
(shared shelf) 4.00dB -7.80dB -16.80dB -21.80dB -1.00dB -7.30dB -16.20dB -19.20dB -1.00dB -6.20dB -8.30dB -9.50dB -9.50dB
6.00dB 12.00dB 12.00dB
Zielona Góra TB
<EDFA<In/Out 1<L/Tx
>L/Rx
R/Rx<
WCM
R/Tx>
SCNFM L/Rx<
L/Tx>>EDFA>In/Out 2
>EDFA>In/Out 1
SCNFM
<EDFA<In/Out 1 <R/Tx
WCM
>R/Rx
(shared shelf)
Poznań MAN Zielona Góra MAN
... 10GE
Switch10GESwitch ..
.
Poznań TB Grodzisk Wolsztyn
15dB
5dB 13dB 13dB
10dB8dB
3dB
3dB
10dB 10dB
5dB5dB
15dB
5dB 13dB 13dB
10dB8dB
3dB
3dB
10dB 10dB
5dB5dB
optical lines: G.655
Lr= 161.1 km
port 2 port 1
12.00dB 6.00dB
1310nm -12.80dB -9.80dB -8.60dB -6.40dB 5.70dB -21.90dB -8.90dB -1.10dB 3.90dB 1310nm
10 km 57 km 26 km 42 km 24 km 4 km
2.20dB 12.10dB 6.30dB 7.80dB 5.00dB 2.10dB
2.00dB 11.80dB 6.30dB 8.90dB 5.20dB 2.10dB
(shared shelf) 4.00dB -7.80dB -14.10dB -22.10dB 3.10dB -2.10dB -4.20dB -5.40dB -10.40dB
6.00dB 12.00dB
Poznań MAN Zielona Góra MAN
10GE Switch
10GESwitch ..
.
Poznań TB Grodzisk Wolsztyn Zielona Góra TB
<L/Tx
>L/Rx
R/Rx<
WCM
R/Tx>
SCNFM
(shared shelf)
Sulechów
<EDFA<In/Out 1 <R/Tx
WCM
>R/Rx
L/Rx<
L/Tx>>EDFA>In/Out 1
SCNFM
3dB
5dB
13dB
8dB
10dB 10dB
5dB5dB
Cost approximately 140KEuro
Cost approximately 90KEuro, cost savings (equipment only): 35%
Community demands as a driving force
• Academic Internet– international connections:
• GEANT 10 Gb/s• TELIA 2 Gb/s• GTS/SPRINT 2 Gb/s
– national connections between MANs (10Gb/s, 622Gb/s leased lambda)– near future – n x 10Gb/s
• High Performance Computing Centers (FC, GE, 10GE)– Project PROGRESS
„Access environment to computational services performed by cluster of SUNs” SUN cluster (3 sites x 1Gb/s) result presented on SC 2002 and SC 2003
– Project SGI „HPC/HPV in Virtual Laboratory on SGI clusters” SGI cluster (6 sites x 1Gb/s)
– Project CLUSTERIX (12 sites x 1 Gb/s) „National CLUSTER of LInuX Systems”
– Project in preparation• National Data Storage system (5 sites x 1Gb/s)
Community demands as a driving force
• Dedicated Capacity for European Projects
– ATRIUM (622Mb/s)
– 6NET (155-622Mb/s)
– VLBI (2x1Gb/s dedicated)
– CERN-ATLAS (>1 Gb/s dedicated per site)
– near future – 6 FP IST
GDAŃSK
POZNAŃ
WROCŁAW
ZIELONA GÓRA
ŁÓDŹ
KATOWICE
KRAKÓW
LUBLIN
WARSZAWA
BYDGOSZCZTORUŃ
CZĘSTOCHOWA
BIAŁYSTOK
OLSZTYN
KIELCE PUŁAWY
RZESZÓW
OPOLE
BIELSKO-BIAŁA
RADOM
GÉANT
10 Gb/s
MetropolitanAreaNetworks
622 Mb/s
155 Mb/s
10 Gb/s
OWN FIBERS
GÉANT
LEASED CHANNELS
Intermediate stage - 10GE over fiber
KOSZALIN
SZCZECIN
PIONIER - the economy behind
Cost reduction via:
• simplified network architecture
IP / ATM / SDH / DWDM IP / GE / DWDM
• lower investment, lower depreciation
ATM /SDH GE
• simplified management
PIONIER - the economy behind...
Cost relation (connections between 21 MANs, per year):
• 622Mb/s channels from telco (real cost) : 4.8 MEuro
• 2.5Gb/s channels from telco (estimate) : 9.6 MEuro
• 10Gb/s channels from telco (estimate) : 19.2 MEuro
PIONIER costs (5200km of fibers, 10GE) : 55.0 MEuro
Annual PIONIER maintenance costs : 2.1 MEuro
Return of Investment in 3 years!
(calculations made only for 1 lambda used)
PIONIER – e-Region
Two e-Regions already defined: • Cottbus – Zielona Gora (D-PL)• Ostrava – Bielsko Biala (CZ-PL)
e-Region objectives:1. Creation of a rational base and possibility of integrated work
between institutions across the border, as defined by e-Europe. (...) education, medicine, natural disasters, information bases, protection of environment.
2. Enchancing the abilities of co-operation by developing new generation of services and applications.
3. Promoting the region in the Europe (as a micro scale of e-Europe concept)
PIONIER – „Porta Optica”
•„PORTA OPTICA” - a distributed optical gateway to eastern neigbours of Poland (project proposal)
• A chance for close cooperation in scientific projects, by the means of providing multichannel/multilambda Internet connections to the neighbouring countries.
• An easy way to extend GEANT to Eastern European countries
PIONIER – cooperation with neighbours
WARSZAWA
ŁÓDŹ
TORUŃ
POZNAŃ
BYDGOSZCZ
OLSZTYN
ELBLĄG
BIAŁYSTOK
GDAŃSK
KOSZALIN
SŁUPSK
SZCZECIN
ZIELONAGÓRA
W ROCŁAW
CZĘSTOCH OWA
KRAKÓW RZESZÓW
LUBLIN
KIELCE
PUŁAW Y
RADOM
KATOW ICEGLIW ICE
BIELSKO-BIAŁA
OPOLE
GE
RM
AN
Y
CZECH REP. SLOVAKIA
UK
RA
INE
BE
LA
RU
S
LITHUANIARUSSIA
e-Region
e-Region
PORTA
OPTICA
PROGRESS (3)
HPC and IST project
WARSZAWA
ŁÓDŹ
TORUŃ
POZNAŃ
BYDGOSZCZ
OLSZTYN
ELBLĄG
BIAŁYSTOK
GDAŃSK
KOSZALIN
SŁUPSK
SZCZECIN
ZIELONAGÓRA
W ROCŁAW
CZĘSTOCH OWA
KRAKÓW RZESZÓW
LUBLIN
KIELCE
PUŁAW Y
RADOM
KATOW ICEGLIW ICE
BIELSKO-BIAŁA
OPOLE
VLBI ATLAS
OTHER PROJECTS?
HPC network (5+3)
CLUSTERIX - National CLUSTER of LInuX Systems
launched in November 2003, 30 months
its realization is divided into two stages:
- research and development – first 18 months
- deployment – starting after the r&d stage and lasting 12 months
in more than 50 % funded by the consortium members
consortium: 12 universities and Polish Academy of Sciences
CLUSTERIX goals
to develop mechanisms and tools that allow the deployment of a production Grid environment with the basic infrastructure comprising local PC- clusters based on 64-bit Linux machines located in geographically distant independent centers connected by the fast backbone network provided by the Polish Optical Network PIONIER
existing PC-clusters, as well as new clusters with both 32- and 64-bit architecture, will be dynamically connected to the basic infrastructure
as a result, a distributed PC-cluster of a new generation with a dynamically changing size, fully operational and integrated with the existing services offered by other projects related to the PIONIER program, will be obtained
results in the software infrastructure area will allow for increasing the portability and stability of the software and performance of the services and computations in the Grid-type structure
CLUSTERIX - objectives
development of software capable of managing clusters with dynamically changing configuration, i.e. changing number of nodes, users and available services; one of the most important factors is reducing the management overhead;
new quality of services and applications based on the IPv6 protocols;
production Grid infrastructure available for the Polish research community;
integration and making use of the existing services delivered as the outcome of other projects (data warehouse, remote visualization, computational resources of KKO);
taking into consideration local policies of infrastructure administration and management, within independent domains;
integrated end-user/administrator interface;
providing required security in a heterogeneous distributed system.
CLUSTERIX: Pilot installation
Architecture
CLUSTERIX: Technologies
the software developed will be based on the Globus Toolkit v.3, using the OGSA (Open Grid Services Architecture) concept
- this technology ensures software compatibility with other environments used for creating Grid systems, and makes the created services easier to reuse
- accepting OGSA as a standard will allow for co-operation of the services with other meta-clusters and Grid systems
Open Source technology
- allows anybody to access the project source code, modify it and publish the changes
- makes the software more reliable and secure
- open software is easier to integrate with the existing solutions and helps other technologies using Open Source software to develop
Integration with existing software will be used extensively, e.g., GridLab broker, Virtual Users Account System
CLUSTERIX: R&D
architecture design accordingly to specific requirements of users
data management
procedures of attaching a local PC cluster of any architecture
design and implementation of the task/resource management system
users account and virtual organization management
security mechanisms in a PC cluster
network resources management
utilization of the IPv6 protocol family
monitoring of cluster nodes and distributed applications
design of a user/administrator interface
design of tools for an automated installation/reconfiguration of all nodes within the entire cluster
dynamic load balancing and checkpointing mechanism
end-user’s applications
High Performance Computing and
Visualisation with the SGI Grid
for Virtual Laboratory Applications
Project No. 6 T11 0052 2002 C/05836Project No. 6 T11 0052 2002 C/05836
Project duration:R&D – December 2002 .. November 2004Deployment – 1 year
Partners:HPC centers
ACK CYFRONET AGH (Kraków)PSNC (Poznan)TASK (Gdansk)WCSS (Wroclaw)
UniversityLodz
End UserIMWM (Warsaw)Institute of Bioorganic Chemistry PAS
IndustrySGI, ATM S.A.
Funds: KBN, SGI
PSNC
TASK
IMWM
WCSS
PŁ
CYFRONET
APPLICATIONS
GRID M IDDLEWARE
VIRTUAL LABORATORY HIGH PERFORMANCE
VISUALISATION APPS EMERGENCY COMPUTING
CENTER
ADVANCED NETWORK INFRASRUCTURE (PIONIER) HPC/HPV INFRASTRUCTURE
Structure
Added value
• Real remote access to the national cluster (... GRID):Real remote access to the national cluster (... GRID):• ASPsASPs• HPC/HPV HPC/HPV • Labour instrumentsLabour instruments
• Better usage of licencesBetter usage of licences• Dedicated Application ServersDedicated Application Servers
• Better usage of HPC resourcesBetter usage of HPC resources• HTCHTC
• Emergency Computing SiteEmergency Computing Site• IMWMIMWM
• Production Grid environmentProduction Grid environment• Midleware we will work outMidleware we will work out
virtual, remote (?)virtual, remote (?)
VERYVERY limited accesslimited access
Main reason -Main reason - COSTSCOSTS
Main Main GOALGOAL - to make - to make accessible on a common accessible on a common wayway
Added ValueAdded Value
Virtual Laboratory
• Remote usage of expensive and unique facilities
• Better utilisation
• Joint venture and on-line co-operation of scientific teams
• Shorter deadlines, faster work
• eScience – closer
• Equal chances
• Tele –work, -science
The Goal
Pilot installation Pilot installation of NMR of NMR SpectroscopySpectroscopy
Optical networkOptical network
HPC, HPV HPC, HPV systemssystems
Data MiningData Mining
... more than remote access
Testbed infrastructure
• Building a national wide HPC/HPV infrastructure:
• Connecting the existing infrastructure with the new testbed.
• Dedicated Application Servers
• Resource Management
• Data access optimisation
• tape subsystems
• Access to scientific libraries
• Checkpoint restart
• kernel level
• IA64 architecture
• Advanced visualization
• Distributed
• Remote visualization
• Programming environment supporting the end user
• How to simplify the process of making parallel applications
Remaining R&D activities
PROGRESS (1)
• Duration: December 2001 – May 2003 (R&D)• Budget: ~4,0 MEuro• Project Partners
– SUN Microsystems Poland– PSNC IBCh Poznań– Cyfronet AMM, Kraków– Technical University Łódź
• Co-funded by The State Committee for Scientific Research (KBN) and SUN Microsystems Poland
PROGRESS (2)
• Deployment: June 2003 – ....– Grid constructors– Computational applications
developers– Computing portals operators
• Enabling access to global grid through deployment of PROGRESS open source packages
PROGRESS (3)
• Cluster of 80 processors
• Networked Storage of 1,3 TB
• Software: ORACLE, HPC Cluster Tools, Sun ONE, Sun Grid Engine, Globus Wrocław
Gdańsk
PROGRESS GPE
EU Projects: Progress and GridLab
What is CrossGrid ?• 5. FP, founded by the EU
– Time frame: March 2002 – February 2005
• Structure of the project
– WP1 - CrossGrid Applications Development– WP2 - Grid Application Programming Environment – WP3 - New Grid Services and Tools
– WP4 - International Testbed Organisation – WP5 - Project Management (including Architecture Team
and central Dissemination/ Exploitation)
Partners• 21 partners• 2 industry partners• 11 countries• The biggest
testbed in Europe
Project structure
WP1
28%
WP2
12%WP3
20%
WP4
32%
WP5
8%
– WP1 - CrossGrid Applications Development
– WP2 - Grid Application Programming Environment
– WP3 - New Grid Services and Tools
– WP4 - International Testbed Organisation – WP5 - Project Management
GRID
MIDDLEWARE
Visualization
Supercomputer, PC-Cluster
Data-storage, Sensors, Experiments
Internet, networks
Desktop
Mobile Access
Hof
fman
n, R
eine
feld
, Put
zer
Middleware
Surgeryplanning &
visualisation
Floodingcontrol
MIS
HEPdata
analysis
Weather &pollutionmodelling
level 1 - special hardware
40 MHz (40 TB/sec)
level 2 - embedded processorslevel 3 - PCs
75 KHz (75 GB/sec)5 KHz (5 GB/sec)
100 Hz(100 MB/sec)data recording &offline analysis
Applications
Migrating DesktopWhat’s the best way to What’s the best way to ‘travel’ ?‘travel’ ?
Roaming
Access
GridsMicrosoft Windows
Linux
take it anywhere
access anyhow
http://ras.man.poznan.pl
GridLabEnabling Applications on the Grid
www.gridlab.org
Jarek Nabrzyski, Project Coordinator
[email protected] Supercomputing and
Networking Center
GridLab Project
• Funded by the EU (5+ M€), January 2002 – December 2004• Application and Testbed oriented
– Cactus Code, Triana Workflow, all the other applications that want to be Grid-enabled
• Main goal: to develop a Grid Application Toolkit (GAT) and set of grid services and tools...:
– resource management (GRMS), – data management,– monitoring, – adaptive components, – mobile user support, – security services,– portals,
• ... and test them on a real testbed with real applications
GridLab Members
PSNC (Poznan) - coordination AEI (Potsdam) ZIB (Berlin) Univ. of Lecce Cardiff University Vrije Univ. (Amsterdam) SZTAKI (Budapest) Masaryk Univ. (Brno) NTUA (Athens)• Sun Microsystems• HP ANL (Chicago, I. Foster) ISI (LA, C.Kesselman) UoWisconsin (M. Livny)
• collaborating with:
– Users!
• EU Astrophysics Network,
• DFN TiKSL/GriKSL
• NSF ASC Project
– other Grid projects
• Globus, Condor,
• GrADS,
• PROGRESS,
• GriPhyn/iVDGL,
• CrossGrid and all the other European Grid Projects (GRIDSTART)
• GWEN, HPC-Europa
GridLab Aims
• Get Computational Scientists using the “Grid” and Grid services for real, everyday, production work (AEI Relativists, EU Network, Grav Wave Data Analysis, Cactus User Community), all the other potential grid apps
• Make it easier for applications to make flexible, efficient, robust, use of the resources available to their virtual organizations
• Dream up, prototype, and test new application scenarios which make adaptive, dynamic, wild, and futuristic uses of resources.
What GridLab isn’t
• We are not developing low level Grid infrastructure,
• We do not want to repeat work which has already been done (want to incorporate and assimilate it …)
– Globus APIs,
– OGSA,
– ASC Portal (GridSphere/Orbiter),
– GPDK,
– GridPort,
– DataGrid,
– GriPhyn,
– ...
…need to make it easier to use
GAT
Application
“Is there a better resource I could be using?”
GAT_FindResource( )
The Grid
GridLab Architecture
The Same Application …
Application
GAT
Application
GAT
Application
GAT
Laptop The GridSuper Computer
No network! Firewall issues!
More info / summary
• www.GridLab.org
• [email protected], [email protected]
• Bring your application and test it with the GAT and our services.
