Customized Approaches to Fibre-based E2E Services · 2008-12-02 · Ethernet-based E2E services, shared capacity, VLANs (not E2E really). ... Customized Approaches to Fibre-based

Jan Radil, Stanislav Ší[email protected]

www.ces.net

Customized Approaches to Fibre-based E2E Services

2 Dec, 2008 TERENA 1st E2E Workshop, Amsterdam 2


Outline

First attempts and trials

Up-to-date situation in CESNET

Conclusions


For E2E services – dark fibre is really needed.

First mile issues with fibres in many places.

Wireless can be used too but limitations may be rather significant (no WDM, 10G predicted, but higher speeds?, reliability and availability...).

First DF in CESNET in 1999, 2.5 Gb/s PoS.

Ethernet-based E2E services, shared capacity, VLANs (not E2E really).

More DFs have been acquired since then.

First international ‚lambda‘ link in the beginning 2003.

Connected to NetherLight, Amsterdam, the first international lambda exchange point of GLIF (now GOLEs ), 2.5 Gb/s SONET.

Ethernet-based E2E services, dedicated EoSONET channels.




Both first (national and international) 2.5 Gb/s links were single channels, no CWDM or DWDM,

Evident limitation for E2E circuits/channels/lambdas when DFs are available.

‚Grey‘ router interfaces with PoS only, no GE.

SONET/SDH repeaters only, no optical amplifiers.

Problem with international links because no DFs were available at that time (this situation has changed).

But the national infrastructure (MAN/WAN) started to utilize DFs more and more and CWDM/DWDM technology became available and economicaly feasible.




GLIF links for new experiments with ‚lambdas‘ (ie E2E circuits).

The 2.5 Gb/s GLIF link to NetherLight upgraded to 10 Gb/s in September 2004.

Another direct 10 Gb/s GLIF link to StarLight, Chicago from December 2006.


Progress with First attempts and trials (GLIF)


Evolution of the CESNET network 1999 – 2007 (not with all details).


Progress in CESNET networks

2000 2001 2005

2005 2007 2007

2002: first optical amplifiers (EDFA) 2004: first CLA


CESNET2 network 2008

4700 km of DFs

Including 980 km

single DF links


Up-to-date situation in CESNET: WAN


10 links Cisco 15454 MSTP n x 10 Gb/s DWDM transmission system (with ROADM).

10 links (including 3 single fibre lines) deployed withopen photonic n x 10 Gb/s DWDM transmission systems.

5 other links will be lit in December 2008 (including 3single fibre lines).

2 international CBF connections to Bratislava and Wien:open photonic n x 10 Gb/s DWDM transmission system.

No bottleneck on international links anymore.

Customized Approaches in WANs.




DFs with DWDM, CWDM not recommended.

Problems with amplification and number of channels.

But 10G CWDM transceivers available for interesting prices.

If distance is not an issue, use CWDM technology.

Equipment from different vendors in one network.

Open photonic DWDM systems are manufactured by photonic companies/vendors.

Not a ‚home made‘ black box without support and guarantee.

ROADMs and OOO switches have been developed and tested and can be deployed too, it is not restricted to ‚big vendors‘.

Booths both on OFC2008 and ECOC2008 – the global scope company.

Customized Approaches in WANs.






OFC2008 ECOC2008


Open photonic DWDM/CWDM transmission systems are used for end-user 1 Gb/s or 10 Gb/s wavelength connections, if needed.

Open DWDM/CWDM systems are used because of economic reasons, they are not that expensive but provide all required demands.

Particle physics research premises in Praha, along with a number of hospitals.

10 Gb/s wavelength connection to the Barrandov Studios under construction.

Customized Approaches in MANs.


Up-to-date situation in CESNET: MAN


With FTTx deployment, usage of metropolitan fibres lit by high speed open photonic DWDM is expected to be common.

Economic reasons.

‚Openness‘ enables the solution of physical layer interoperability issues and decreases expenses as well as reducing equipment delivery time.

One example - equalizing power levels between two DWDM systems during GLIF 2007 demos.


Up-to-date situation in CESNET: MAN

Demos07


There are neither applications or E2E connections without physical layer (dark fibre) and photonic transmission technology.

Let‘s applications (end users/researchers) decide what technologies really fit their needs.

Meetings/workshops to share information.

We see demand for E2E services but rather permanent or semi-permanent, no on-demand.

Fibre in first miles is the most difficult issue.

With CWDM/DWDM it is easy to provide connectivity 24/7.

So reservation systems not have to be necessary.

Bottleneck are international (transatlantic) links.


Applications and E2E Connections I.

CESNET2


GLIF demo on 8th Annual Global LambdaGrid Workshop 2008, Seattle, USA.

HD video with real optical multicast.

GLIF demos on 7th Annual Global LambdaGrid Workshop 2007, Praha, Czech Republic.

HD, 4k video transmissions.

Multiple 10 Gb/s links to a medieval building with open DWDM transmission system.

HEP - data access and processing for ATLAS and ALICE on LHC, D0 on TEVATRON, STAR on RHIC.

First VINI sites in Europe (Praha, Plzeň).

http://www.vini-veritas.net/

Intercontinental Remote Education on High Performance Computing between Masaryk University Brno and Louisiana State University.

CESNET-TWAREN Hinchu lightpath: peering CESNET-TWAREN, Focused on Multicasting and IPv6.


Applications and E2E Connections II.


LHC (Large Hadron Collider at CERN) distributed data simulation, processing and physics analysis under the WLCG (Worldwide LHC Computing Grid) project (cern.ch/lhc, cern.ch/lcg) for experiments ATLAS (cern.ch/atlas) and ALICE (aliceinfo.cern.ch).

Experiment D0 at Fermilab data simulation and processing.

http://www-d0.fnal.gov/public/index.html

Experiment STAR at Brookhaven data simulation and processing.

http://www.star.bnl.gov/

Praha serves as Tier2 regional computing centre for HEP experiments.

Connection to Tier1 preferred centre Karlsruhe (via GÉANT2).

Connections to BNL, Fermilab, Taipei via GLIF are used as additional connections (it is not part of LHCOPN network between LHC Tier1 centers).

Further references about Czech participation:

www.particle.cz/farm/, http://www-hep2.fzu.cz/d0/


Applications and E2E Connections III.



GLIF 2007 demos solution

B

P

CLA

PB01

PASNET

Dark Fibre, G.652, 13 dB

(DWDM over CWDM)Demos

PCs with XFP

Switches

Carolinum

λ1

λ2

λ3

λ4

λ1

λ2

λ3

λ4

GOLE

StarLight

Chicago

GOLE

NetherLight

Amsterdam

E300

GOLE

CzechLight

Praha

λ1

λ2

λ3

λ4

λ1

λ2

λ3

λ4

DW

DM

MU

X

DW

DM

DE

MU

X

CW

DM

MU

X

CW

DM

DE

MU

X

CW

DM

DE

MU

X

CW

DM

MU

X

DW

DM

DE

MU

X

DW

DM

MU

X

CESNET2

OC-192c OC-192c

N x 10GE

Combination of

10G international lambdas, CWDM, DWDM

Open and ‚big‘ vendor transmission systems



GLIF 2007 demos solution with CLM

Brno

Comodity IP

CL Multicast switch demo

Iperf sender PRG

DWDM XFP

HD sender PRG

DWDM XFP

CESNET 2 DWDM

HD receiver PRG - BRN

LX XFP

HD camera

CLA

CL MCAST SWITCH

HD recv PRG - PLZ

DWDM XFP

DWDM over CWDM

HD receiver BRN

DWDM XFP

HD sender BRN

LX XFP

Iperf receiver BRN

DWDM XFP

Zikova

Comodity IP

Celetna

Iperf receiver PLZ

DWDM XFP

PlzeňComodity IP

LX 10GE optics

DWDM 10GE optics

T Ethernet



GLIF 2007 demos solution with CLM



UCSD

Calit2

PNWGP

CW

6506

GLIF

Mog

HD RX

7609

CW

6506

HD RX HD TX

CW

65066509

e1200

HD RX

StarLight

CzechLight

Brno

C6506

Praha

E300

CAVEwave

(2155)

C-Wave

(2155)

C-W

ave

(2155)

(440)(441)

CESNET

(trunk 440,441)

(2155)

EVL

e600

CESNET

CLM

(441)

I1O1O2

O3

O4

(445)(2155)

IP: 10.200.200.200

MAC: 10:10:10:10:10:10

IP: 10.200.200.200

MAC: 10:10:10:10:10:10

IP: 10.200.200.200

MAC: 10:10:10:10:10:10

‚TX only‘RX onlyRX only

RX onlyTx

Rx

6/2

6/66/106/14

GLIF 2008 CLM Demonstration in Cinegrid demos

Combination of

10G international lambdas, DWDM




GLIF Lightpaths (E2E) to HEP projects participants in the Czech Republic

Nuclear Physics Institute(AS) Bulovka, Praha

Institute of Experimental

and Applied Physics (CTU)

Horská, Praha

Faculty of Mathematics

and Physics (CU), Trója, Praha

Nuclear Physics Institute (AS)

Řež

Faculty of Nuclear Sciences

and Physical Engineering (CTU)

Břehová, Praha

TA

IPE

I

BN

L

FN

AL

Access 1GE

Trunk 10GE

E300

GOLE

CzechLight

Praha

ONS15454

Cisco6506

CW

DM

CW

DM

Institute of Physics

(IoP AS), Na

Slovance, Praha

10GE

GOLE

NetherLight

AmsterdamOC-192c

n x 1GE

over SONET

DF

DF

DF

CWDM

CWDM

AS – The Academy of Sciences of the

Czech Republic

CTU – Czech Technical University, Praha

CU – Charles University, Praha

Combination of

10G international lambdas, CWDM



Customized Approaches to Fibre-based E2E ServiceGOLE

CzechLight

Praha

E300

vini1.cesnet.cz

vinix.cesnet.cz

1GE

1GE

1GE

1GE

CESNET

ONS15454

CESNET2

ONS15454

Plzeň

viniy.cesnet.cz vini2.cesnet.cz

1GE1GE

GOLE

NetherLight

Amsterdam

GOLE

StarLight

Chicago CESNET2

n x 10Gb/s

CESNET EF

n x 10Gb/s

OC-192c

OC-192c

vini network in the USA

CESNET2

PoP Praha

CESNET2

ONS15454

viniz.cesnet.cz

1GE

ONS15454

1GE over

SONET

1GE over

SONET

simple extension

possibilities

Lightpath Chicago/Amsterdam/Praha for VINI

Combination of

10G international lambdas, DWDM


Brno



TWAREN-CESNET via

Taiwan/USA/Amsterdam/Praha

Applications

CESNET2

GOLE

CzechLight

Praha

GOLE

NetherLight

Amsterdam

10GE

TAIWANLight/T

WAREN

Taiwan

Applications

TWAREN

New York City

Chicago

Palo Alto

Los Angeles

OC-192c

OC-12c

OC-12c

Combination of 10G international lambdas Open and ‚big‘ vendor

transmission systems



TWAREN-CESNET via Taiwan/USA/Amsterdam/Praha

15454

@Chicago

15454

@LA

I2 ONS

@MANLAN

IEEAF/VSNL

AMS/TXO2-

NYC/TX01

OC12cS00001

I2 HDXc

@MANLAN

15454

@Palo Alto

OME 6500Asd002a_ome04

7609P

@Hsinchu

15454

@Hsinchu

Taiwan U.S. Europe

CA*net4 :

NYC-Chicago

Back to back

fiber

TAIWANLight

/TWAREN :

Chicago-

Palo Alto-

LA-

Taipei-

Hsinchu

Back to back

fiber

15454

@Taipei

15454

@Praha

E300

@Praha

s1/p1

GE1/17Back to back

fiber

s7/p1

STS97-108GEANT2

DANTE




GEANT2: ams-

pra_LHC_CESNET-

SURFNET_06005

HDXc503/0/1

Amsterdam

5002LE-TWHSZ-

CZPRG_NL(NBD-

TWAREN-CESNET)

BigIron

@Praha

C7609

@Praha

BGP peer

622 Mb/s

1 GE

OC-192c

10 GE

Applications

CESNET2

TE5/7

TE1/3

GE2/9

GE3/2

Applications

TWAREN

drawed by TWAREN/SURFnet/SARA/CESNET



POSN Private Optical Networks for Hospitals

CESNET2

GigaPoP

Praha

CESNET2

GigaPoP

Ústí n.L.

PASNET

CW

DM

MU

X

CW

DM

8 c

ha

nn

els

University

Hospital

Motol

Praha

IP router

MDS switch

Masaryk

Hospital

Ústí n.L.

IP router

MDS switch

Central

Military

Hospital

Praha

IP router

MDS switch

DW

DM

8 c

ha

nn

els

All IP routers and MDS FCIP

switches are connected with 1GE

MDS FCIP - Multilayer Director

Switch Fibre Channel Over

TCP/IP

8 x 10 Gb/s DWDM channels

1 Gb/s CWDM channel (for MDS switch,

IP router connected via PASNET)

Combination of

CWDM, DWDM



Open photonic systems offer freedom of design.

Device itself, following networks.

Important (but not limited) for first mile issues for E2E services.

Open photonic systems can offer beneficial cost advantages.

Ability to make savings on energy consumption, housing space usage, travel to remote huts.

The early adoption of the new photonic technology in the production network.

Independent knowledge of possible solutions.


Conclusions I.


The use of experimental dark fibre facilities (after laboratory testing) to verify feasibility of deployment in the production network.

The use of some services of the experimental facility for CESNET2 or end-user wavelength connections (support of long term experiments with real usage).

Presented examples of E2E services are usable globally.

When DFs are available, of course.

Rather permanet or semi-permanent E2E fibre services then SW reservation systems.

This is our experience with researchers – no additional SW.

But our CLM and CLSs devices include remote management and a SW time scheduler.


Conclusions II.


CESNET is ready to support pilot deployments of E2E fibre services.

Designs and concepts.

Theory or paper works.

CESNET can provide details of already deployed projects.

Practical aspects and hands-on experience.

The vendor is a global company.

Not black boxes manufactured in a garage.

Support, guarantee.


Conclusions III.



Invitation to Future Internet Conference

11-13 May 2009, CLARION CONGRESS HOTEL Prague

Organized by the EC and CESNET during the Czech Republic presidency

http://www.future-internet.eu/events/eventview/article/eu-conference-the-future-of-the-internet.html


















Invitation to 5th Customer Empowered Fiber Network Workshop

May 14 – 15, 2009 Praha, Organized by CESNET

Universitas Carolina, founded 1348.

Previous workshops http://www.ces.net/doc/seminars/cef2007/

http://www.ces.net/doc/seminars/cef2007/



References

[1] Josef Vojtěch, „CzechLight and CzechLight

Amplifiers “, In: 17th TF-NGN Zurich, 2005.

[2] Petr Holub, Josef Vojtech, Jan Radil, et. al., „Pure Optical (Photonic) Multicast“, GLIF 2007 Demo, Praha,

2007.

[3] Josef Vojtěch, Jan Radil, „Transparent all optical switching devices in CESNET“, 25th APAN meeting,

Honolulu, 2008.

[4] Stanislav Šíma, et. al., „ LTTx: Lightpaths to the application, From GOLEs to dispersed end users “, GLIF

2008 Workshop, Seattle, 2008.


Jan Gruntorád, Jiří Navrátil, Josef Vojtěch, Miloslav Hůla,

Václav Novák, Lada Altmannová, Jan Nejman


Acknowedgement



Thank you for your attention.

Documents

Customized Approaches to Fibre-based E2E Services · 2008-12-02 · Ethernet-based E2E services, shared capacity, VLANs (not E2E really). ... Customized Approaches to Fibre-based