Carrier Ethernet Title - EANTC: EANTC

Public Multi-VendorCarrier Ethernet

Interoperability Test

Geneva, September 2007

Carrier Ethernet World Congress 2007 Public Interoperability Event

Editor’s NoteFor the third time since 2005,EANTC invited network equip-ment manufacturers to a compre-hensive interoperability test ofCarrier Ethernet solutions. Thisbecame our largest CarrierEthernet event so far — a total of24 vendors participated, repre-senting a large majority of theCarrier Ethernet market. Theoverwhelming vendor interest

clearly shows that Carrier Ethernet services are a hottopic, and that more products are becoming available.

This time, our event focused on new infrastructure solu-tions in the core, metro and access areas. We are proudto report ground breaking progress at the first publicmulti-vendor interoperability verification of new metrotransport technologies: Carrier Ethernet services weretested successfully over a PBT network (with ninevendors) and a T-MPLS cloud (with four vendors).

Our tests of MPLS (with nine vendors) confirmed its statusas a mature and reliable solution for large Carrier Ether-net networks. Multi-vendor interoperability of establishedaccess solutions (Ethernet over copper, fiber, radio links)was confirmed at an advanced level. Furthermore, weduly noted that the number of implementations support-ing Ethernet OAM has grown substantially since our lastevent. Last but very certainly not least we are pleased toreport that multi-vendor provisioning solutions weretested for the first time at a public EANTC event.

The number and level of achievements has certainly beenoutstanding given an extremely heterogeneous test envi-ronment of 65 different systems. Preparations began inApril 2007 and culminated in two weeks of intense hot-stage testing with more than 70 engineers from partici-pating vendors at EANTC in August 2007. We hope thatthe test results as documented in this white paper canprovide guidance to service providers and enterprises ina very progressive and rapidly changing technology.

IntroductionThe third EANTC multi-vendor Carrier Ethernet interoper-ability event addresses four groups of industry stakehold-ers:

• Service providers: To understand the status of newCarrier Ethernet technologies, the choices and impli-cations for every-day operations.

• Enterprises: To gain a better understanding of thetypes and service levels of Carrier Ethernet servicesthey can expect from service providers today.

• Participating vendors: To greatly improve the interop-erability of their implementations, understand thechallenges and opportunities of heterogeneous envi-ronments.

• Industry forums and standards bodies (MEF, IEEE): Toprovide feedback regarding the readiness of stan-dards for real world deployment.

Image 1: Hot-staging at EANTC, Berlin

Based on EANTC’s experience in organizing and execut-ing multi-vendor interoperability events a 10 workingdays, closed doors, hot-staging event was conducted inEANTC’s lab in Berlin, Germany. The results are summa-rized in this white paper.

Carsten RossenhoevelManaging Director

Document StructureParticipants Page 3

Network Topology Page 4

Access Technology Tests Page 6

Metro Technology Tests Page 7

OSS and Provisioning Tests Page 10

MPLS Backbone Tests Page 10

Ethernet OAM Tests Page 12

Resiliency Tests Page 14

End-to-End Services Page 17

References Page 19

2


Participants and Devices

Carrier InvolvementT-Systems engineers participated for the duration of thewhole EANTC hot-staging event. T-Systems engineerswere responsible for performing the Ethernet OAM tests.

Alcatel-Lucent 1850TSS/3207450 ESS-17450 ESS-7

Nokia SiemensNetworks

hiD 6670hiD 6650

Nortel Metro Ethernet Routing Switch 8600Metro Ethernet Services Unit 1850

ANDA Networks ER1006ER1002EE4000 RAD Data

CommunicationsACE-3200ACE-3400ETX-102ETX-202ETX-202ARICi-16LA-210Egate-100OP-1551ASMi-54

Ceragon Networks FibeAir IP-MAX2

Ciena CN 3102CN 3106CN 5060

Cisco Systems ME 3400G-12CSME 3400G-2CSME 3400-24TSME 3400-24FSME 6524Catalyst 3750 MetroCatalyst 6504 (Supervisor 720)7604

Shenick NetworkSystems

diversifEye 8400

Soapstone Networks PNC

SpirentCommunications

Spirent TestCenter

Extreme Networks Black Diamond 12802 Telco Systems,a BATM Company

T-Metro-200T5C-24FT5C-24GT-Marc-250T-Marc-254T-Marc-340T-Marc-380Edge Gate483

Gridpoint Systems VMS

HammerheadSystems

HSX 6000

Harris StratexNetworks

Eclipse (Gigabit) Radio

HuaweiTechnologies

Quidway CX600Quidway CX300AQuidway CX300BQuidway CX200AQuidway CX200B

Tellabs 6305 Ethernet Media Converter6315 Metro Ethernet Node6325 Edge Node6345 Switch Node

Ixia Communications XM12/IxNetwork TPACK LongmornSeagram

Juniper Networks MX960M7i

World Wide Packets LE-311v

MRVCommunications

OS304OS910OS9024-M

ZTE Corporation ZXMP S385

3


ANDAMTU

niper M7i

Tellabs 6315

PLS

Hammerhead

Alcatel-Lucent

o ME 6524

Cisco 6524

MTU

EE4000

ienaTU

5060

T-Marc340

HSX 6000

Telco

ANDA Netw

RAD

7450 ESS-7

CFibeA

o

ER1006

Tellabs 6305

Shenick diversifEyeTelco System

T-Marc380

Telco SystemsMTU

T-Metro200

CiscoME3400 24

tel-Lucent50ESS-1

Ixia XM12

Ceragon FibeAir IP-MAX2

Nor

Nortel MERS

MRV OS304

Telco SystemMTU

T-Metro200

Ixia XM12

Nokia Siemens hiD 6670

TX-202A

isco 7604

ore

Physical Network Topology

Ju

M

Telco Systems

Alcatel-Lucent

ZTE ZXMP S385

Cisc

MRVMTU

OS9024-M

RAD Egate-100

Telco SystemsMTU

T-Metro-200

CiscoME 3400-24FS

HuaweiMTU

Quidway

CM

CN

Telco Systems

TPACK Seagram

HuaweiMTU

Quidway

RAD ETX-202RAD ETX-202

RAD ACE-3400

Telco Systems

Huawei

Alcatel-Lucent

CiscoMTU

Catalyst

Huawei CX200A

Cisco ME3400G-12CS

Telco SystemsT-Marc-380

Ciena CN 3102

Cisco 7604

EdgeGate483

T5C-24FT5C-24G

Eclipse

1850TSS/320

MRVOS910

ANDA ER1006


Telco SystemsT5C-24G

Cisco

MTU

1850TSS/320T-MPLS Metro

MPLS Metr

TPACK

Ixia XM12

Harris Stratex

Shenick diversifEye

RAD ETX-202

SpirentTestCenter

Shenick

World Wide PacketsMTU

LE-311v

Cisco

Ixia XM12Cisco ME 3400G-2CS

SpirentTestCenter

Juniper MX960

Alca74

MRV

MTU

OS910

Quidway CX600

E

Longmorn

Alcatel-Lucent1850TSS/320

Tellabs 6325

Catalyst 6504diversifEye

Catalyst 6504

CX300B CX300A

Tellabs 6345

RAD ETX-102

ShenickdiversifEye

C

C

3750-ME

RAD RICi-16

RAD OP-1551

4


ANDAMTU

Juniper M7i

Tellabs 6315

CoreAlcatel-Lucent

Cisco 6524

Cisco

MTU

MRVMTU

OS9024-M

EE4000

CienaMTU

CN5060

T-Marc-340

HuaweiMTU

CX300A

2

Huawei

Huawei CX200A

Tellabs 6345

Telco

RAD

Cisco 7604

4

7450 ESS-7

Eclipse

MPLS Metro

Harris Stratex

enick diversifEye

Tellabs

4

Alcatel-Lucent7450ESS-1

Ixia XM12

Ceragon F

MRV OS

Ixia

Nokia Sie

Quidway CX600

ETX202A

ME 652
Metro network
Gigabit Ethernet

Access network device

FastEthernet

Core network

Wireless

SDH STM-1SDH STM--16

Metro UNI provider edge

Management system

TesterSDH STM--64

Huawei

RAD

Harris StratexHammerhead

N x E1

ETX-202


HSX 6000

PBT Metro

Nokia Siemens hiD 6650

Ciena CN3106

ANDA ER1002

Extreme BD 12802

ANDA ER1006

Eclipse

Quidway CX600

RAD ACE-3200

CeragonFibeAir IP-MAX2

provider edge

Metro network

Access network

6305

Ixia XM12LE-311vWorld Wide Packets

MRV OS9024-M


Telco SystemsMTU

T-Metro-200

CiscoME3400 24TS

ibeAir IP-MAX2

Nortel MERS 8600

ANDA NetworksER1002

Nortel MERS 8600

TPACK Longmorn

Nortel MERS 8600

Huawei

GridpointVMS

SoapstonePNC

304

SpirentTestCenter

SpirentTestCenter

Nortel ESU 1850

XM12

mens hiD 6670

Quidway CX200B

4

ShenickdiversifEyeTelco Systems

MTU

T-Metro-200

RAD LA-210RAD ASMi-54

EFM bondingover SHDSL.bis

5


Network DesignThere were two design guidelines for the Carrier EthernetWorld Congress Interoperability Network:

• Resemble as much as possible next generationservice provider networks for Carrier Ethernetservices

• Allow for as many vendors as possible to test interop-erability with each other

The participating vendors intended to test three metro/aggregation transport technologies: MPLS (Multi ProtocolLabel Switching), PBT (Provider Backbone Transport,which is being standardized by the IEEE as PBB-TE: Provi-der Backbone Bridge – Traffic Engineering) and T-MPLS(Transport-MPLS). We therefore planned for three metroarea clouds, one for each transport technology.

In addition a broad range of access solutions andnetwork termination devices were to be tested. Technolo-gies such as Ethernet over PDH and SDH as well asEthernet over microwave and Ethernet over DSL wereverified. In order to realize residential and businessservices we attached access areas to each metro cloud.

Today, Metro Ethernet Forum (MEF) standards focus onUNIs (User-Network Interface) – the logical interfacebetween the user and the service provider. MEF servicesare, therefore, defined over that interface. In the testnetwork, the UNI was located between the access andthe metro areas. Since the network was to resemble acarrier’s network, we facilitated the creation of userservices regardless of the type of metro cloud throughwhich they connected, in much the same way that carri-ers strive to offer all network services independent of theunderlying transport solution. Our intent was to verify ifparticipating vendors implemented Carrier Ethernetservices with comparable functionality and resilienceacross all technologies, so that customers would not feelany difference in the service regardless of their point ofattachment to the network.

Last, but most certainly not least, was the interconnectionof the different metro clouds. We decided, together withparticipants, to use MPLS in the core of the network asservice providers commonly use MPLS in their corenetworks.

Interoperability Test ResultsThis time we extended the hot-staging test to full twoworking weeks in order to accommodate the largenumber of participants and the growing number of testareas. The substantial amount of test results collectedduring the hot-staging test are summarized in the follow-ing sections.

Access Technology TestsA number of technologies are available today to carryEthernet frames over xDSL lines, point-to-point radio links,PDH, SDH/Sonet and obviously fiber. Ethernet’s ubiquityis one of the highlighted reasons for its success.

Figure 1: Diverse Access Solutions

The vendors participating in this test presented thefollowing solutions1:

• Tellabs 6325 and TPACK Seagram tested the interop-erability of Ethernet over PDH over SDH. This solutionis typically applied where Ethernet services areprovided over bonded PDH access lines, which inturn are backhauled over SDH.

A total of 16 concatenated E1 containers wereconfigured over an STM-16 interface to transportnative Ethernet frames according to G.8040 (GFPFrame mapping into PDH), G.7043 (VCAT, VirtualConcatenation of PDH) and G.7042 (LCAS, LinkCapacity Adjustment Scheme). LCAS can, whencombined with physical diverse routing of VCG(virtual concatenation group) members, substantiallyincrease the available network capacity whencompared to a traditional SDH full protectionscheme, which reserves spare bandwidth that is notutilized under normal conditions. LCAS can alsochange bandwidth during operation, meaning aVCG can be up- or downgraded in a hitless manner.

1. Please note that we use the term »tested« when reporting onmulti-vendor interoperability tests. The term »demonstrated«refers to scenarios where a service or protocol was termi-nated by equipment from a single vendor on both ends.

Harris StratexHarris Stratex

RAD Egate-100 RAD RICi-16

TPACKTellabs

Ethernet over PDH over SDH

Ethernet over

Ethernet over PDH over SDH

Eclipse Eclipse

Microwave

Seagram6325

RAD LA-210 RAD ASMI-54

Ethernet over EFM (2baseTL)

CeragonCeragon

Ethernet over

IP-MAX2 IP-MAX2

Microwave

6


• RAD demonstrated a similar Ethernet over PDH/SDHservice based on GFP, VCAT and LCAS using theRAD Egate-100 aggregator and RICi-16 CPE whereeach bonded group was mapped to a VLAN into theMetro network.

• Ethernet frames were also carried over point-to-pointmicrowave links. One such solution was presentedby Ceragon using their FibeAir IP-MAX2 solution andthe second by Harris Stratex using their Eclipseproduct line. The radios demonstrated their capabili-ties to drop low priority traffic in the event that thewireless access link suffers degradation (such as seenby bad weather), using Adaptive Coding Modulationbased on QoS parameters. The tests showed thatboth vendors can transport high priority traffic anddrop low priority traffic when radio link capacity isreduced. This ability is different from traditional TDMor SDH links which fail under degraded conditions.

• RAD demonstrated Ethernet over EFM based SHDSL(2baseTL) in LA-210 using eight bonded wiresemploying EFM bonding.

Metro Transport TestsA large number of service providers worldwide plan toupgrade, extend or exchange their metro aggregationnetworks in the near future. Increased requirements forscalability and coverage, range of Ethernet servicesoffered and competitive pricing force carriers to searchfor new solutions for their metro business services, resi-dential triple play offerings and inter-carrier services.

In response to these requirements, standards bodies andindustry forums are working to develop new technologiesto carry Ethernet services.

All metro aggregation devices were assigned to the threegroups based on the transport technology of the vendors’choice: MPLS, Provider Backbone Transport (PBT) andTransport MPLS (T-MPLS). Transport was realized mostly

over Gigabit Ethernet links, however, some links usedSTM-16, STM-64 and point-to-point microwave links.

As an independent test lab we were aiming to test theinteroperability of common baseline services for all trans-port technologies. Since residential and businessservices were going to be offered we asked the partici-pants in each cloud to support point-to-point and multi-point-to-multipoint services. We required each of thethree transport technologies to establish the logicalconnections to support the end-user services and allowedfor resiliency demonstrations within each cloud.

Provider Backbone TransportPBT, like many other networking protocols, has started itslife as a proprietary vendor solution. We decided to testits interoperability for three reasons:

• There is a tremendous interest from service providersto learn more about PBT’s latest status and interopera-bility.

• PBT is being morphed into a multi-vendor industryspecification: It has just begun its way through theIEEE standardization process under the name Provi-der Backbone Bridge – Traffic Engineering (PBB-TE,802.1Qay). The standard will be considered anextension to the already established Provider Back-bone Bridges (IEEE 802.1ah). The current draftcarries the version number »0.0«.

The work on the PBB-TE standard was officiallystarted in 2007 and as a result most of the vendorsimplementations are still based on PBT, so wedecided to base all tests on the PBT specification. Forupcoming test events, we will switch to the PBB-TEstandard.

• We were surprised to see that, albeit the protocol isso young, nine vendors signed up their products forthe PBT interoperability testing.

The following devices were part of the PBT metro cloudand have successfully tested PBT trunk establishment anddata forwarding: ANDA ER 1002 and ER 1006,Ciena CN 3106, Extreme Black Diamond 12802,Hammerhead HSX 6000, Huawei CX600,Nokia Siemens Networks hiD 6650 and hiD 6670,Nortel Metro Ethernet Routing Switch (MERS) 8600,TPACK Longmorn and World Wide Packets LE-311v. Inaddition, Ixia XM12 and Spirent TestCenter emulatedPBT trunk endpoints and generated Ethernet L2 trafficover these ports.

The majority of the PBT trunks were configured manuallysince the current PBT specifications aims to separate thedata plane (implemented in the switch) and the controlplane (implemented externally). Several trunks were

Multipoint Service SupportBoth PBT and T-MPLS are considered pure transporttechnologies over which Metro Ethernet Forumdefined services can be supported. Multipoint-to-multipoint services require explicit configuration ofpoint-to-point tunnels or paths between all the serviceend points, plus a bridging function on those points.

A protocol called VPLS exists for multipoint servicesover MPLS. There are two incompatible IETF specifi-cations, one based on LDP (supported by mostplayers in the industry) and one based on BGP(supported by Juniper and Huawei in addition).

7


provisioned by two multi-vendor management systems:Gridpoint VMS and Soapstone PNC (please see theprovisioning section for a discussion on the topic).

The external control plane must support all devices in thetarget network in order to be able to set up end-to-endtrunks. In a diverse network environment as wasconstructed, provisioning and trunk setups was timeconsuming and performed by hand. In a real worlddeployment carriers are likely to adopt their provisioningsystem to their choice of PBT devices to allow a fasterconfiguration and provisioning of trunks.

PBT is based on existing Ethernet standards, and asexpected data forwarding worked without any prob-lems. Since some of the vendors required ConnectivityFault Management (CFM) in order to establish the PBTtrunks, CFM was used. CFM, however, is not required byall vendors in order to establish the trunks. We alsofound that one vendor had a proprietary version of CFMwhich required assistance from Ixia in order to setup thePBT trunks in our multi-vendor environment.

Figure 2: Provider Backbone Transport Trunks

Two further issues arouse before the PBT trunks could beestablished. In order to exchange CFM messages anEthertype must be used. All vendors require a commonEthertype and since it is not yet defined in the standard,the best solution was to find an Ethertype that all partici-pants can support. The Ethertype chosen was non-stan-dard, however, it allowed all vendors to exchange CFMmessages. The CFM message interval was also a sourceof discussion as some vendors could not change theirCFM interval to the one agreed upon by the group. As aresult, some trunks could not initially be established.Vendors were able to receive new code that allowed the

CFM intervals to be changed and matched the group’sconfiguration and were able to bring up the trunks.

Since there is no standard implementation of PBT, itcomes as no surprise that there were inconsistencies inthe implementations of different vendors with regard toEtherType values, protection and OAM. However, onceagreement was reached, full interoperability was demon-strated by all the vendors. The conclusion is, therefore,that while there are currently differences in the implemen-tation of PBT, interoperability is achievable.

Transport MPLS (T-MPLS)T-MPLS is another new transport technology aiming toprovide a way for carriers to virtualize a wire and offerthe same predictive paths as carriers are used to in theSDH world. T-MPLS is defined in five ITU-T draft docu-ments either approved or under approval (see referencesection). T-MPLS and MPLS have a common forwardingplane: from this perspective, T-MPLS paths (TMP) corre-spond to MPLS Tunnels, while T-MPLS channels (TMC)correspond to MPLS Pseudowires.

The T-MPLS reference standard does not define a specificcontrol plane, but does not preclude the future use of theGMPLS control plane (e.g. RSVP-TE for signaling; OSPF-TE and ISIS-TE for routing). At the time of the hot-staging,only manual configuration was available. Four vendorsparticipated in the T-MPLS interoperability. Given thesmall size of the network, manual configuration was nota problem and T-MPLS Paths (TMPs) were configuredquickly.

Figure 3: T-MPLS Paths (TMPs)

The following devices were part of the T-MPLS metrocloud and have successfully tested and demonstrated theT-MPLS Path (TMP) and T-MPLS Channel (TMC) establish-ment, data forwarding over T-MPLS and T-MPLS Multi-plexing in which a number of TMCs were transportedwithin a TMP:Alcatel-Lucent 1850TSS/320, Tellabs 6345, TPACKLongmorn and ZTE ZXMP S385. Between Tellabs 6345

ANDA

ExtremeBlack Diamond

World Wide PacketsLE-311v

IXIA XM12

Nortel MERS 8600

SpirentHuaweiQuidway CX600

HammerheadHSX 6000

TestCenter

TPACK Longmorn

Metro Network Provider Edge

PBT TrunkMetro UNI Provider EdgeTest Equipment

CienaCN3106

Nokia

hiD 6650

Nokia SiemenshiD 6670 ANDA

ER1002

12802

ER1006

Siemens

Tellabs 6345Ixia XM12

Alcatel-Lucent1850TSS/320 TPACK Longmorn

ZTE ZXMP S385

8


and TPACK Longmorn, T-MPLS over SDH was successfullytested over an STM-64 link. Another SDH link (STM-16)was successfully tested between ZTE ZXMP S385 andTellabs 6345. The Ixia XM12 emulated T-MPLS endpointsand together and generated Ethernet traffic over thesame ports. Spirent TestCenter, also, generated Ethernettraffic to verify data forwarding.

Since T-MPLS borrows the concept of label switchingfrom MPLS some of the participants required IPaddresses for their interfaces and used the ARP protocolto resolve Ethernet addresses for next-hop forwarding.Other vendors did not support ARP — hence the Ethernetaddresses of some of the participating interfaces had tobe entered manually into the partnering T-MPLS switches.

EANTC defined a list of end-to-end Ethernet VirtualCircuits (EVCs). They needed to be established from theaccess network across the T-MPLS cloud and thenhanded over to the core network on a single joint physi-cal interface. This required that T-MPLS equipment at theedge of the T-MPLS cloud would be able to map fromVLAN tags to TMCs. Some implementations had notreached that level of maturity yet — they supported onlyport-based mapping where all VLANs at an access portare mapped into a single T-MPLS channel.

Another issue discovered during the event was the rangeof values for channels (TMCs) and paths (TMPs). By defi-nition, the T-MPLS label range is the same as MPLS. Dueto some vendor software issues a subset of the valueswas initially supported. This was fixed with a softwareupgrade during the test event.

MPLS for AggregationThe tests in this area were based on previous experiencelargely gained in previous Carrier Ethernet WorldCongress and MPLS World Congress interoperability testevents. 16 MPLS vendors took part in the most recenteffort in February this year. The technology is consideredmature and the standards defined by the IETF are enjoy-ing wide deployment. In addition to using MPLS as acore technology, the participating vendors also testedMPLS interoperability as metro/aggregation technology.

Participants in the MPLS cloud proposed to configurepoint-to-point services over Ethernet Pseudowires (PWs)and multipoint-to-multipoint services over Virtual PrivateLAN Services (VPLS).

The MPLS metro/aggregation cloud included 9 vendorsand operated independently from the core cloud (whereMPLS was used as well). MPLS-enabled Carrier Ethernetplatforms participating in this area included Alcatel-Lucent 7450 ESS-1, Cisco 7604 and Juniper MX960.All three routers constructed an LDP-based VPLS domain

— this was the first public interoperability test of LDP-VPLS including Juniper.

The above devices served as Provider Edge routers (PEs)and were surrounded by hierarchical VPLS (H-VPLS)multi-tenant units (MTUs). ANDA EE4000,Ciena CN 5060, Cisco Catalyst 6504, CiscoME 6524, Cisco Catalyst 3750-ME, Huawei QuidwayCX300B, Huawei Quidway CX300A, MRV OS910,MRV OS9024-M, Telco Systems T-Metro-200 andWorld Wide Packets LE-311v all participated as MTUs.The connection between Telco Systems T-Metro-200 andAlcatel-Lucent 7450 ESS-1 was realized over a point-to-point microwave link using Ceragon’s FibeAir IP-MAX2.

A few problems were encountered while constructing theH-VPLS domain. In one case, a signaling session couldnot be established due to a mismatch in the maximumPDU size for the signaling protocol LDP. In another case,a system stripped VLAN headers before forwarding theEthernet frames, a major cause of confusion on thereceiving end of the MPLS tunnel which were expecting aVLAN header.

In one case, a Label Switched Path (LSP) could not beestablished because an implementation did not acceptthe implicit null or explicit null label values it received.

These issues were found in rather new implementationsparticipating at an EANTC interoperability event for thefirst time — an indication that new MPLS implementa-tions still need to be verified for compliance and interop-erability carefully. We are certain that participatingvendors will fix the issues identified in Berlin so that theinteroperability of all participating devices will improvebased on the event.

Figure 4: H-VPLS Metro/Aggregation

In conclusion, MPLS was the only metro transport technol-ogy at the event that demonstrated native support forboth point-to-point and multipoint-to-multipoint services.

Alcatel-Lucent7450 ESS-1


MRVOS9024-M

ANDAEE4000

CienaCN5060

Telco SystemsT-Metro 200-1

CiscoCatalyst 6504

HuaweiQuidway CX300A

HuaweiQuidway CX300B

MRVOS910

CiscoME 6524

Telco SystemsT-Metro 200-2

Cisco7604

MTU

MTU

MTU

MTU

MTU

MTU

JuniperMX960

LDP VPLS(full-mesh connected)

MTU

MTU

MTU

MTU

MTU

MTU

“Spoke“ H-VPLS pseudowires

VPLS connections

VPLS provider edge

H-VPLS Multi-Tenant Unit

CiscoCatalyst 3750-ME

9


As we will describe later in the "MPLS Backbone Tests"section, MPLS also played a key role in enabling multi-point-to-multipoint connectivity for other metro transporttechnologies.

OSS and Provisioning TestsTo solve the provisioning challenges in building servicesprimarily (but not limited to) across PBT networks,Gridpoint VMS (Value Management Suite) andSoapstone PNC (Provider Network Controller) imple-mented service activation tools for the PBT network undertest. Since the PBT standard does not define a controlplane, an external software-based control plane wasintegrated with the network to verify PBT equipmentdeployment readiness. Furthermore, managementvendors strive to decouple services from the underlyingnetwork technology, although the two system typespresent at CEWC 2007 focused just the PBT controlplane for the moment.

The primary tasks were to test the ability:

• To engineer and provision PBT tunnels across multiplevendors’ equipment automatically

• To identify the optimum path for PBT tunnels betweenany two endpoints while meeting its QoS require-ments and optimizing business rules

• To create suitable backup PBT trunks across (if possi-ble) disjunct parts of the network infrastructure

Figure 5: Multi-Vendor PBT Provisioning

Gridpoint VMS and Soapstone PNC configured primaryand backup trunks across the PBT cloud.

Gridpoint VMS interfaced with Nortel MERS 8600 andNortel ESU 1850 equipment. With the PBT trunk groupin place, Gridpoint VMS created a service, delivering arate-limited 5 Mbps service.

Soapstone's PNC participated in the PBT demo where itdynamically configured five services across a multi-nodePBT network comprised of Nortel Networks MERS8600and Extreme Networks BD12802 switches.

Soapstone PNC mediated between multiple resourcerequests from the services/control layer and OSS. ThePNC delivered network modeling and network-to-servicefault as well as SLA correlation.

In a separate step, Gridpoint VMS modeled the completetest network with 80 network elements. The modelingwas carried out manually, not by reading out switchconfigurations. The highly interconnected networkpresented more that 50,000 possible routes across thenetwork to form a service. VMS was used to calculatethe most suitable paths across the network, with regardsto bandwidth and service SLAs. Backup paths werecalculated along disjunct routes where possible. Theresulting end-to-end paths could be read off the screenand configured manually by participating vendors’engineers.

Figure 6: Network Model for Trunk Planning

MPLS Backbone TestsOne major design aspect for service providers will bethe interface between the metro/aggregation and thecore network. We built an MPLS backbone to intercon-nect the three aggregation networks using an Alcatel-Lucent 7450 ESS-7, a Cisco 7604, a Huawei QuidwayCX600 and a Juniper M7i. The backbone providedtransport for both point-to-point and multipoint-to-multi-point Ethernet services. Point-to-point services wereimplemented with Ethernet pseudowires while multipointservices were implemented using VPLS with LDP signal-ing between the four vendors.

World WidePackets LE-311v

Nortel MERS8600

TPACKLongmorn

Metro Network Provider Edge

Primary PBT TrunkMetro UNI Provider Edge

Nokia SiemenshiD 6650

Nortel MERS8600Nortel

Nortel MERS8600

Soapstone PNC

ESU 1850

Extreme NetworksBlack Diamond

Backup PBT TrunkManagement Link

Gridpoint VMS

12802

10


The connections between the three metro/aggregationclouds are discussed below. We made no assumptionsregarding the inter-provider or intra-provider nature ofthe network-to-network interface.

MPLS Metro to MPLS Core Connectivity. An MPLSinterface with multi-segment pseudowires was used forthe connection between the MPLS metro cloud and theMPLS core. This MPLS network-to-network interface (NNI)confined all external signaling between the two networksexclusively to the peering devices. This minimized thereachability information shared across the two networksand maintained common data, control and managementplanes across both networks and their interconnectioninterface. Participating vendors opted for this approach,instead of a native Ethernet interface, in order to allowtesting of multi-segment pseudowire solution.

The participating vendors decided to test the followingcombinations:

• Cisco 7604, Huawei Quidway CX600 and JuniperMX960 interoperated using eBGP with label (RFC3107). In this case, the peering devices functionedas Autonomous System Border Routers (ASBRs).

• Alcatel-Lucent 7450 ESS-1 and Cisco 7604 weretested using LDP and a separate OSPF processbetween the directly attached interfaces.

eBGP and LDP (according to the list above) were used toexchange tunnel label information while three pseudow-ire segments were stitched together for each end-to-endpoint-to-point service:

1. Pseudowire segment within the MPLS metro area2. Pseudowire segment between the metro ASBR and

the core ASBR3. Pseudowire segment inside the core to the egress

point form the core networkFor the multipoint-to-multipoint service, an H-VPLS spoke(between Alcatel-Lucent 7450 ESS-1 and Cisco 7604)was created to join the VPLS instances in the MPLS metroand MPLS backbone.

PBT Metro to MPLS Core Connectivity. In the caseof PBT, where the external control plane solutionsprovided by the participating vendors did not extend intothe core backbone, the data forwarding between PBTand MPLS core clouds was accomplished in the follow-ing two ways:

• IEEE 802.1Q interface between Nortel MERS 8600and Alcatel-Lucent 7450 ESS-7

This connection was used for point-to-point servicesonly. The Nortel MERS 8600 terminated the PBTtrunks and forwarded traffic to Alcatel-Lucent 7450ESS-7 via Ethernet VLANs. The Alcatel-Lucent router

then mapped each of the VLANs into a separateMPLS pseudowire.

• MPLS pseudowire interface between HammerheadHSX 6000 and Cisco 7604.

This link was used to establish multipoint-to-multipointservices based on the HSX 6000‘s ability to map PBTtrunks directly to MPLS pseudowires. Hammerhead wasthen connected to Cisco 7604 in the core of the networkwhich treated the Hammerhead as a spoke H-VPLS MTU.As discussed above, PBT supports only point-to-pointtunnels so instead the Hammerhead HSX 6000 termi-nated multiple PBT trunks into multiple EoMPLS PWs (oneto one mapping). These PWs were then associated as H-VPLS spokes to a common VPLS instance on the Cisco7604 and thus bridging traffic among the spokes.

Figure 7: MPLS Core Logical Connections

T-MPLS Metro to MPLS Core Connectivity. Similarto the previous scenario, no control plane was providedon the T-MPLS side to interface with the MPLS core andtherefore, a native Ethernet interface was used toconnect both networks. The data connections betweenthe MPLS core and the T-MPLS cloud were constructedusing the following two solutions:

• VLAN-based (IEEE 802.1Q) interface betweenTellabs 6345 and Juniper Networks M7i

The TMCs terminating on the Tellabs 6345 weremapped to VLAN tagged Ethernet frames and were

VPLS connections

Metro network provider edge

Alcatel-Lucent7450 ESS-7Juniper

M7i

LDP VPLS(full-mesh connected)

Cisco7604


JuniperMX960

Cisco7604

HuaweiQuidway CX600

PBTMetro

T-MPLSMetro

MPLSMetro

MPLSCore

HammerheadHSX 6000

NortelMERS 7600

Tellabs6345

Alcatel-Lucent1850TSS/320

“Spoke“ H-VPLS pseudowiresIEEE 802.1Q connectionseBGP+Label connections

OSPF+LDP connections

11


s0

s0

CS

then forwarded to the Juniper Networks M7i locatedin the core. The frames coming from the core devicewere mapped to T-MPLS channels based on theirVLAN tags by the receiving Tellabs 6345 device.

• T-MPLS TMCs originating from Alcatel-Lucent 1850TSS320 and stitched together by Huawei QuidwayCX600 to a PW through the core to the MPLS aggre-gation cloud.

The multipoint-to-multipoint service was realizedbuilding a bridge instance on the Tellabs 6345device for a set of T-MPLS channels and physicalport. As a result, the external multipoint service imple-mentation was designed similar to the PBT cloud.

Ethernet OAM TestsWe tested two Ethernet Operation, Administration andMaintenance (OAM) standards in previous Carrier Ether-net World Congress Interoperability events:

• IEEE 802.1ag – Connectivity Fault Management(CFM)

• IEEE 802.3ah – Ethernet in the First Mile (EFM),specifically section 57.

In 2006, only a few implementations existed so we werepleased to receive a great deal moreinterest in both test areas this time.

Native Ethernet fault managementprotocols are essential for the deliveryof high quality end-to-end services tobusiness customers and increasinglyalso consumers (for example for IPTV orVoIP). The service availability is key formany customer applications; it can bemanaged using Ethernet OAM proto-cols. The tests described below are asubset of the rich set of OAM toolsavailable today for Ethernet.

Ethernet Link OAMThe Ethernet in the First Mile IEEE stan-dard extends the Ethernet MediaAccess Control (MAC) layer to addi-tional physical layers such as voicegrade copper cable. As part of thestandard, a set of OAM tools aredefined aiming to help carriers monitorlink operations. We tested the followingthree functions:

• Link OAM Discovery – Detecting the existence andconfiguration of the OAM sublayer in the neighbor-ing Ethernet equipment

• Link OAM Loopback – Configuring a loopback modefor fault localization and link performance testing

• Link OAM Dying Gasp – Notifying the neighborabout an imminent system shutdown

The OAM discovery test involved 28 unique vendor-pairsfor the discovery test. Only four implementations hadproblems detecting neighbor capabilities at this initialdiscovery phase of the testing.

A slightly smaller group of 17 devices supported theloopback testing. Seven of these implementationsmanaged to enter loopback mode against their test part-ner. The problems plaguing this test were wide in rangeand included systems not able to reach “SEND ANY”state or being stuck in loopback state once the test wasdone, just to name a few.

We believe OAM interoperability testing is specificallyimportant: It is very likely that different vendors willsupply the customer premises equipment (CPE) and theswitches terminating the metro/aggregation cloud. Carri-ers need to rely on remote fault localization and perfor-mance verification in order to keep SLAs and offer a

OAM DiscoveryLoopback Mode

AccessMetro/Aggregation

Dying Gasp Messages


Tellabs6315

RAD

Telco SystemT-Marc-34

JuniperMX960

CiscoME 3400G-12CS

MRV

ETX-202

OS9024-M

MRVOS304


CienaCN5060

Tellabs6305

Telco SystemsT-Metro 200

Telco SystemT-Marc-25

CiscoME 3400G-2



CiscoCatalyst 3750-ME

Cisco7604

CiscoME3400-24TS

CiscoCatalyst 6504

CiscoME 6524

MTU

MTU

MTU

MTU

MTU

MTU

MTU

Metro network provider edge

Access network deviceMetro UNI provider edge

12

Figure 8: Ethernet Link OAM Tests


profitable service. While it is positive to see a growingnumber of Ethernet OAM implementations, a lot of workstill needs to be done in testing compliance and interop-erability.

The Dying Gasp test was performed between 16 pairs ofvendors. Participating devices’ roles were asymmetric:The customer-side device generates the dying gasp whilethe network-side device is responsible for receiving andprocessing the message.

This test was successful between almost all pairs. In onecase we monitored an outgoing dying gasp messagewhich was not registered by the receiver and one nodefailed to generate the dying gasp message as it wasbeing powered off.

End-to-End Ethernet Service OAMIn contrast to link OAM, service OAM (as defined in IEEE802.1ag) allows up to eight layers of end-to-end servicemonitoring through different management domains andcan be used across logical connections. This standard isoften called Connectivity Fault Management (CFM).

The major use of CFM for service providers and enter-prises is to verify connectivity across different manage-ment domains. A carrier can define a managementdomain level to be used internally while allowing theircustomers to verify end-to-end connectivity over thenetwork using a different CFM level.

At EANTC, we already tested service OAM – specificallythe Continuity Check functionality – at CarrierEthernet World Congress 2006. Last year, sixdevices from four vendors (early adopters of thedraft standard) had participated in the test.

This time, 16 vendors with an impressive 34 CPEimplementations expressed interest in joining theservice OAM tests. We were able to add LinkTrace and Loopback testing to the test plan basedon the progress of implementations.

Because IEEE 802.1ag is still in draft status — andnaturally, different draft versions are not back-wards compatible — it was important to check theversions supported by participants. Implementa-tions ranged from draft version 1.0 to the mostcurrent draft 8.1. This fact by itself is interesting asseveral vendors are waiting for the standard to befinalized before making any further CFM codeupdates.

At the beginning, we noticed a lot of configurationissues including Ethertype mismatches, Connectiv-ity Check (CC) interval times incompatibility prob-lems as well as a range of domain name formatissues.

Figure 9: Back to Back Service OAM Tests

Once the configuration issues were resolved, we wereable to test a large number of devices successfully. Inorder to use the testing time effectively, 19 CPE vendorsstarted by testing CFM Continuity Check (CC) messagesback-to-back, in directly connected configurations. Six ofthese implementations tested CFM loopback and twotested CFM link trace as well. Figure 9 shows an over-view of the back-to-back tests.


JuniperMX960

Tellabs 6325

RADRICi-16


RAD ETX-202

Nortel

Telco Systems

RADLA-210

IXIA XM12

JuniperM7i

Metro network

LoopbackAccess network device

Link TraceContinuity check

Metro UNI provider edgeTest Equipment

provider edge

T-Marc-380

CiscoCatalyst

MTU

MRV

MTU

MERS 8600

OS9103750-ME

ANDA ER1002

ExtremeBlack Diamond


RADLA-210

IXIAXM12

Tellabs 6325

ANDA ER1006


RAD ETX-202

Nortel MERS 8600


HuaweiCX600

HammerheadHSX 6000

RADRICi-16

TPACKLongmorn

Metro Network

Loopback

Access NetworkLink TraceContinuity Check

Metro UNI Provider Edge Test EquipmentProvider Edge Device

MRVOS910

CiscoCatalyst

CiscoME 6524

Cisco

CiscoME3400G-12CS

6504

12802

Figure 10: End-to-End Service OAM Tests (CFM)

MTU

MTU

MTU

MTU

SpirentTestCenter


7604

13


Further tests were conducted across the completenetwork once it was set up. Connectivity Check testswere conducted with 36 device pairs; Loopback func-tionality was successfully evaluated in nine configura-tions and eight pairs successfully tested CFM’s link trace.End-to-end test pairs had been carefully chosen based oninformation collected from participants in advance inorder to facilitate testing between vendors whosupported the same IEEE 802.1ag draft versions. Wewere pleased to see that careful planning helped achievefull interoperability, however, full mesh testing might havehad different results.

Ethernet Local Management Interface (E-LMI).The E-LMI protocol is specified by the Metro EthernetForum in MEF 16. It enables the customer premisesequipment to request and receive status and serviceattributes information from the Carrier Ethernet Network,including Ethernet Virtual Channel (EVC) and remote UNIstatus information as well as EVC information (includingC-VLAN to EVC map). The information propagated by E-LMI is provided by the other OAM protocols asmentioned above.

Cisco demonstrated E-LMI capabilities using CiscoME3400G-2CS, ME3400-24FS, ME3400-24T andCatalyst 3750-ME, showing end-to-end propagation ofEVC status to the customer equipment. This informationallowed the customer equipment to dynamically reroutetraffic around failed services/VLANs.

Ethernet OAM Summary. Altogether, the EthernetOAM test area proved to be extremely popular incomparison to last year’s test event. Both native Ethernettransport and Carrier Ethernet services are gaining trac-tion in the market so more and more carriers requireEthernet OAM tools, on par with the tools available inexisting networks.

Resiliency TestsA range of resiliency technologies can be employed todeliver Carrier Ethernet services with high availability.Resiliency is the ability of the network to react to node orlink failures and to reconverge quickly. Users expectdifferent levels of resiliency according to service types.TDM services are typically expected to restore servicewithin 50 ms, Voice over IP services in a similar order ofmagnitude. Sub-second restoration is typically acceptedfor video and data services. In the test results presentedhere, Ixia XM12 was used to generate test traffic andcalculate the out of service time during failover and resto-ration.

Resiliency using Link aggregationLink aggregation is typically used for two reasons: Toscale bandwidth and to protect links. This test casefocused on link resiliency.

Link aggregation facilitates the grouping of several inter-faces into a single logical unit. Once the unit is createdthe device can use any set of algorithms to divide trafficacross the link aggregation group (LAG). Traffic across aLAG with n links must be kept below (n – 1)/n of the totalavailable bandwidth (i.e. 1/2 = 50% for two links). Inthis case the LAG can carry all traffic during a single linkfailure. LACP, the Link Aggregation Control Protocol, isresponsible for maintaining links and synchronizingneighboring switches.

A single fiber of a duplex link was disconnected duringeach test run. We measured how fast the unidirectionallink failure was relayed to the other side using LACP andhow quickly both systems agreed not to use the linkanymore and to rearrange for the data streams to takedifferent links.

The following devices participated in the link aggrega-tion tests with LACP: Ciena CN5060, MRV OS9024,Nokia Siemens Networks 6670, Nortel MERS 8600,Telco Systems T-Marc-380 and World Wide Packets LE-311v.

We recorded an out of service time of less than 100 msin all failover situations – a very positive result for carri-ers planning on using LACP as a resiliency mechanism.On the other hand we recorded service outage of almostseven seconds during restoration in one case and inanother test run the restored link was not used to forwardtraffic at all again, pointing to a few implementationissues that can certainly be resolved based on the experi-ence the vendors gained in our event.

Static link aggregation groups (without LACP) wereconfigured between Harris Stratex Eclipse and theTellabs 6345 and a failover time of less than 150 mswas recorded between the two with only 1.4 ms out ofservice time when the link was restored.

Another demonstration in this area was Alcatel-Lucent’s“Multi-chassis LAG” (MCLAG), a solution combiningMPLS-based protocols to provide protection against linkand node failure by terminating the LAG end points overdifferent routers. The demonstration consisted of Alcatel-Lucent’s 7450 ESS7 and 7450 ESS1 which wereconnected to a VPLS domain. The MCLAG control trafficutilized the VPLS service between the two nodes tocontrol the active and standby LAG ports.

Several resilience test scenarios were explored withineach individual metro/aggregation cloud, according tothe abilities of each technology. All three metro areas

14


provided native resilience mechanisms. In addition,several innovative alternatives were proposed by someof the participating vendors to further improve coverageand efficiency of resiliency solutions.

PBT ProtectionResiliency within the PBT domain was accomplished bydefining two PBT trunks (»working path« and »protectionpath«) to the same destination. As the PBT ingress devicedetects that the working path has failed, it switches trafficover to the protected path. The failure detection is doneby CFM (Connectivity Fault Management, see the expla-nation in the box on the right).

A large majority of the PBT vendors present at the eventalso participated in the PBT resiliency testing. We inten-tionally failed each primary path and measured frameloss as an indication of the time required to switch to thebackup path. Since participants had agreed to a Conti-nuity Check (CC) interval of 10 ms, we expected around50 ms out of service time. This was achieved in almostall test configurations — a reassuring result given thatthese were multi-vendor configurations. In very fewcases, out of service times up to 350 ms were measured.

The restoration phase of the test showed less consistentresults. Two of the trunk pairs performed the restorationphase of the test without losing a single frame while anout of service time of up to 800 ms was recordedbetween two other pairs.

PBT allows for an optional external system to calculateand configure primary and backup paths. The GridpointVMS system provisioned both paths in the test and wasable to trigger a switchover manually. No frames werelost during the scheduled rerouting. When switchingback to the primary trunk, we measured an out-of-servicetime of 0.6 ms.

Soapstone PNC provided automated path selection forcustomizable services, thus instantiating, disabling,tearing down and failing over services across the PBTnetwork.

T-MPLS ProtectionT-MPLS path protection and PBT trunk protection aremostly similar in design. Again, we planned to define aprimary and a backup path for each trunk and measurethe out of service time when the primary path was failed.

An interoperability issue was discovered during prepara-tion of the test. One of the vendors involved in the testsupported T-MPLS’s Connectivity Verification protocolversion 1 (CV_v1), two other supported version 0(CV_v0) and the fourth vendor supported both CV types.

Consequently we were able to evaluate only two multi-vendor combinations in this area because each testscenario required three devices (ingress, egress,midpoint): Two devices using CV_v0 and two devicesusing CV_v1 with one of the other units as the bypasspoint.

Tellabs 6345 and TPACK Longmorn-1 used Y.1711 forconnectivity verification (CV) and Y.1720 for resiliencyacross their TMPs and over an STM-64 link. The CV inter-val used was 50 ms which is defined in the standard asFast Failure Detection (FFD).

Alcatel-Lucent 1850TSS/320 tested resiliency againstZTE ZXMP S385 using CV_v1. The two agreed on a CVinterval of 3.3 ms which is in accordance to the ITU-TG.8114 standard. Both units were configured to wait forthree lost CV messages before switching to the backuppath. The results showed a switchover time of 34 ms andan out-of-service time during restoration of 19 ms.

Path Redundancy Using CFMRAD demonstrated the use of CFM on customer edgedevices (outside the metro areas) to determine pathfailure in the network and allow the device to switch toan alternative path.

In this demonstration two TMCs were configuredbetween two RAD ETX-202 devices across the T-MPLScloud. One path was defined as primary while thesecond configured as a backup. The two RAD ETX-202units exchanged CFM Loopback messages across theprimary path and noted when a failure in the primarypath occurred. Upon detection, the units shut down theprimary path and switched to the backup path.

PBT, T-MPLS and Connectivity FaultManagementBoth PBT and T-MPLS use similar mechanisms forconnectivity verification, as recommended inIEEE802.1ag and ITU-T G.8114 respectively. Conti-nuity Check messages are exchanged between theend points in a connection (PBT trunk end points andTMP end points). Once a number of messages areno longer received, the devices declare the pathdead and switch over to the alternative path.

A carrier using PBT and T-MPLS native failure detec-tion mechanisms must keep in mind the resultspresented above in the CFM tests. Even within thetwo technology specific clouds we discoveredinteroperability issues in such areas as Ethertype,intervals and lack of support for CFM.

15


We recorded less than 185 ms out of service time duringfailure. Restoration could not be tested because theimplementation is non-revertive.

Resiliency through MPLSSeveral resilience mechanisms exist in MPLS and mosthas been tested in previous MPLS World Congress Intero-perability (MPLSWC) events. MPLS Fast Reroute (FRR) isone such mechanism that was tested in MPLSWC 2006and showed under 50 ms rerouting times (the whitepaper is available on EANTC’s web site). MPLS FastReroute has the ability to protect both against node andlink failures.

Dual Homed Multi-Tenant Units (MTUs). Dualhomed MTUS are discussed in two specifications relatedto multipoint MPLS services, RFC 4761 and RFC 4762.Since the multipoint service protocol H-VPLS is a realiza-tion of a hub and spoke design with the MTU acting as aspoke and the PE router as a hub, the standards authorssaw a need to address an inherent drawback of hub andspoke – the spoke could easily be separated from thehub and, therefore, lose connectivity. In order to protectthe MTU from such a disaster, an additional backuppseudowires can be defined to a separate provider edge(PE) router. If the primary pseudowire fails, failover to thesecondary pseudowire is facilitated by use of nativeMPLS protocols or other mechanisms.

Four MTUs participated in this test: Cisco Catalyst 3750-ME, MRV OS910, Telco Systems T-Metro-200 andWorld Wide Packets LE-311v. They were homed to acombination of the three PE routers in the MPLS metroaggregation network: Alcatel-Lucent 7450 ESS-1,Cisco 7604 and Juniper MX960.

Figure 11: H-VPLS Dual Homed MTUs

The standard describes the concept of MAC addresswithdrawal to further reduce convergence time by explic-itly withdrawing learned MAC addresses from the VPLSdomain. Alcatel-Lucent 7450 ESS-1 and MRV OS910supported MAC address withdrawal.

A number of the vendors succeeded in switching over tothe backup pseudowire within less than 100 ms.We would have required more testing time to evaluatethe root cause for inconsistent results in other configura-tions. Vendors will investigate this topic further.

MPLS Fast Reroute. Two Telco Systems T-Metro-200MTU devices were connected in a ring topology to theAlcatel-Lucent 7450 ESS-1 router. The primary pathtraversed Ceragon’s FibeAir IP-MAX2 microwave linkwhile the secondary path was directly connected to theAlcatel-Lucent router. We simulated a failure of the wire-less link by shutting down both wireless transmitters. FastReroute mechanisms were evaluated successfully.

Cisco demonstrated a link failure scenario with a trafficoutage under 50 ms in a three-node REP ring built withCisco ME 3400 family switches.

Figure 12: MTU Protection – MPLS Fast Rerouteand Ethernet Resiliency

Native Ethernet ResiliencySeveral native Ethernet solutions were demonstrated atthe event. One such solution was Cisco’s Resilient Ether-net Protocol (REP). The protocol provides an alternative toSpanning Tree Protocol (STP) in the access network tocontrol network loops while handling link and node fail-ures. Cisco demonstrated a link failure scenario in a 3-node REP ring with 3 vlans over Cisco ME 3400 familyswitches.

MPLS Metro

Ixia XM12 Ixia XM12

Ixia XM12

Ixia XM12Telco SystemsT-Marc250

Working Path

Protection Path

Access network devicePhysical connection

Metro Network edge

Metro User edgeMTU

Traffic Generator

Telco SystemsT-Metro200 MRV

OS910


Cisco 7604

Cisco3750ME

Juniper MX960

World Wide PacketsLE-311v-1

MTU

MTUMTU MTU

MPLS Metro Ceragon IP-MAX2

Juniper MX960

HuaweiCX300A

Ixia XM12

Huawei

SpirentTestCenter

Ixia XM12

Telco Systems

Cisco 7604

Alcatel-Lucent7450ESS-1

Cisco ME3400G-12CS

CX300B

T-Metro-200

Working Path

Protection Path

Access network devicePhysical connection

Metro Network edge

Metro User edgeMTU

Traffic Generator

MTU

Ixia XM12

MTU

MTUTelco SystemsT-Metro-200

SpirentTestCenter

Cisco ME3400-24FS

Cisco ME3400G-2CS

16


Cisco also demonstrated its Flex Link feature whichallows a switch to make a local protection decision bydesignating an interface as a backup for another one.Flex Link provides an alternative solution to the SpanningTree Protocol (STP) in order to provide link and noderedundancy in hub-and-spoke topologies. In the demon-stration, a Cisco ME 3400G-12CS was dual homed to aJuniper MX960 (configured as primary NPE) and Cisco7604 (secondary NPE). Upon failure of the primary link,the Cisco ME 3400G-12CS switched to the secondarylink in under 50 ms.

Harris Stratex Networks Eclipse wireless transport plat-form demonstrated resilience of wireless Ethernetnetworks using a ring-based architecture. They demon-strated their Resilient Wireless Packet Ring (RWPR) proto-col which uses Ethernet’s Rapid Spanning Tree protocol(RSTP) in combination with a rapid failure detection algo-rithm to improve convergence times.

Huawei demonstrated their Rapid Ring Protection Proto-col (RRPP) between Huawei CX300B and HuaweiCX300A. The protocol is meant to provide fast conver-gence in ring topologies regardless of the number ofnodes in the ring. In a non-failed condition, master nodewill maintain a port in a blocked mode. Once a linkfailure occurs, the node detecting the link failure willsend a “link_down” message immediately to the masternode, which then opens the blocking port to forwardtraffic.

Telco Systems demonstrated their RingProtection Protocol between TelcoSystems T5C-24F and Telco SystemsT5C-24G. The protocol is meant toprovide fast convergence in anCarrier-Ethernet based ring regard-less of the number on nodes in thering or various types of other vendorssharing the same ring as illustrated inthe MPLS Metro area of the networktopology for this event.

End-to-End ServicesTwo types of end-to-end UNI serviceswere implemented in the test network,spanning all metro clouds and thecore. Point-to-point Ethernet PrivateLine services (EPL) were defined fromUNI to UNI and provided a highdegree of transparency for ServiceFrames between the UNIs. The goal ofdelivering identical Service Frames atthe destination UNI as were seen inthe source UNI was verified using the

Ixia XM12 and Spirent TestCenter. As described in MEF6 “Ethernet Services Definitions - Phase I“ the whole UNIwas dedicated to the service and no Service Multiplex-ing was allowed.

The second end-to-end service configured was Ethernetmultipoint-to-multipoint (EP-LAN). The service was config-ured on all customer edge devices and was verifiedusing Shenick DiversifEye 8400 emulating Triple PlayClients and Servers. The emulated IP based Multicastvideo server, as well as unicast HTTP and SIP servers wasplaced in the MPLS cloud and the triple play clients wereemulated on a dedicated port on various customer edgedevices.

In both cases provisioning the services across the diverseclouds was extremely time consuming. Engineers wererequired to manually configure the services on the UNIsand provide mapping configuration from the UNIs to thetransport (regardless of the transport technologies).

Emulation of ATM and TDM servicesover Metro EthernetRAD demonstrated the emulation of TDM and ATM overthe PBT and T-MPLS metro clouds and across the core.The service emulation used E1 TDM and E1 ATM as istypically seen in cellular base stations. Both E1 TDM and

RAD ETX-202

RAD RICi-16

Tellabs 6325

SpirentTestCenter

ANDA ER1002


MRVOS910



CiscoME3400-24TS

NortelESU 1850

ANDA ER1006

CiscoCatalyst 6504

Harris StratexEclipse

MRVOS9024-M

HuaweiQuidway CX200A

HuaweiQuidway CX200B

Tellabs

CiscoME 6524

MTU

Telco SystemsT-Metro-200

MTU

IXIAXM12

CiscoCatalyst Ciena

RAD LA-210


EPL ServiceAccess network deviceMetro UNI provider edge

3750-ME

6305

CN 3102

Figure 13: End-to-End Ethernet Private Line (EPL) Services

17


ATM services were generated into RAD Hub SiteGateway ACE-3200 tunneled over the metro clouds andterminated on RAD’s ACE-3400. Timing synchronizationwas maintained to prevent clocking issues in the end-to-end ATM and TDM services.

Flexible UNIIn addition to the standard UNI services defined by theMEF, Cisco demonstrated the "flexible UNI" service onthe 7604 router in the MPLS Metro/Aggregation cloud."Flexible UNI" facilitates fine granularity Ethernet framematching, VLAN tag translation and manipulation andflexible service mapping. Cisco showed the “flexibleUNI” using one physical port, where several serviceinstances were defined. Each service instance selectivelymatched single IEEE 802.1Q VLAN tags and doubletags based on unique or a range of or list of tags. Addi-tionally, the demonstration included VLAN tag manipula-tion such as pop, push and translation, and mappingservice instances to VPWS, VPLS and L3/VRF.

Quality of Service TestsSeveral Quality of Service (QoS) tests were planned forthe event. The tests focused on the devices’ abilities toenforce the MEF defined service attributes per UNI, EVCor Ethernet’s CoS bits. One additional test was definedto verify hierarchical QoS capabilities over the UNI.

Four vendors were interested in performing the tests andthe Ixia XM12 tester was used to verify the results,however, due to time constraints, the testing could not beproperly completed.

AcknowledgmentsWe would like to thank Ralf-Peter Braun and ThomasKessler from T-Systems for their extensive support duringthe hot-staging event.

We would also like to thank Mark Lum for his support.

Many thanks to Jonathan Morin and Kyle Price from theUniversity of New Hampshire Interoperability Lab (UNH-IOL) for helping to move things forward at the hot-staging test.

The test would not have been possible without the steadyflow of coffee and smiles supplied by Melanie Schern.

This document has been edited by Jambi Ganbar,Sergej Kaelberer, Carsten Rossenhoevel and GabrieleSchrenk (EANTC).

Acronyms

Term Definition

B-MAC Backbone MAC

B-Tag Backbone Tag

BSC Base Station Controller

BNC Broadband Network Connector

CBS Committed Burst Size

CE Customer Edge.

CE-VLAN Customer Edge Virtual LAN

CFM Continuous Fault Management

CIR Committed Information Rate

CM Color Mode

CoS Class of Service

DTE Data Terminal Equipment

EBS Excess Burst Size

EEPP End to End Path Protection

EIR Excess Information Rate

E-LAN A multipoint-to-multipoint Ethernet service.A LAN extended over a wide area.

E-Line Point-to-Point Ethernet Service similar to aleased line ATM PVC or Frame Relay DLCI.

EVC Ethernet Virtual Connection

FCS Frame check sequence (4 bytes at the end ofEthernet frame)

LAG Link aggregation Group

LSP Label Switched Path

MPLS MultiProtocol Label Switching

MTU Multi Tenant Unit

MTU Maximum Transmission Unit

Metro NPE Metro Network Provider Edge device

OSPF Open Shortest Path First

PBB Provider Backbone Bridge

PBB-TE Provider Backbone Bridge Traffic Engineering

PE Provider Edge.

QoS Quality of Service

RNC Radio Network Controller

RSVP-TE Resource reSerVation Protocol Traffic Engi-neering

SLS Service Level Specifications

STP Spanning Tree Protocol

TMP T-MPLS Path

TMC T-MPLS channel

T-MPLS Transport MPLS

UNI User to Network Interface

Metro UPE Metro UNI Provider Edge device

VPLS Virtual Private LAN Service

VPWS Virtual Private Wired Service

18


ReferencesEthernet Services Attributes Phase 2 (MEF 10.1)

Requirements and Framework for Ethernet Service Protection in Metro Ethernet Networks (MEF 2)

Metro Ethernet Network Architecture Framework - Part 1: Generic Framework (MEF 4)

Metro Ethernet Services Definitions Phase I (MEF 6)

Abstract Test Suite for Ethernet Services at the UNI (MEF 9)

Abstract Test Suite for Traffic Management Phase 1 (MEF 14)

Media Access Control (MAC) Bridges (IEEE 802.1D)

Media Access Control (MAC) Bridges - Amendment 2: Rapid Reconfiguration (IEEE 802.1W)

Virtual Bridged Local Area Networks (IEEE 802.1Q)

Link Aggregation Control Protocol (IEEE 802.3, clause 43)

Media Access Control Parameters, Physical Layers, and Management for Subscriber Access Networks, also referred as“Ethernet in the First Mile“(IEEE 802.3, clause 56 - 67)

Virtual Bridged Local Area Networks - Connectivity Fault Management (IEEE 802.1ag/D6.1-D8.1)

Pseudowire Setup and Maintenance Using the Label Distribution Protocol (LDP) (RFC 4447)

Virtual Private LAN Service (VPLS) Using Label Distribution Protocol (LDP) Signaling (RFC 4762)

Provider Backbone Bridges Traffic Engineering (IEEE 802.1Qay)

Architecture of Transport MPLS (T-MPLS) Layer Network (G.8110.1)

Interfaces for the Transport MPLS (T-MPLS) Hierarchy (G.8112)

Operation & Maintenance mechanisms for T-MPLS layer networks (G.8114)

Characteristics of Transport MPLS equipment functional blocks (G.8121)

Linear protection switching for Transport MPLS (T-MPLS) networks (G.8131)

GFP Frame mapping into PDH (G.8040)

Virtual Concatenation of PDH (G.7043)

Operation & Maintenance mechanism for MPLS networks (Y.1711)

Protection switching for MPLS networks (Y.1720)

Link Capacity Adjustment Scheme (G.7042)

19

EANTC AGEuropean Advanced Networking Test Center

IIR Conferences

Einsteinufer 1710587 Berlin, GermanyTel: +49 30 3180595-0Fax: +49 30 [email protected]

29 Bressenden PlaceLondon SW1E 5DR, UKTel: +44 (0) 20 7017 7483Fax: +44 (0) 20 7017 [email protected]

This report is copyright © 2007 EANTC AG. While every reasonable effort has been made to ensure accuracy andcompleteness of this publication, the authors assume no responsibility for the use of any information contained herein.

All brand names and logos mentioned here are registered trademarks of their respective companies in the UnitedStates and other countries.

Documents

Carrier Ethernet Title - EANTC: EANTC