Public Multi-Vendor Mobile Backhaul Interoperability Test ......Nokia Siemens Networks hiD 6650 Flexi WCDMA BTS RNC and Core Network Emulator (Racel) Nortel Metro Ethernet Routing

Public Multi-VendorMobile Backhaul

Interoperability Test

MPLS and Ethernet World CongressParis, February 2008

METRO ETHERNET FORUM

2

MPLS and Ethernet World Congress 2008 Mobile Backhaul Interoperability Event

Editor’s NoteThis year marks the tenth anni-versary of the MPLS WorldCongress and the sixth EANTCmulti-vendor interoperability testevent at the congress. Since2003, we have showcased thelatest carrier transport infrastruc-ture every year. Are thingsbecoming "business as usual" inour interoperability events?A few times during the hot-

staging in the past weeks, I wished our work could bethat easy. Wouldn’t MPLS testing be accomplished sometime in the near future? (In fact, things are few stepsaway from perfection. See sections below.)Unfortunately, consumers worldwide seem to agree ontwo challenging ideas lately: They begin to favor watch-ing digital TV and on-demand video over their broad-band access lines and desire ubiquitous mobile dataaccess with as much bandwidth as possible. (Well, infact our industry was keen on jumping on these opportu-nities, frequently ignoring the challenges, but that is adifferent story.)At the same time, business customers are asking forpremium mobile data services and a large footprint ofEthernet VPN services. Even worse, none of thesecustomer groups is willing to pay a premium price for theservices anymore now that almost all major marketsworldwide are liberalized and competitive.This scenario set the scene for our latest interoperabilitytest. The lab setup resembled the major service providerchallenges — to implement an integrated, multi-service,next-generation core, aggregation and access networksupporting high quality mobile backhaul services.After four months of careful preparation, we were over-whelmed with more than 85 devices from 15 participat-ing vendors in our lab. 50 vendor engineers installedmore than six tons of equipment mounted in 16 racks.

This was definitely not "business as usual" showing thatthe increasingly complex user requirements are takingtheir toll.The EANTC technical team, complemented by engineersfrom the University of New Hampshire InteroperabilityLab and from T-Systems (Deutsche Telekom Group), foundreassuring progress in interoperability. Not only didimplementations of the three major aggregation technolo-gies (MPLS, PBB-TE, T-MPLS) interwork really well withintheir respective areas, they also interfaced decently withthe MPLS core once the MPLS, PBB-TE, and T-MPLSvendors had agreed on a common interface. (The E-NNIinterface is in needof better defini-tions, though. Seesection below).Furthermore, resil-iency test results metour expectationsacross all thevendor combina-tions we tested. Notall vendors,however, partici-pated in resiliencytesting – the resultssection containsmore specificdetails. Also, ATMand TDM servicesacross the packet networks and the diverse clockingmethods worked very well in most cases.The fine art of mobile backhaul seems to be the knowl-edge of how to put the pieces together. Fortunately wehad the support of both the IP/MPLS Forum and theMetro Ethernet Forum (MEF) mobile backhaul standard-ization projects.Vendors also used quite a few ITU-T and IETF standardsfor mobile backhaul transport and services. There is awealth of solutions and options out there. The majorupcoming work for the standards organizations will be to

Hot-staging Test Setup at EANTC, Berlin

Carsten RossenhoevelManaging Director

Document StructureParticipants Page 3Network Design Page 4Physical Topology Page 6Mobile Backhaul Tests Page 8Metro Technology Tests Page 10MPLS Backbone Tests Page 12Ethernet OAM Tests Page 13Resiliency Tests Page 15Quality of Service Tests Page 16End-to-End Services Page 18

3


streamline and condense the specifications in collabora-tion with each other.The traditional »sneak preview« service provider guidedtour at EANTC during the hot-stage event enjoyed atten-dance of network designers from major European incum-bent service providers which confirmed that next-genmobile backhaul is a hot topic for the big carriers now(or will become so soon). Competition is expected to betough so careful yet quick equipment selection isrequired. Mobile operators, some of which are knownfor being very conservative (for good reasons), are usingstandard DSL services to offloading bulk data, but oncefull integration of voice and data services is requiredover Carrier Ethernet and MPLS networks this approachmay not be satisfactory. All aspects will have to workalike: Network performance, availability, management,clocking and the infamous quality of service.Given the complexity of the infrastructure, carriers aremost likely to choose a mix of systems from multiplevendors. With next-gen mobile backhaul, interoperabilitybecomes even more important than before: Core andaggregation have traditionally been purchased fromdifferent vendors, but now wireless/microwave access,cell-site CPE systems and ATM/TDM interworking unitsare added to the picture.This report summarizes the test areas and our findings indetail. We hope you will find it beneficial.

Carrier InvolvementT-Systems engineers participated for the duration of theEANTC hot-staging event. T-Systems engineers wereresponsible for performing the Ethernet OAM tests.

Participants and Devices

Vendor Participating Devices

Alcatel-Lucent 1850 TSS-31850 TSS-51850 TSS-401850 TSS-3207450 ESS-17450 ESS-7

Ceragon Networks FibeAir IP-MAX2

Ciena CN 3106CN 5060

Cisco 76067604CRS-1 [4 slot]

Ericsson Marconi OMS 2400RBS2409

Harris StratexNetworks

Eclipse (Gigabit) Radio

HuaweiTechnologies

SmartAX MA5600TOptiX OSN 1900OptiX OSN 3900Quidway CX600Quidway CX380Quidway CX300BQuidway CX200BQuidway NE5000EQuidway NE40E

Ixia XM12

Lightstorm Networks Brooklyn-10

MRVCommunications

OptiSwitch OS304OptiSwitch OS910-MOptiSwitch OS9024-M

Nokia SiemensNetworks

hiD 6650Flexi WCDMA BTSRNC and Core NetworkEmulator (Racel)

Nortel Metro Ethernet RoutingSwitch 8600Multiservice Switch 15000

RAD DataCommunications

ACE-3200ACE-3400ETX-202

SpirentCommunications

TestCenterAX/4000

Telco Systems, aBATM Company

T-Metro-200T5C-XGT5C-24FT5C-24GT-Marc-250T-Marc-254T-Marc-340T-Marc-380

Vendor Participating Devices

4


Network DesignWe designed the test network with the target to build onthe services and technologies tested during last year’sMPLS World Congress and Carrier Ethernet WorldCongress interoperability events, and to add next-gener-ation mobile backhaul services on top.In all EANTC interoperability events we aim to constructnetworks that will resemble real world deployment asmuch as possible. This time, the addition of Mobile Back-haul to the application mix heightened scalability, resil-iency and flexibility requirements.Our lab network design resembles a number of currentcarrier topology scenarios:

• Access: The fixed network area adjacent to thecustomer site. Business and residential devices areterminated at the access. A range of access technolo-gies such as Ethernet over SONET/SDH, Ethernetover microwave radio links and Ethernet over copperwere evaluated.

• Radio Access Network (RAN): The wireless networkreaching out to mobile end-systems, providing voiceand data services (where mobile video can be a partof the data service). In contrast to the fixed accessnetwork, the RAN uses completely different transporttechnologies like 3GPP (GSM, WCDMA,CDMA2000), 4G (LTE) or WiMax. We used GSMand WCMDA as part of the mobile applicationdemonstrations.

• Metro/Aggregation: The network area in whichmany different services and access technologies areaggregated. We verified diverse technical solutionsto implement metro networks for Carrier Ethernetservices. On the access facing devices, MEF-definedUNI interfaces were tested while in the core facingdevices Network to Network (NNI) connections wererealized. Three metro/aggregation clouds wereconstructed: MPLS, Provider Backbone Bridge TrafficEngineering (PBB-TE) and T-MPLS.

• Core: We tested an MPLS-based backbone infrastruc-ture, transporting multiple packet-based services. Thecore supported connectivity between the differentaggregation clouds and end-to-end services.

The three metro areas were connected over the core.Each emulated a separate metro service provider admin-istrative domain. The backbone itself was administra-tively divided from the metro areas, resembling a long-haul inter-carrier network.

Mobile Backhaul IntroductionMobile Backhaul is a term used to describe connectivitybetween mobile base stations and radio controllers. In3G services, a base station is called a NodeB. Individualnames for radio controllers also vary — GSM: basestation Controller (BSC), 3G: Radio Network Controller(RNC). We will use base station and base station control-lers as generic terms in this white paper.Traditionally, the interface between 2G base stations andradio controllers was based on a number of parallelTDM circuits utilizing E1 or T1 links. Since cell phones’main usage was voice communication, such a decisionwas logical as TDM was the standard way to transportvoice.Since managing a large bundle of E1 circuits (typicallydelivered as channelized STM-1) was cumbersome at theradio controllers, the transport protocol was changed toATM in the 3G world to facilitate better multiplexing andscalability. Still, E1 bundles were used, but they carriedvoice services over ATM inverse multiplexing (IMA)instead of directly using E1 channels. Data and voiceservices were separated only at the radio controller, trav-eling the last mile together.The amount of data traffic in 3G networks has grownfurther, so E1-ATM bundles became insufficient. Basestation vendors invented hybrid uplinks (E1-ATM forvoice, Ethernet/IP for data). In some of today’s mobilenetworks, data traffic is handed to the packet switchednetwork (PSN) already at the base station.Several market studies show that the transport networkcosts account for 20–30% of a mobile operator’s opera-tional expenditure (OPEX). The service revenue growthis, however, not matched by service growth rate. Theaverage revenue per user (ARPU) has decreased by 25%in the last two years for European mobile operators.

Figure 1: Mobile Backhaul Scope

AggregationNetwork

CoreNetwork

RNC

BSCMSC

GGSN

SGSN

Mobile Backhaul Focus

AccessNetwork

Radio

SGSN — Serving GPRS Support NodeGGSN — Gateway GPRS Support Node

MSC — Mobile Service Switching Center

BSC — Base Station ControllerRNC — Radio Network Controller

AccessNetwork

Radio

5


Clearly hybrid offload will not be the end of mobile back-haul development. The amount of base stations per areais growing and new standards like LTE (»4G«) or WiMaXwill require even more bandwidth (in these standardsTDM interfaces are not even an option). In order tosupport all these, the number of E1/T1 links must befurther increased and for WiMAX and LTE new interfacesmust be supported. As a result service providers areconsidering mobile backhaul over packet switchednetworks (e.g. MPLS, PBB-TE and T-MPLS), as theseprovide enough bandwidth for the predicted increase indata traffic and are more cost effective than the currentTDM networks.To test mobile backhaul properly, not only the end goal(voice and data over packet networks) needs to be veri-fied; the migration paths are even more important. Thou-sands of base stations will not be upgraded immediately(some never). So in addition to the increase of band-width requirements, new types of interfaces must besupported between the new base stations and the newbase station controllers while still supporting the olderinterfaces that are already in use.In this interoperability test event, we investigated a widerange of solutions to meet the requirements facing mobileoperators. These requirements and solutions are summa-rized in the table below and are expanded on in thiswhite paper.

Table 1: Mobile Backhaul Service Aspects

Mobile operators looking to connect base stations withtheir respective network controllers over a packetnetwork are facing several options:

• Splitting the real time traffic (typically voice andcontrol traffic) at the base station using a GenericInterworking Function (GIWF). This method, calledmixed mode transport, allows operators to keep theircurrent network operation using TDM for backhaulwhile off-loading the additional traffic generated byHSDPA or WiMax systems to a packet basednetwork. This solution allows the timing and synchro-nization to be provided by the original TDM trans-port.

• The second option uses a Generic Interworking Func-tion (GIWF) to transport all traffic over the MetroEthernet Network. This option requires that the GIWFis able to perform at least one of the encapsulationmethods transporting TDM or ATM traffic across theMEN and is able to provide clock synchronization tothe base station. The GIWF can either be located atthe base station site or at the edge router (centraloffice) site.

• When the cell sites include base stations equippedwith legacy and Ethernet interfaces, the legacy inter-face could be used to deliver voice and clocksynchronization, while the Ethernet interfaces couldbe used to offload broadband data applications fromthe legacy network.

• New base stations are being introduced to themarket equipped with native Ethernet interfaces thatare able to transport all traffic over the Metro Ether-net Network. This is enabled by standardized (3GPPIub over IP) and proprietary (Abis over IP) interfacesbetween the base station and radio controllers.

Mobile OperatorRequirements Solutions / Test Areas

Support bandwidthgrowth and acompetitive costmodel

Packet services in the RadioAccess Network (RAN)

Support a diverse setof interface types atcell site

MPLS ATM and TDMPseudowires, Ethernet andIP

Implement network-based clocksynchronization

IEEE 1588v2, Real TimeProtocol (RTP), SynchronousEthernet, Network TimeProtocol Version 3 (NTPv3),external reference clock

Resiliency on parwith TDM network

MPLS, PBB-TE, T-MPLS andnative Ethernet resiliencymechanisms

6


Physical Network Topology

RAD ACE-3200

7450 ESS-1Alcatel-Lucent

Telco SyT-Marc

Telco SystemsT5C-XG

Harris StratexEclipse(Gigabit) Radio

Huawei SmartAXMA5600T

EricssonRNC Emulatio

Network Co

1850 TSS-3Alcatel-Lucent

MPLS Metro

Telco SystemsT-Marc-340

Ixia XM12

RAD LA-110

T-MPLS Metro

T

HuQuidwa

Alcatel-Lucent7450 ESS-7

Cisco7604

HuQuidwa

Quid

Telco SystemT-Marc-380

Cisco

CiscoCRS-1

Ixia XM12

Alcatel-Lucent1850 TSS-320

Quidway NE40EHuawei

Cisco 7606

MRV OptiSwitchOS910-M

HuaweiOptiX OSN 3900

Telco SystemsT-Metro-200

Ixia XM12

Marconi OMS 2400Ericsson


Ciena CN 5060

LightstormBrooklyn-10

SpirentTestCenter



EricssonMarconi OMS 2400

Huawei OptiXOSN 1900

CeragonFibeAir IP-MAX2

RAD ACE-3200






Ixia XM12



SpirentTestCenter

HuaweiQuidway CX200B

RAD ACE-3400

EricssonMarconi OMS 2400

Telco ST-Met

HuaweiQuidway CX300B


7


MSS 15000Nortel


PBB-TE Metro


ystemsc-250

RAD ETX-202

RAD ETX-202

EricssonRBS2409


on

re

o

Ixia XM12



Ixia XM12

T5C-24GTelco Systems

SpirentTestCenter

SpirentTestCenter


Spirent

SpirentTestCenter

SpirentTestCenter

uaweiay NE5000E

aweiay CX380

Huaweidway CX600

NSNhiD 6650


Ixia XM12

CN 3106Ciena

Nortel MERS 8600

Nortel MERS 8600

RAD ACE-3200

NSNFlexi WCDMA

ms0

T5C-24FTelco Systems


Huawei OptiXOSN 1900



o 7606

Systemstro-200

CienaCN 5060

NSNRacel Emulator

MRVOptiSwitch OS304

TestCenterHuawei

Quidway CX300B

Metro Network Device

Gigabit Ethernet

Access Network Edge Device

Metro Network

Access Network

UNI Metro Edge Device

Tester

Device Functionality in the Test

Network Areas

Connection Types

RAN BS

User-Network

Network-Network

Core Network

Fast EthernetTDMoNxE1/STM-1ATMoNxE1/STM-1

Wireless/Microwave10 Gbit/s G.709

RAN NC

Core Device

Radio Transmission System

SHDSL

Interworking Function

STM-16EoPDH10Gbit Ethernet

NC Emulator

BS Emulator

(Base Station)

(NetworkController)

Interface (E-NNI)

Interface (UNI)

8


Mobile Backhaul Test ResultsWe started by testing the migration path options asmentioned in the introduction above. Circuit emulationcapabilities were tested in several different areas in theinteroperability test network:

Figure 3: Circuit Emulation Tests

A Spirent AX/4000 was used to offer bidirectional trafficin addition to providing a reference clock source. Theinterfaces involved were STM-1, ATM IMA, T1, and E1.

Circuit Emulation in AccessWe successfully verified Circuit Emulation Service (CES)in the access between MRV OptiSwitch OS910-M andTelco Systems T-Marc-254 devices by using the MetroEthernet Forum (MEF 8) specification to encapsulate TDMtraffic arriving at an E1 interface over the packetswitched network (PSN). The circuit used an MEF-definedE-Line transport service across the PBB-TE metro, theMPLS core and the MPLS metro networks.Emulated ATM services in the access were tested in twocombinations: Between Nortel Multiservice Switch15000 and RAD ACE-3200 and between Nortel Multi-service Switch 15000 and Nokia Siemens NetworksFlexi WCDMA BTS. Both CES services used ATMoMPLSas described in RFC 4717 to encapsulate the ATMcircuits. The circuits were transported over the PBB-TE

metro area. For the circuit emulation service betweenNortel and RAD, Nortel terminated the service via ATMSTM-1 interfaces, while RAD terminated it using ATMIMA n x E1 interfaces. On the E1 connection betweenRAD ACE-3200 and the TDM tester, the Eclipse (Gigabit)Radio from Harris Stratex provided wireless transport.The circuit emulation service between Nokia SiemensNetworks and Nortel showed the evolution of new basestations, as the Flexi WCDMA BTS (a UMTS base station)is equipped with a Fast Ethernet and Gigabit Ethernetuplink to the network, providing the interworking functioninternally. The base station was connected to the MRVOptiSwitch OS304 which acted as a cell site CPE andprovided connectivity towards the aggregation network.

In addition, RAD demonstrated1 CES for TDM E1 circuitsbetween two RAD ACE-3200 devices, and Alcatel-Lucentdemonstrated CES using the SAToP encapsulation (IETFRFC 4553) over MPLS for TDM T1 circuits on the Alcatel-Lucent 1850 TSS-5 devices in the T-MPLS area.

Circuit Emulation in MetroCircuit emulation in the metro refers to the scenario inwhich the base station is connected using traditionalTDM circuits to a router which is responsible for encapsu-lating the circuits and forwarding the traffic to the basestation controller.Huawei demonstrated TDM circuit emulation servicesbetween the Huawei OptiX OSN 1900 and OptiX OSN3900 in the T-MPLS metro area for E1 physical interface.ATM circuit emulation service in the MPLS metro areawas successfully tested between Huawei QuidwayCX300B and Ciena CN 5060. An E1 interface wasterminated on the Ciena router and an STM-1 interfaceon the Huawei side. Some other vendors met interopera-bility challenges with circuit emulation services. Oneservice could not be established since the two implemen-tations for signaling of circuit emulation services wereincompatible. The issue could have been identified bymapping protocol support mentioned in the two datasheets. The second interoperability issue encounteredwas between two systems terminating an ATM pseudow-ire using a targeted LDP session. We identified the issueas a "malformed TLV" error which was caused by apseudowire FEC mapping.Huawei also demonstrated ATM circuit emulationbetween their systems, the Quidway CX300B and theOptiX OSN 3900 as well as between the OptiX OSN3900 and OptiX OSN 1900. Both demonstrations wereperformed in the T-MPLS metro using STM-1 interfaces.

ATM Service TDM Service

Ethernet Based(MEF-8)

MPLS Based

MRV

HuaweiOptiX OSN

HuaweiOptiX OSN39001900

HuaweiQuidwayCX300B

CienaCN 5060

HuaweiQuidwayCX300B

HuaweiSmartAXMA5600T

NortelMSS 15000

RADACE-3200



Cisco7606

RADACE-3200

RADACE-3200

RADACE-3200

NortelMSS 15000

Nokia

Flexi

TelcoSystemsT-Marc-254

OptiSwitchOS910-M

SiemensNetworks

WCDMA BTS

1.Please note that we use the term »tested« when reporting onmulti-vendor interoperability tests. The term »demonstrated«refers to scenarios where a service or protocol was termi-nated by equipment from a single vendor on both ends.

9


Inter-Domain Circuit EmulationServicesAs was discussed above, one of the deployment scenar-ios involves a Generic Interworking Function (GIWF).These units are commonly installed by the mobile opera-tor at the cell site. The mobile operator then managesthese GIWF devices and hands over the emulatedcircuits to a packet-based metro network which aggre-gates the circuits towards the base station controller.ATM E1 and TDM E1 circuit emulation was testedbetween Cisco 7606 and RAD ACE-3200. The RADdevice was located in the access while the Cisco devicewas located in the metro aggregation. The two deviceswere connected using a microwave link provided byCeragon Networks FibeAir IP-MAX2 and used LDP-signalled MPLS pseudowires to provide the circuit emula-tion service. Cisco and RAD tested three types of circuitemulation over MPLS: SAToP (RFC 4553), ATMoMPLS(RFC 4717) and CESoPSN (RFC 5086).Huawei Technologies demonstrated ATM CES betweentheir devices SmartAX MA5600T acting as a cell siterouter, Quidway CX300B and OptiX OSN 3900, bothof which were positioned in the MPLS metro area. ATME1 CES was configured on the SmartAX MA 5600T sideand ATM STM-1 CES on the OptiX OSN 3900 side.Huawei Technologies also demonstrated TDM E1 CESbetween their OptiX OSN 3900 and Quidway CX200B.In all demonstrations Huawei used LDP for signalling ofMPLS CES pseudowires. The Huawei Quidway CX200Band SmartAX MA 5600T devices were located in theaccess, while the Huawei OptiX OSN 3900 andQuidway CX300B devices were located in metro. Thetransport was realized by MPLS tunnels, statically config-ured on the UNI and dynamically signalled over theMPLS metro network.

Clock SynchronizationThe base stations within a mobile operator’s domainmust operate using a common clock. This clock synchro-nization requirement is set according to the variousmobile technologies (e.g. GSM, CDMA, UMTS etc).A mobile backhaul network transporting a certain mobiletechnology must be able to fulfill the requirements associ-ated with said technology at the radio interface.When a time or frequency reference is carried across thenetwork, other requirements apply. These requirementsare not yet standardized. ITU-T G.8261 mentions in itsmost recent version (WD11, September 2007) a goal foraccuracy of the reference frequency of significantlybelow 50 ppb, if possible around 16 ppb. The value of16 ppb is derived from ITU-T G.812 Type II frequencyaccuracy.We measured the frequency accuracy at the network

egress and compared it to these requirements.At our interoperability event two solutions were tested foradaptive service clock synchronization. In general,adaptive clock synchronization regenerates the clockinformation from the data stream. Assuming that thetransmitter sends packets at a known rate and preciseintervals, the adaptive mode is usually accomplished onslaves by either measuring packets inter-arrival time ormonitoring a buffer fill level (some adaptive clock recov-ery mechanisms may also use timetamps). Adaptiveclock works only for constant bit rate services.One test was performed between MRV OS910-M andTelco Systems T-Marc-254. The clock synchronizationsolutions of MRV and Telco Systems are based on the linetiming method described in MEF3/MEF8, which is anadaptive clock recovery method. The slave recovered theservice clock based on the received E1 data rate, deriv-ing the master’s transmission rate.Another test was performed between Cisco 7606 andRAD ACE-3200. In this test scenario the Cisco 7606 wasconfigured to be the master clock while the RAD ACE-3200 recovered the clock from a CESoPSN MPLSpseudowire provisioned with a single DS0.The external reference clock method (as defined inMEF 8) was used in two tests: Between RAD ACE-3200and Nortel Multiservice Switch 15000, and betweenNokia Siemens Networks Flexi WCDMA BTS and NortelMultiservice Switch 15000. In this method, an externalreference clock was recovered from the legacy TDMnetwork. The Spirent AX/4000 supplied the externalreference clock source to the devices under test using alegacy E1 interface.Alcatel-Lucent 1850 TSS-5 demonstrated differentialservice clock recovery for circuit emulation services usingIEEE 1588 version 2 to distribute the reference clock.

Figure 4: Sample Jitter/Wander Measurement

Huawei demonstrated a clock synchronization methodbased on synchronous Ethernet using their SmartAX5600T and OptiX OSN 3900 and a combination of twosynchronization methods for a single circuit emulation:Proprietary adaptive clock with Synchronous Ethernet.The circuit emulation was configured between Huawei

10


devices Quidway CX200B and OptiX OSN 3900 asdescribed in the previous section. The transport was real-ized via Huawei OptiX OSN 1900. The clocks on theHuawei OptiX OSN 1900 and the OptiX OSN 3900were synchronized using Synchronous Ethernet while theclocks on the OptiX OSN 1900 and Quidway CX200were synchronized by proprietary adaptive solution.We measured wander according to the ITU-T G.823standard over a duration of 900 seconds between:Cisco 7606 and RAD ACE-3200; Huawei TechnologiesSmartAX 5600T and OptiX OSN 3900; MRV OS910-Mand Telco Systems T-Marc-254. During the wandermeasurements, we observed frequency deviation belowthe expected 16 ppb which meant that all participants inthe test could deliver the required frequency accuracy.

Mobile Application DemonstrationsTwo scenarios between real Radio Access Network basestations and Radio Access Network controllers weresuccessfully demonstrated. The first scenario demon-strated a packet based mobile backhaul for WCDMAtechnology between Nokia Siemens Networks devices, aFlexi WCDMA base station and a Racel network emula-tor. The transport service was provided by the PBB-TEmetro network by Nortel Metro Ethernet Routing Switch8600 and Nokia Siemens Networks hiD 6650. TheNortel Multiservice Switch 15000 configured an MPLSbased ATM CES in the access network for Racel Emula-tor. The ATM CES was terminated on the Flexi WCDMAbase station. The Racel device emulated a WCDMANetwork controller along with an entire mobile operatorcore network. The common clock source was providedby Spirent AX/4000. Nokia Siemens Networks wereable to demonstrate a mobile host receiving video over aradio connection.The second scenario demonstrated a packet basedmobile backhaul for GSM mobile technology betweenan Ericsson RBS2409 GSM base station and Ericssonmobile core emulation. The transport was realized in thePBB-TE metro network by Nortel Metro Ethernet RoutingSwitch 8600, Huawei Technologies Quidway CX600and CX380. In the access network the base station wasconnected over the RAD ETX-202 and Ceragon FibeAirIP-MAX2. The Ericsson base station translated the cellulardata using a proprietary CES and transmitted it to theRAD ETX-202 over Ethernet. At the other end, the TelcoSystems T-Marc-380 provided connectivity for an Erics-son system which used an IP/VPN tunnel over the Inter-net to the Ericsson mobile core emulation. The clocksynchronization between the base station and theemulated network controller was achieved using NTPv3.Ericsson was able to demonstrate real phone callsbetween mobile phones attached to the base station. Inaddition, this connection maintained a constant datastream for video.

Alcatel-Lucent 1850 TSS-3 demonstrated Ethernet overPDH (EoPDH) using the standardized ITU-T G.7041 GFPwith G.7043 for PDH virtual concatenation. EoPDH is astandard defining the transport of Ethernet frames overcopper infrastructure.

Metro Transport TestsWe tested application services (mostly Carrier Ethernetbased) across three different types of metro transportnetworks:

• MPLS – A set of protocols defined by the IETF and theIP/MPLS Forum; positioned to deliver layer 2 andlayer 3 services including Ethernet services asdefined by the MEF; fairly agnostic to the underlyingtransport technology.

• PBB-TE – Focuses on transport using Ethernet as thetransport medium; aiming to offer a similar connec-tion oriented model and resiliency capabilities asSONET/SDH networks; IEEE draft is in version 1.1and is making its way through the standards process.

• T-MPLS – ITU-T standardized networking layer forpacket transport networks, based on MPLS dataplane, designed for providing SONET/SDH-likeOAM and resiliency for packet transport networks.

We assigned all participating aggregation devices to thethree groups based on the transport technology of thevendors’ choice. The physical layer was realized using adiverse set of technologies dominated mostly by GigabitEthernet links, in addition to point-to-point microwavelinks, OTU2 (ITU-T G.709) and STM-16.The different metro/aggregation cloud tests aredescribed in detail below.

MPLS for AggregationThe tests in this area were based on previous experiencelargely gained from EANTC’s Carrier Ethernet WorldCongress and MPLS World Congress interoperability testevents. Many carriers are considering MPLS as a maturetechnology with well defined standards and a wideinstall base. Still, the question was, in just how far isMPLS ready to support Mobile Backhaul transport?The MPLS metro/aggregation cloud included equipmentfrom seven vendors and operated independently fromthe core network (where MPLS was used as well). MPLSrouters participating in this area included Alcatel-Lucent7450 ESS-1, Ciena CN 5060, Cisco 7606 and CRS-1,Huawei Technologies Quidway NE40E, OptiX OSN1900, OptiX OSN 3900 and Quidway CX300B, MRVOptiSwitch OS9024-M and Telco Systems T-Metro-200.The Eclipse Gigabit Radio from Harris Stratex providedan adaptive modulation microwave link between theAlcatel 7450 ESS-1 and Cisco CRS-1. The Cisco CRS-1,physically a core router, was put in the MPLS aggrega-

11


tion to provide more test opportunities and more connec-tions to other vendors' systems.A VPLS domain implementing multipoint-to-multipointtransport services was established between the followingrouters without any issues: Alcatel-Lucent 7450 ESS-1;Cisco 7606 and CRS-1; Huawei Technologies QuidwayNE40E. Ixia XM12 participated as a partially meshedPE in the MPLS metro by emulating a VPLS PE with a fullprotocol stack implementation, forming an adjacencywith one other PE. A few pseudowires between a VPLSPE device and MTU devices required longer time toconfigure than expected since the VPLS PE device did notsupport an H-VPLS spoke function initially. The problem,however, was quickly solved through a code downgradeand the spokes were established.The following devices participated as Multi-Tenant Units(MTUs) in an H-VPLS domain: Ciena CN 5060; Cisco7606; Huawei Technologies OptiX OSN 1900 andQuidway CX300B; MRV OptiSwitch OS9024-M andTelco Systems T-Metro-200.OSPF was used amongst most of the participants,however, a few devices supported only IS-IS. A routegateway was configured to link both IGP domains.Point-to-point transport services were realized usingVirtual Private Wire Services (VPWS). No problems wereencountered when activating the VPWS services.

Provider Backbone BridgeTraffic EngineeringWe had tested PBT during the Carrier Ethernet WorldCongress 2007 interoperability test event as a precursor,based on a draft version 0.0, to what is now referred toas PBB-TE. Since then the IEEE standard has movedforward and is now in draft version 1.1 (as of December21, 2007). Devices from five vendors participated in thetest of PBB-TE trunk establishment and data forwarding:Ciena CN 3106; Huawei Quidway CX380 andQuidway CX600; Lightstorm Brooklyn-10; NokiaSiemens Networks hiD 6650; and Nortel Metro EthernetRouting Switch 8600. Ixia XM12 emulated PBB-TE trunkend-points with a full protocol stack implementation.Spirent TestCenter maintained PBB-TE trunks where neces-sary through the emulation of Ethernet CFM. Betweentwo of Nortel’s Metro Ethernet Routing Switch 8600 amicrowave link was created using Ceragon FibeAir IP-MAX2 showing trunk level prioritization and protectionwith adaptive modulation.The PBB-TE standard draft recommends the use of anexternal control plane/management system to configuretrunks. Two such management systems participated inEANTC’s Carrier Ethernet World Congress interoperabil-ity testing in 2007 (please see EANTC’s web site for thereport). Since no PBB-TE management vendor partici-pated in the interoperability event, all PBB-TE trunks were

configured manually.PBB-TE is based on existing Ethernet standards, and asexpected data forwarding worked without any problems.The draft requires the use of Connectivity Fault Manage-ment (CFM) in order to establish the PBB-TE trunks. Notall vendor implementations supported CFM and so IxiaXM12 was used to generate CFM frames on behalf ofthese switches. Furthermore, one device could not trans-mit CCM messages on its PBB-TE trunks. The vendor wasable to install a code update and bring the trunks upwithin the course of testing.Multiple transport services within the PBB-TE metro areawere mapped into a single PBB-TE trunk using a unique I-SID and B-VID. Services meant to cross the MPLS corenetwork originating from the same device were mappedto a single PBB-TE trunk using a different I-SID. One PBB-TE trunk was configured between Ciena CN 3106 andNortel Metro Ethernet Routing Switch 8600, traversingthe Alcatel-Lucent 7450 ESS-7 and Huawei TechnologiesQuidway NE5000 in the core network.

Figure 5: Logical Provider Backbone BridgesTraffic Engineering Trunks

Transport MPLS (T-MPLS)T-MPLS was tested for the second time at an EANTCinteroperability event; several new vendors and devicesparticipated in the test this time. T-MPLS is a transporttechnology aiming to deliver a way for carriers to virtual-ize a wire and offer the same predictive paths as carriersare used to in the SONET/SDH world. T-MPLS is definedin several ITU-T draft documents either approved orunder approval (see references section). T-MPLS paths(TMP) are end-to-end tunnels which aggregate T-MPLSchannels (TMC) representing the services. In addition,the T-MPLS protocol family defines OAM and protectionmechanisms.The T-MPLS reference standard does not define a specificcontrol plane, but the future use of the GMPLS controlplane is not precluded (e.g. RSVP-TE for signaling; OSPF-

Huawei

PBB-TE TrunkTester

IXIA

Spirent

Huawei

Metro network provider edge

Quidway

TestCenter

Quidway

NSN hiD6650

CX380

CX600 XM12Ciena

CN 3106


NortelMERS 8600

12


TE and ISIS-TE for routing). SInce no control plane wasused during the testing, the T-MPLS connections betweendifferent vendors were manually configured. The connec-tions between the different Alcatel-Lucent devices in the T-MPLS area were configured using a proprietary Alcatel-Lucent 1350 Management System.

Figure 6: Logical T-MPLS Channels (TMCs)

During the configuration of TMCs and TMPs we discov-ered an interoperability issue between two of the partici-pants. One vendor required the TMP labels and TMClabels to be allocated from two different label poolswhile another vendor had the opposite configurationlimitations: If a TMC label came from a certain label poolits TMPs also had to come from the same label pool. Theproblem was solved by simply reconfiguring the limits ofthe label pools on the first vendor’s equipment.The following devices were part of the T-MPLS metrocloud and have successfully tested and demonstrated theT-MPLS Path (TMP) and T-MPLS Channel (TMC) establish-ment, data forwarding over T-MPLS and T-MPLS Multi-plexing in which a number of TMCs were transportedwithin a TMP:Alcatel-Lucent 1850 TSS-40, 1850 TSS-320; Ciena CN5060; Ericsson-Marconi OMS 2400; Huawei OptiXOSN 3900 and OSN 1900; Ixia XM12; LightstormBrooklyn-10. In addition, Harris Stratex Eclipse (Gigabit)Radio provided the UNI connection between an

emulated base station and Huawei’s OSN 1900.An addition to the devices’ capabilities in the T-MPLSmetro area was bridging instance support, which couldbe used to map a TMC or a VLAN to interfaces, andfunctionality similar to H-VPLS spoke connection. Thisallowed the T-MPLS vendors to offer a multipoint-to-multi-point service much like MPLS’ Virtual Private LANServices (VPLS) in MPLS networks. MAC learning wasdone using flooding and dynamic address learning orusing MAC address static configuration. However, unlikeVPLS in MPLS networks, no signalling was used. At themoment no standards exist for native multipoint to multi-point services in T-MPLS networks.

MPLS CoreConnectivity between the three different transport/metroareas was facilitated using MPLS. The core area wasconstructed using an Alcatel-Lucent 7450 ESS-7, Cisco7604 and Huawei Quidway NE5000E. The coreprovided transport both point-to-point and multipoint-to-multipoint Ethernet services in addition to establishing anIP/MPLS L3VPN service between Huawei and Cisco(based on RFC 4364). Point-to-point services were imple-mented with Ethernet pseudowires while multipoint-to-multipoint services were implemented using VPLS withLDP signaling between the three vendors.

Connecting AdministrativeDomainsPoint-to-point services over the MPLS core were realizedusing MPLS pseudowires. The services were mapped intopseudowires based on Provider Bridges tags as theywere leaving the metro areas. Each metro area termi-nated the point-to-point service using an S-VLAN (A vlansignificant only to the service provider) and handed theframes, now with two VLAN tags, to the core. The coreused the S-VLAN tags as service delimiters and thenforwarded the frames to their destination metro area.

IXIA

Ericsson MarconiAlcatel-Lucent

Ciena

Lightstorm

Huawei

Huawei

Alcatel-Lucent

OMS 2400

CN 5060

XM12

OptiXOSN 3900

1850 TSS-320

Brooklyn-10

1850 TSS-40

OptiXOSN 1900

T-MPLS TMCTester


“Spoke“ H-VPLS pseudowire

VPLS connection

VPLS provider edge

H-VPLS Multi-Tenant Unit

Cisco7606

Alcatel-Lucent7450 ESS-1

HuaweiQuidway NE40E

CiscoCRS-1



CienaCN 5060


Cisco7606

LDP VPLS(full-mesh connected)

Figure 7: Logical H-VPLS Metro Connections

13


In one case a service was transported across the T-MPLSand MPLS network boundary without Ethernet bridging.A TMC was statically configured in the T-MPLS areabetween two Alcatel-Lucent 1850 TSS-40 devices and anMPLS pseudowire was dynamically signaled betweenAlcatel-Lucent 1850 TSS-40 and Cisco 7604. The TMCto pseudowire interworking (stitching) between the staticand dynamic segments was performed by the Alcatel-Lucent 1850 TSS-40.One point-to-point Ethernet service was implementedacross two metro areas using a TMC between HuaweiOptiX OSN 3900 and Alcatel-Lucent 1850 TSS-320,and handing off to the Ciena CN 5060 in the MPLSmetro via a static MPLS label. Unfortunately none of thecore vendors found time to test the T-MPLS/MPLS inter-working on any of the Inter-AS interfaces based on thestatic label configuration.For multipoint-to-multipoint services, three VPLS instanceswere configured: One in the T-MPLS metro, the MPLScore and the MPLS metro networks each. Two VPLSspoke connections were configured in the core towardsthe T-MPLS network. One used a dynamically signalledMPLS pseudowire, which was configured betweenAlcatel-Lucent 1850 TSS-40 and Cisco 7604. Thesecond connection was performed by the Alcatel-Lucent1850 TSS-40 performing the interworking betweenMPLS and T-MPLS.Cisco demonstrated inter-provider L3VPN connectivitybased on RFC 4364 option C. The connection was madebetween the Cisco CRS-1 positioned in the MPLS aggre-gation area and the Cisco 7604. Several solutions existto enable an L3VPN to cross different administrativedomains. In particular, the IETF defined three optionswhere MPLS/BGP VPNs can be supported across an ASboundary using readily available protocols. To customersthe three options is transparent. To service providers,each option requires a different level of trust relationshipat the boundary. Option C is the most integrated methodin which the two provider networks must function as one,forming one single large end-to-end VPN domain. Eachoption has its pros and cons, as well as its value proposi-tion. All three options were tested.

Ethernet OAM TestsWe tested two Ethernet Operation, Administration andMaintenance (OAM) standards in previous Carrier Ether-net World Congress Interoperability events:

• IEEE 802.1ag – Connectivity Fault Management(CFM)

• IEEE 802.3ah – Ethernet in the First Mile (EFM),specifically section 57

For this event we added the ITU-T standard Y.1731“OAM functions and mechanisms for Ethernet basednetworks” to the agenda.

Native Ethernet fault management protocols are essentialfor the delivery of high quality end-to-end services tobusiness customers, mobile base stations and increas-ingly also consumers (for example for IPTV or VoIP). Theservice availability is key for many customer applica-tions; it can be managed using Ethernet OAM protocols.The tests described below are a subset of the rich set ofOAM tools available today for Ethernet.

Ethernet Link OAMThe Ethernet in the First Mile IEEE standard extends theEthernet Media Access Control (MAC) layer to additionalphysical layers such as voice grade copper cable. Aspart of the standard, a set of OAM tools are definedaiming to help carriers monitor link operations. Wetested the following three functions:

• Link OAM Discovery – Detecting the existence andconfiguration of the OAM sublayer in the neighbor-ing Ethernet equipment

• Link OAM Loopback – Configuring a loopback modefor fault localization and link performance testing

• Link OAM Dying Gasp – Notifying the neighborabout an imminent system shutdown

The following vendors participated successfully in theOAM discovery test: Alcatel-Lucent 7450 ESS-1 and7450 ESS-7; Ciena CN 3106 and CN 5060; Cisco7604 and 7606; Huawei Quidway CX380 andQuidway CX600; MRV OptiSwitch OS9024-M; RADETX-202; Spirent TestCenter; Telco Systems T5C-XG, T-Marc-250 and T-Marc-380. Spirent TestCenter facilitatedthe testing with a full stack implementation of 802.3ah.No problems were discovered during the initial discov-ery phase of the testing.

Figure 8: Ethernet Link OAM Tests


Access network device

Metro UNI provider edge

Cisco

Alcatel-Lucent Core

Telco Systems

Spirent

Cisco7604

7450 ESS-7TestCenter

7604 TelcoSystems

T-Marc-380Ciena

CN 5060Cisco7606

RADETX-202Spirent

TestCenter

Aggregation

T-Marc-250

OAM DiscoveryLoopback Mode Dying Gasp Messages

Tester

AccessTelco Systems

T5C-XG

CienaCN 3106

HuaweiQuidwayCX380

MRVOptiSwitchOS9024-M

14


12 device pairs participated in the EFM loopback test-ing. Loopback testing varied between vendors based onthe modes they supported. Passive devices supportedentering and exiting loopback state while Active devicescommanded Passive devices to enter Loopback. The testswere verified in one of two ways depending on thedevices involved in the testing. When two devices wereinvolved the roles were agreed between the vendors.When only one device was involved, Spirent TestCenteremulated either passive or active role.Some interoperability issues were discovered during theLink OAM Loopback testing. In one case a vendor wassending unexpected PDU types which caused its peer todrop the PDUs. In another case we discovered that avendor was not incrementing the TLV revisions andanother vendor could only operate in a Passive mode.OAM interoperability testing is specifically important: Itis quite likely that the customer premises equipment (CPE)and the switches terminating the metro/aggregationcloud will be supplied by different vendors in realnetworks. Service providers need to rely on remote faultlocalization and performance verification in order tokeep SLAs and offer a profitable service. While it is posi-tive to see a growing number of Ethernet OAM imple-mentations, a lot of work still needs to be done in testingcompliance and interoperability.The Dying Gasp test was performed between eight pairsof devices. Participating devices’ roles were asymmetric:The customer-side device generated the dying gasp whilethe network-side device was responsible for receivingand processing the message.This test was successful between almost all pairs. Oneinteroperability issue was found between two devices inwhich a vendor was sending Dying Gasp message in anOrganization Specific (OAMPDU code 0xFE) PDU typewhile the receiving vendor was expecting the PDU in theInformation type (0x00). As a result, the vendor receivingthe Dying Gasp did not detect the message.

End-to-End Ethernet Service OAMIn contrast to link OAM, Connectivity Fault Management(as defined in IEEE 802.1ag) allows up to eight layers ofend-to-end service monitoring through different manage-ment domains and can be used across logical connec-tions. This standard is referred to as Service OAM inMEF 17.The major use of CFM for service providers and enter-prises is to verify connectivity across different manage-ment domains. A service provider can define a manage-ment domain level to be used internally while allowingtheir customers to verify end-to-end connectivity over thenetwork using different management domains.At EANTC, we already tested service OAM – specificallythe Continuity Check functionality – at Carrier Ethernet

Figure 9: Ethernet Service OAM Tests

World Congress 2006 and 2007. Each year we receiverequests from vendors to test this set of standards as thevendors recognize the standard as important to carriers.Although the focus of this event was to include mobilebackhaul interoperability testing, we also acknowledgedthe vendor requests to expand on Ethernet OAM tests.In the event, six vendors with a total of 11 devices partic-ipated in Ethernet OAM testing: Ciena CN 3106;Huawei OptiX OSN 1900, OptiX OSN 3900, QuidwayCX380, Quidway CX600 and SmartAX MA5600T; MRVOptiSwitch OS910-M; Nortel Metro Ethernet RoutingSwitch 8600; RAD ETX-202; Telco Systems T-Marc-254and T-Marc-380. Both test vendors Ixia and Spirentparticipated in the CFM testing by generating test trafficand having a full stack protocol implementation. In addi-tion to the Continuity Check Messages, we also testedLink Trace and Loopback functionality. Spirent TestCenterdemonstrated simulation of multi-hop CFM link tracethrough emulated intermediate and end point mainte-nance entities.As opposed to previous years we encountered almost noissues during the testing. While we had been plagued byincompatible Ethertype values, mismatches betweenMAC address and hard-wired CCM intervals before, wesailed through the testing nicely this time. All participantswere able to configure a common Ethernet type value of0x8902 and Maintenance Domain (MD) levels 2 or 7.One vendor required a code patch, but that wasprovided quickly and the vendor was able to join the test-ing. We were pleased to see that careful planning andthe experience and results from previous interoperabilityevents brought such high level of success to the test.

RAD


IXIA XM12

Metro network

LoopbackAccess network device

LinktraceContinuity check

Metro UNI provider edge

Tester

provider edge

ETX-202

Huawei

Nortel MERS8600

Huawei


SpirentTestCenter

CienaCN 3106

MRV OptiSwitch QuidwayCX600OS910-M

SmartAXMA5600T


15


ResiliencyA high degree of resiliency is expected from a packetswitched network to support mobile traffic. Mobile oper-ators are accustomed to the resiliency offered by TDMnetworks. Voice traffic is sensitive to loss which is theobvious result of a failure in the network. In order todemonstrate Ethernet and MPLS readiness to supportmobile traffic, several resilience mechanisms wereconfigured and tested which are detailed below. Theresults are reported as the duration of service interruption(called failover time) and are calculated based on thenumber of frames lost. Restoration time (time to restoreservice after a failure) is reported in the same way.

Ethernet Link AggregationAs in previous events, vendors expressed interest inshowing native Ethernet resiliency mechanism. LinkAggregation (IEEE 802.3ad) provides such a mechanismby grouping several Ethernet interfaces into a Link Aggre-gation Group (LAG) which is then presented to thenetwork device as a logical interface. The interface canbe used as long as at least one of its members is stilloperating.Link Aggregation Control Protocol was tested betweenAlcatel-Lucent 7450 ESS-7; Ciena CN 5060 and CN3106; Cisco 7604; Ericsson Marconi OMS 2400;Huawei SmartAX MA5600T, Quidway NE40E,Quidway NE5000E; Nortel Multiservice Switch 15000;Telco Systems T5C-XG. All participants brought up theLAG groups without issues.

Figure 10: Ethernet Link Aggregation Tests

Dual Homed Access. Telco Systems demonstrated theability of their T5C-XG to redundantly connect to twometro aggregation devices using a backup Ethernet

connection. The primary connection was brought upagainst the Ericsson Marconi OMS 2400 and thebackup connection was configured against a secondEricsson Marconi OMS 2400 device. When the primaryEthernet interface detected a loss of signal, the T5C-XGwas able to switch to the backup path using the sameVLAN ID.

Resiliency Through MPLSSeveral resilience mechanisms exist in MPLS and mosthave been tested in previous EANTC MPLS WorldCongress interoperability events. MPLS Fast Reroute (FRR)is one such mechanism that was tested in 2006 andshowed under 50 ms rerouting times (the white paper isavailable on EANTC’s web site). MPLS Fast Rerouteprovides link and node protection for Label SwitchedPaths (LSPs).

Dual Homed Multi-Tenant Units (MTUs). Dualhomed MTUs are discussed in two specifications relatedto multipoint MPLS services, RFC 4761 and RFC 4762.Since the multipoint service protocol H-VPLS is a realiza-tion of a hub and spoke design with the MTU acting as aspoke and the PE router as a hub, the standards authorssaw a need to address an inherent drawback of hub andspoke – the spoke could easily be separated from thehub and, therefore, lose connectivity. In order to protectthe MTU from such a disaster, an additional backuppseudowire can be defined to a separate provider edge(PE) router. If the primary pseudowire fails, failover to thesecondary pseudowire is facilitated by use of nativeMPLS protocols or other mechanisms.MRV OptiSwitch OS910-M and Telco Systems T-Metro-200 tested Dual-Homing MTUs against Alcatel-Lucent7450 ESS-1, Cisco 7606 and Huawei Quidway NE40Ethe latter three serving as VPLS PEs for a redundant H-VPLS architecture. The failover times measured wereunder 10 ms.

MPLS Fast Reroute. The core vendors – Alcatel-Lucent, Cisco and Huawei tested MPLS based resiliencyusing Fast Reroute and in one case using Ethernet LinkAggregation Groups (according to IEEE 802.3ad). Twotest scanarios were configured, one with Huawei actingas the mid-point and Alcatel-Lucent and Cisco as point oflocal repair/merge points. In the second test scenario theCisco was configured as a mid-point and Alcatel-Lucentand Huawei as point of local repair/merge points.MPLS-FRR showed yet again its carrier grade naturemeasuring, through several iterations of testing, at most85 ms out of service time during link failure and at best 1ms. During the restoration phase of the tests no frameloss was recorded. Ixia XM12 and Spirent TestCenterwere used to measure the Fast Reroute convergence time.Huawei demonstrated Fast Reroute between their OptiXOSN 3900 and Quidway NE40E systems, with the

Nortel Ciena

Huawei Huawei

Alcatel-Lucent

Ciena

Telco Systems Ericsson

Cisco

LACP LAG (Static)

CN 3106

QuidwayNE40EHuaweiQuidwayNE40E

MarconiOMS 2400

7450 ESS-7

CN 5060

MSS 15000

SmartAXMA5600T

7604

T5C-XG

EricssonMarconi

OMS 2400

Telco SystemsT5C-XG

16


Huawei OSN 1900 acting as an intermediate node —recording a failover time of 63 ms and a restoration timeof 7 ms. Telco Systems demonstrated Fast Reroutebetween two T-Metro-200 devices with an Alcatel-Lucent7450 ESS-7 acting as an intermediate node. The out ofservice time recorded at this test was 11 ms duringfailover and 30 ms during restoration.

PBB-TE ProtectionResiliency within the PBB-TE domain would have beendone by defining two PBB-TE trunks (»working path« and»protection path«) to the same destination. As the PBB-TEingress device detects that the working path has failed, itswitches traffic over to the protected path. The failuredetection is done by CFM (CFM is described in the end-to-end Ethernet Service OAM section).Out of the five vendors participating in the PBB-TE cloudtwo vendors did not support the generation of CFMContinuity Check Messages which are required forfailure detection. Another vendor did not supportprimary and secondary B-VIDS which were required forthe testing. Huawei demonstrated PBB-TE protectionbetween the Huawei Quidway CX380 and the QuidwayCX600.No further resiliency testing was performed during thisevent, however, during EANTC’s interoperability event atCarrier Ethernet World Congress 2007, PBT resiliencyhad been verified. The CEWC 2007 report is availableon EANTC’s web site.

T-MPLS ProtectionSeveral resiliency tests, using 3.3 ms connectivity verifi-cation intervals, were performed in the T-MPLS metrocloud in accordance to the ITU-T G.8131 specification.

T-MPLS path (TMP) protection. Two path protec-tion schemes exist for T-MPLS. 1:1 protection requirestwo paths to be built to the same destination. One path isthen declared as active and the other is set to backupmode. When the active path fails, traffic is switched overto the backup path. The second scheme, 1+1 protection,replicates all frames across both active and backuppaths such that when the primary path fails, the backuppath is already carrying the identical frames.Alcatel-Lucent 1850 TSS-320 and Huawei OptiX OSN3900 tested T-MPLS 1:1 protection using EricssonMarconi OMS 2400 as an intermediate node. 1+1 T-MPLS protection was tested between Huawei Technolo-gies OptiX OSN 1900 and Ericsson Marconi OMS2400. The same pair of devices tested 1:1 T-MPLSprotection with APS signaling. Another 1:1 protectiontest was performed between Alcatel-Lucent 1850 TSS-320 and Ericsson Marconi OMS 2400. Alcatel-Lucentused their management system to configure the protec-

tion tests in which they participated. The TMP protection1+1 and 1:1 with APS signaling tests included both uni-directional and bi-directional failover scenarios. The TMPprotection 1:1 (without APS signaling) tests only includedbi-directional failure scenarios. Failover times rangedbetween 0.8 ms and 40 ms.

T-MPLS Channel (TMC) protection. Alcatel-Lucentdemonstrated TMC protection with three 1850 TSS-320devices. The working TMC was mapped over a TMPpassing through Ericsson Marconi OMS 2400. A multi-hop TMC was used for the protection channel. When abidirectional failure was simulated on the working TMC,the traffic was switched on the protecting TMC. TheAlcatel-Lucent devices were set to switch traffic back to itsprimary TMC once the TMC was brought back into oper-ation.

Quality of Service SupportQuality of Service describes the ability of the network tosupport differential treatment to network traffic based onsome identification in the packets, frames or cells. Thisidentification allows network devices such as routers andswitches to prioritize one class of traffic over another. Inthe mobile backhaul case this is an important require-ment for several reasons:

• Voice traffic is highly sensitive to loss and delay andshould be guaranteed by the network

• Data services carry a premium price tag in themobile world as compared to fixed line data services

• Communication between the base station and thecontroller is imperative to the healthy operations ofthe radio network

Under optimal network conditions where no resourcesare oversubscribed little can be gained from classifyingtraffic into different classes. However, when networkresources are starved and oversubscription occurs, thenetwork must prioritize traffic according to the operator’sService Level Agreements (SLA) and the type of serviceoffered over the network.Support for different QoS levels was verified in all fourtransport network areas. We defined three differentclasses of service for three different services offered bythe network:

• Mobile Backhaul – the characteristics of this serviceclass were low delay and jitter with relatively smallbandwidth requirements

• Video Service – this class consisted of relativelyrelaxed delay and jitter requirements, but a steady(guaranteed) amount of bandwidth

• Data Backup Service – The VPN for a data backupservice required a large amount of bandwidth thatcould vary over time. No specific requirements forjitter and delay were given in this class.

17


The three classes were configured across all three trans-port areas; MPLS, PBB-TE and T-MPLS metro networksand in the MPLS core network. Between all networks andthe MPLS core, a mapping of CoS bits to MPLS EXP bitswas performed in order to facilitate end-to-end QoS. TheCoS bits in Ethernet headers, which are limited to eightclasses of service, were directly mapped to EXP bits ofthe same values.In the T-MPLS cloud, Alcatel-Lucent 1850 TSS-320 andEricsson Marconi OMS 2400 showed the expectedbehavior during the oversubscription situation. ThreeTMCs were defined to match the three classes of servicedefined above. Then, the data backup service was usedto oversubscribe one of the interfaces. We expected noloss from the Mobile Backhaul marked traffic and indeedrecorded none. On the other hand, as expected, thedata backup service was showing frame loss, allowingthe video and mobile classes to continue operatingwithout any adverse effect.In the PBB-TE cloud, a similar test to the one describedabove was successfully performed between NokiaSiemens Networks hiD 6650 and Nortel Metro EthernetRouting Switch 8600. Two traffic classes were definedbetween the two devices (one as Data Backup Serviceand the second Mobile Backhaul). The test showed thatwhen a data backup service traffic was used to congestthe link, the Mobile Backhaul traffic was correctly priori-tized and did not drop any frames.

Performance MonitoringThe MEF standard 10.1 defines attributes of EthernetServices observable at a User Network Interface (UNI).The ITU-T standard “OAM functions and mechanisms forEthernet based networks” (Y.1731) defines mechanismsfor two Ethernet stations to verify a certain level ofservice performance such as Frame Delay, Frame Delayvariation and Frame Loss Ratio.The following devices participated successfully in thetests: Alcatel-Lucent 1850 TSS-5, Ceragon FibeAir IP-MAX2, Ciena CN 3106, Ericsson Marconi OMS 2400,Ixia XM12, Lightstorm Brooklyn-10, MRV OptiSwitchOS910-M, Nokia Siemens Networks hiD 6650 andNortel Metro Ethernet Routing Switch 8600. The testingwas facilitated by Ixia XM12 and Spirent TestCenter.All devices tested and measured Frame Delay, FrameDelay Variation and Frame Loss Service Performances. Inone case a device responded with Delay MeasurementReplies (DRM), but did not properly populate the transmit-ted and received timestamps required for time basedmeasurements. A solution was not found during theexecution of the tests.

Bandwidth ProfilesA Bandwidth Profile is a characterization of the lengthsand arrival times for ingress Service Frames at a refer-ence point. In the test network, the reference point wasthe UNI. When a Bandwidth Profile is applied to a givensequence of ingress Service Frames, each Service Framein the sequence is declared to be compliant or notcompliant with the Bandwidth Profile.The Bandwidth Profile defines the set of traffic parame-ters applicable to a sequence of Service Frames. Associ-ated with the Bandwidth Profile is a rate enforcementalgorithm to determine Service Frame compliance withthe specified parameters. Rate enforcement also includesthe disposition of non-compliant Service Frames by eitherdropping or marking them. Bandwidth Profiles could beapplied at the UNI, an Ethernet Virtual Channel (EVC)and a Class of Service (CoS).Five vendor combinations were tested for both Band-width Profile per EVC and Bandwidth Profile per CoS:Lightstorm Brooklyn-10 and Ciena CN 3106; LightstormBrooklyn-10 and Ericsson Marconi OMS 2400; TelcoSystems T-Metro-200 and Cisco CRS-1; Telco Systems T-Metro-200 and Alcatel-Lucent 7450 ESS-7; Ixia XM12and Ciena CN 5060.In the Bandwidth Profile per EVC scenario, the devicesunder test configured two EVCs each with 5 Mbit/sCommitted Information Rate (CIR) and 2 Mbit/s ExcessInformation Rate (EIR), so that the total allowed rate was7 Mbit/s. A bidirectional traffic flow was generated forthese EVCs using both analysis devices Ixia XM12 andSpirent TestCenter. When the 5 Mbit/s stream wasgenerated, our expectations were met not to see anyloss. When the second stream was generated, with morethan 7 Mbit/s of traffic we expected some frame lossand in almost all test cases recorded loss. In one testcase we recorded 0.5 Mbit/s more traffic in one direc-tion than was allowed. Obviously, such level of inaccu-racy in Bandwidth Profile enforcement would be prob-lematic to a carrier network utilization plan.In the Bandwidth Profile per CoS the devices under testconfigured three different class of services: Voice, videoand best effort. The voice service was configured to128 kbit/s Committed Information Rate and 0 kbit/sExcess Information Rate. The video was configured to5 Mbit/s CIR and 1 Mbit/s EIR, so that the total allowedbandwidth for this service was 6 Mbit/s, and the besteffort class was set to 1 Mbit/s CIR and 6 Mbit/s EIR.A bidirectional traffic stream was generated for twodifferent EVCs with different CoS markers. In most casesthe results met our expectations and Bandwidth Profileswere correctly enforced. In one case an unexpectedamount of traffic was dropped in one direction.

18


End-to-End ServicesTwo types of end-to-end services were implemented inthe test network, spanning all metro clouds and the core.The services were constructed according to the MEFEthernet Service Definitions (MEF 6).

• Point-to-point Ethernet Virtual Private Line services(EVPL) were defined from UNI to UNI. EVPL servicesare the ones expected to be used to support MobileBackhaul services initially, as they allow the CarrierEthernet network operator to use UNIs for more thanone Ethernet Virtual Channel (EVC). The goal of deliv-ering identical service frames at the destination UNIas were seen in the source UNI was verified using theIxia XM12 and Spirent TestCenter.

• The second end-to-end type of service configuredwas Ethernet multipoint-to-multipoint (E-LAN).

In both cases provisioning the services across the diverseclouds was extremely time consuming. Engineers wererequired to manually configure the services on the UNIsand provide mapping configuration from the UNIs to thetransport (regardless of the transport technologies).MRV demonstrated E-Tree service according to an MEFdraft version of Ethernet Service Definitions phase 2. Thedemonstration conducted with Ixia XM12 showed thatMRV OS910-M, acting as a root in an E-Tree service,was able to completely separate downstream ports (userfacing) from directly exchanging traffic with each other.

AcknowledgmentsWe would like to thank Thomas Kessler from T-Systemsand Kyle Price from the University of New HampshireInteroperability Lab (UNH-IOL) for their extensive supportduring the hot-staging event.Telco Systems provided two T5C-48T switches to facili-tate the joint network management subnet between allvendors. The T5Cs provided Ethernet service across alllocations during the hot-staging and the live showcases.

Editors. This document has been edited by JambiGanbar, Sergej Kaelberer, Jonathan Morin and CarstenRossenhoevel.

Acronyms

Term Definition

B-MAC Backbone MAC

B-Tag Backbone Tag

BSC Base Station Controller

BNC Broadband Network Connector

CBS Committed Burst Size

CE Customer Edge

CE-VLAN Customer Edge Virtual LAN

CFM Connectivity Fault Management

CIR Committed Information Rate

CM Color Mode

CoS Class of Service

DTE Data Terminal Equipment

EBS Excess Burst Size

EEPP End to End Path Protection

EIR Excess Information Rate

E-LAN A multipoint-to-multipoint Ethernetservice. A LAN extended over a widearea

E-Line Point-to-Point Ethernet Service similar toa leased line ATM PVC or Frame RelayDLCI

EVC Ethernet Virtual Connection

FCS Frame check sequence (4 bytes at theend of Ethernet frame)

LAG Link aggregation Group

LSP Label Switched Path

MPLS MultiProtocol Label Switching

MTU Multi Tenant Unit

MTU Maximum Transmission Unit

MetroNPE

Metro Network Provider Edge device

OSPF Open Shortest Path First

PBB Provider Backbone Bridge

PBB-TE Provider Backbone Bridge Traffic Engi-neering

PE Provider Edge

QoS Quality of Service

RNC Radio Network Controller

RSVP-TE Resource reSerVation Protocol TrafficEngineering

SLS Service Level Specifications

STP Spanning Tree Protocol

TMP T-MPLS Path

TMC T-MPLS Channel

T-MPLS Transport MPLS

UNI User to Network Interface

MetroUPE

Metro UNI Provider Edge device

VPLS Virtual Private LAN Service

VPWS Virtual Private Wired Service

Term Definition

19


References"Requirements and Framework for Ethernet Service Protection in Metro Ethernet Networks", MEF 2"Circuit Emulation Service Definitions, Framework and Requirements in Metro Ethernet Networks", MEF 3"Metro Ethernet Network Architecture Framework - Part 1: Generic Framework", MEF 4"Ethernet Service Definitions - Phase 1", MEF 6"Implementation Agreement for the Emulation of PDH Circuits over Metro Ethernet Networks", MEF 8"Ethernet Services Attributes Phase 2", MEF 10.1"User Network Interface (UNI) Requirements and Framework", MEF 11"User Network Interface (UNI) Type 1 Implementation Agreement", MEF 13"Abstract Test Suite for Traffic Management Phase 1", MEF 14“Service OAM Requirements and Framework – Phase 1”, MEF 17"Media Access Control (MAC) Bridges", IEEE 802.1D"Media Access Control (MAC)Bridges Amendment 2: Rapid Reconfiguration", IEEE 802.1W"Virtual Bridged Local Area Networks", IEEE 802.1Q"Link Aggregation Control Protocol", IEEE 802.3ad"Carrier Sense Multiple Access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications.Amendment: Media Access Control Parameters, Physical Layers, and Management", IEEE Std 802.3ah-2004"Carrier Sense Multiple Access with Collision Detection (CSMA/CD) Access Method & Physical Layer SpecificationsAggregation of Multiple Link Segments", IEEE 802.3-2005"Virtual Bridged Local Area Networks - Connectivity Fault Management", IEEE 802.1ag/D8.1"Provider Backbone Bridges", IEEE 802.1ah"Provider Backbone Bridges Traffic Engineering", IEEE 802.1Qay/D.1.1"Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control Systems", IEEE 1588"Network Time Protocol (Version 3) Specification, Implementation and Analysis", RFC 1305"Multi-Protocol Label Switching (MPLS) Support of Differentiated Services", RFC 3270"RTP: A Transport Protocol for Real-Time Applications", RFC 3550"Traffic Engineering (TE) Extensions to OSPF Version 2", RFC 3630"Fast Reroute Extensions to RSVP-TE for LSP Tunnels", RFC 4090"Protocol extensions for support of Diffserv-aware MPLS Traffic Engineering", RFC 4124"BGP/MPLS IP Virtual Private Networks (VPNs)", RFC 4364"Pseudowire Setup and Maintenance Using the Label Distribution Protocol (LDP)", RFC 4447"Structure-Agnostic Time Division Multiplexing (TDM) over Packet (SAToP)", RFC 4553"Encapsulation Methods for Transport of Asynchronous Transfer Mode (ATM) over MPLS Networks", RFC 4717“Structure-Aware Time Division Multiplexed (TDM) Circuit Emulation Service over Packet Switched Network(CESoPSN)”, RFC 5086"Circuit Emulation Service Interoperability Specification Version 2.0", ATM Forum af-vtoa-0078.000, January 1997"Framework for MPLS in Mobile Backhaul Networks", IP/MPLS Forum 2007.102.03"Architecture of Transport MPLS (T-MPLS) Layer Network", G.8110.1"Interfaces for the Transport MPLS (T-MPLS) Hierarchy", G.8112"Linear protection switching for Transport MPLS (T-MPLS) networks", G.8131"Management aspects of the T-MPLS network element", G.8151"Timing and synchronization aspects in packet networks", G.8261/Y.1361"Timing characteristics of synchronous ethernet equipment slave clock (EEC)", G.8262"OAM functions and mechanisms for Ethernet based networks", Y.1731"Characteristics of Transport MPLS (T-MPLS) equipment functional blocks", G.8121

19

EANTC AGEuropean AdvancedNetworking Test Center

Upperside IP/MPLS Forum MEFMetro Ethernet Forum

Einsteinufer 1710587 Berlin, GermanyTel: +49 30 3180595-0Fax: +49 30 [email protected]

54 Rue Du Fbg St. Antonie75012 Paris, FranceTel: +33 1 53 46 63 80Fax: +33 1 53 46 63 85www.upperside.fr

48377 Fremont Blvd., Suite117Fremont, CA 94538, USATel: +1 510 492 [email protected]

9841 Airport Blvd, Suite1200, Los Angeles, CA90045, USATel: +1 310-258-8032info@metroethernetforum.orgwww.metroethernetforum.org

This report is copyright © 2008 EANTC AG. While every reasonable effort has been made to ensure accuracy andcompleteness of this publication, the authors assume no responsibility for the use of any information contained herein.All brand names and logos mentioned here are registered trademarks of their respective companies in the United Statesand other countries.

METRO ETHERNET FORUM

Documents

Public Multi-Vendor Mobile Backhaul Interoperability Test ......Nokia Siemens Networks hiD 6650 Flexi WCDMA BTS RNC and Core Network Emulator (Racel) Nortel Metro Ethernet Routing