IEEE Communications Magazine • November 2013 1050163-6804/13/$25.00 © 2013 IEEE

INTRODUCTION

With the explosion of handheld devices andwired networks, the compounded average growthrate (CAGR) of bandwidth is almost 63 percent[1]. Further analysis of the CAGR concludesthat the increase in the number of users is notproportional to the increase in bandwidth con-sumption — implying that users are indulging inmore bandwidth-heavy applications as well asusing a larger number of bandwidth-consumingdevices. As a result, the stress on existing net-works continues to grow, while, due to the stag-nant number of users, the revenue continues tobe flat. This implies that for a provider to holdits market share, recurring investment in the net-work is mandatory without substantial returns

[1]. This problem is particularly pronounced inthe metro and core networks, where ports needto be added and the line rate continues toincrease with the explosion of bandwidth usage.Ways to compensate include:• Usage-based billing — a strategy that has

inertia among users• Smart network planning — with enough

foresight into bandwidth growth to be ableto meet future needs with existing infra-structure

As part of the continued efforts to plan net-works in the metro and regional segments, pro-viders are adopting an initiative to create amulti-vendor-supported network infrastructurethat facilitates better provisioning and the abilityto scale in terms of number of users and band-width. While initial capital expenditure(CAPEX) optimization is critical, it must also benoted that operation expenditure (OPEX) mini-mization becomes a key objective, given its morepotent business impact. A key aspect of this min-imization is to enable data to be kept in thelower layers of the stack, resulting in minimalprocessing (and hence lower energy consump-tion by avoiding the use of expensive networkprocessor-based routing elements). To be able toarchitect a network that is scalable and future-proof, an operator has to rely and influencestandards that would support multi-vendor envi-ronments as well as facilitate growth that is notdependent on frequent network upgrades.

To this end, the vision of packet-optical inte-gration — a process that has already begun, andwould potentially lead to the best of both the IPand the WDM worlds, using enablers such ascarrier Ethernet and the optical transport net-work (OTN) — has arisen.

In this article, we capture the essence of thework being done in the IEEE, InternationalTelecommunication Union (ITU), and InternetEngineering Task Force (IETF) standard bodiesfrom the perspective of next generation networkplanning in the optical, data, and network layers.

ABSTRACT

Network operators are currently in the midstof a situation in which there is doubling of band-width requirement every two years, although thegrowth in revenue is nearly flat. In this situationproviders must plan their networks to optimizefor future growth and take into considerationflat-to-declining operating margins. The OPEXbenefits of keeping the data in the lower layersof the network stack can be leveraged to planfuture networks. We focus on the impact ofevolving standards in the lower layers of thetelecommunications stack through the contribu-tions of the IEEE, ITU, and IETF. Specifically,we focus on the WDM, carrier Ethernet, OTN,MPLS, and MPLS-TP set of transport standards.These standards have evolved and are beingadapted to meet the requirements of next gener-ation networks and currently being considered inthe standard bodies. We show through a simula-tion study that by adopting the packet-opticalintegrated standard families, providers wouldbenefit because of CAPEX reduction and there-by enhance margins — benefiting the e-commu-nity through better services.

OPTICAL COMMUNICATIONS

Ashwin Gumaste, Indian Institute of Technology, Bombay

Shamim Akhtar, Comcast Cable Communications Inc.

Evolution of Packet-Optical Integrationin Backbone and Metropolitan High-Speed Networks: A Standards Perspective

GUMASTE_LAYOUT_Layout 10/28/13 2:25 PM Page 105

IEEE Communications Magazine • November 2013106

We discuss a vision of a managed transportinfrastructure based on low-CAPEX-and-OPEXequipment. We discuss how standards will drivethe bigger picture of a low-cost network that canbe soft-upgraded to meet new service and userneeds. Specifics of contributions in the IEEE,ITU, and IETF are discussed across the threelayers of the stack. We discuss enablers to stan-dard development. We showcase numericalresults that justify the inclusion of standards innext generation networks.

THE PACKET-OPTICAL NETWORK:THE BIG PICTURE

The packet-optical network is optimized for bothtransport as well as user services. In parallel, sucha network achieves scalability at low price pointsusing platforms that support low energy consump-tion and adaptable policies for optimized switch-ing and routing. Providers currently have networksthat are not optimized for efficiency and usemulti-layer equipment that only adds redundancyacross the layers. What is desired is a networkdesigned to scale and optimized for performance.In quest of such a design, standards play a criticalrole, as markers that guide deployment as well asfacilitate multi-source agreements (MSAs). Wedesire to be able to achieve the following networkhierarchy through the evolution of packet-opticalstandards.

SCALABLE, FAULT-TOLERANT, TOPOLOGY-AWARE ROUTING

The network layer is dominated by the IP/multi-protocol label switching (MPLS) overlay.IP/MPLS allows operators good control overtraffic. The problem is the inherent high cost perport. The problem is particularly pronouncedwhen we consider the scalability of IP/MPLSrouters. As we populate more ports to existingrouters, these saturate the chassis, and there isin fact a degradation of throughput due to lackof scalability of the non-blocking fabric. It ishence desired to be able to scale the port countof IP/MPLS routers to meet future service needs.A second issue is that of line-rate. As we moveto 40/100 Gb/s and beyond, the ports must notonly support the existing line rate, but in manycases be able to seamlessly migrate to new ones.Due to the higher degree of functionality ofIP/MPLS routers, these are desired to be mini-mized across a network to conserve bothCAPEX and OPEX. Currently, there is no directestablished relationship between traffic (andtraffic churn) and the amount of IP/MPLS corerouters required. Standardization of such a rela-tionship can lead to an optimized network andgood quality of experience (QoE) for the enduser. Perhaps this challenge is difficult toaddress.

CARRIER-CLASS SERVICE-AWAREFORWARDING PLANE

Layer 2 services and devices have efficientlymigrated to 10 Gb/s and in many instances to 40Gb/s as well as 100 Gb/s per port. Layer 2 tech-

nologies, especially Ethernet-based are carrier-class, implying good operations, administration,maintenance, and provisioning (OAM&P) capa-bilities, in addition to efficient sub-50 millisec-ond restoration using connectivity faultmanagement (CFM) standards. The functionalityof carrier Ethernet (CE) is not fully utilized. Theperformance of CE is such that it is possible toget circuit-like-performance at a fraction ofCAPEX of synchronous optical network/digitalhierarchy (SONET/SDH) systems. However, thereplacement of SONET/SDH gear with carrierEthernet is slow. This is because of the lack ofinsight into the full capabilities of CE networkarchitecture. While the standards in this area,such as the 802.1Qay for provider backbonebridging with traffic engineering (PBB-TE) andRFC 5317 (and beyond) for MPLS with trans-port profile (MPLS-TP), have matured, these donot dwell on network-level issues. There is a gapin how CE standards can be applied within anetwork and how efficient mapping of servicesto CE occurs. From the work in both the IETFand IEEE, the new drafts in this area focus onaspects such as segment protection (802.1Qbf),data center bridging, shortest-path bridging, gen-eralized MPLS (GMPLS) control of CE net-works, and so on — all of which point to thelack of a relationship between the technology(CE) and the service.

TRANSPORT-FRIENDLY OPTICS; INTELLIGENT,DYNAMIC OPTICAL CROSSCONNECTS

Since the photonic bubble burst in the early2000s, there has been new effort in the photonicnetworking domain. We have witnessed thedevelopment of the reconfigurable optical add-drop multiplexer (ROADM) as an infrastructureto support multi-wavelength add-drops andcrossconnect functionality, leading to the con-cept of a wavelength switched optical network(WSON) [2]. In parallel, we have also experi-enced the advent of new photonic technology forcoherent optics to support 40 Gb/s and 100 Gb/ssystems in metro and regional networks. Thedeployment of photonic networks as an efficientunderlay to IP/MPLS infrastructures has impliedthat there is a need for the optical network tounderstand, follow, and perhaps even optimizethe overlay. The power of the optical bypass (asis popularly known) is a tool that can withstandthe lack of scalability of IP/MPLS routers (asseen through our simulation study). There is thediscussion around flexible grid spacing [3] (grid-less optics) that aim to pack more channels inthe fiber by flexibly using the spectrum anddoing away with rigid channel spacing, resultingin more bandwidth in the network. Neither opti-cal line rate nor wavelength count increases inproportion to the rate of data growth. The lackof MSAs for new optical components and stan-dards that can modularize the different aspectsof higher-line-rate transponders means there areseveral non-technical bottlenecks that impactfaster production of higher-speed systems. Theconcept of software-defined optics and pluggablemodules that can be replaced to achieve a higherrate are instructive for future standards develop-ment.

The network layer is

dominated by the

IP/MPLS overlay.

IP/MPLS allows

operators good

control over traffic.

The problem is the

inherent high cost

per port. The prob-

lem is particularly

pronounced when

we consider the

scalability of

IP/MPLS routers.

GUMASTE_LAYOUT_Layout 10/28/13 2:25 PM Page 106

IEEE Communications Magazine • November 2013 107

MALLEABLE, SERVICE-CENTRIC CONTROL PLANE

With disparate layers, it is of interest to controlthe network using a unified control plane. The dis-cussions around the creation of a unified controlplane are at least a decade old. However, despitemuch effort, not much deployment of a unifiedcontrol plane has happened in the field. Scalabili-ty, inter-domain issues, multilayer equipment,upgrade of network gear, and service rollouts aresome of the issues that impact control planedeployment. There is a significant agreementbetween vendors of layer 1–3 gear about the useof GMPLS as an over-the-top standard. Despitethis agreement, the problem is that most vendorsdeploy control through a proprietary protocol.The functionality of the control plane is restrictedto only service provisioning or support for low-impact equipment. This implies that between thecontrol plane and the network management sys-tem (NMS) there is a clear distinction, preventingthe same control plane from being used acrossequipment from different vendors. There is a seri-ous lack of effort (and perhaps business will) inbeing interoperable at the control plane level.

To be able to meet the above features, weneed specific technology changes, primarilybrought about by standards that are quintessen-tial to the next generation network. These arecaptured as below.

ABILITY TO SUPPORTFLEXIBLE SPACING OPTICAL NETWORKS

With the massive growth of traffic, there is a need tomaximally utilize the fiber. Advanced modulation for-mats such as quadrature phase shift keying (QPSK)and orthogonal frequency-division multiplexing(OFDM) are enablers at high line rates (100 Gb/sand beyond). With the advent of coherent optics, it ispossible to provision a large number of channels atnarrow spectral spacing. Such ultra-dense multiplex-ing techniques would not only facilitate the optimiza-tion by an optical bypass, but may also facilitate thelong-term vision of a wavelength to an enterprise.Flexible-spacing-capable optical network proliferation(also called gridless networks) would require a signifi-cant cost reduction of coherent receivers by means ofphotonic integration technologies.

ABILITY TO SUPPORTCARRIER-CLASS PACKET TRANSPORT

Most services today are in packet format. Forboth application and network flexibility, packetmode communication is a must. This means thata provider must be able to provision a packet -ized service that is billable, and can be adminis-tered and maintained. Carrier-class packettransport technologies are maturing, and theiradoption would be critical to support next gener-ation services, particularly those within a cloudenvironment or for mobile backhaul.

ABILITY TO SUPPORT MASS NETWORKMIGRATION THROUGH DYNAMIC NETWORKING

Contemporary providers have to facilitate a deli-cate provisioning balance between content distri-bution network (CDN) traffic and traffic

generated by users. At times, CDN traffic canlead to uncontrolled surges, especially video traf-fic, which is voluminous and delay-sensitive. Forexample, a webcast of a popular sporting eventhas the capability of disrupting network opera-tions; such spurts of demand must be wellplanned or accounted for while designing thenetwork. There is not sufficient effort in theCDN standards community to interact with pro-viders leading to mutually beneficial provisioningand peering norms.

MULTI-DOMAINNETWORK ANALYTICS AND MONITORING

With the growth and diversity of services, mobilerollouts, data center/cloud providers, and so onmake the Internet a collection of specializedentities. Tier 1 providers have the onus of inter-acting with multiple tier 2 and tier 3 entities. Insuch a multi-domain environment with diversetechnology deployments, managing end-to-endservices and maintaining network analytics is achallenge. Individual technology standards suchas the Metro Ethernet Forum for Ethernet ser-vices exist. However, there is an absolute lack ofstandards that focus on multiple technologiesacross multiple domains for the same service.This creates QoE discrepancy and leads to non-optimized service performance.

NETWORK SCALABILITY USING SOFT UPGRADESLack of standardization specific to scalabilityimplies that it is difficult to scale existing equip-ment to newer technologies and line rates. Withthe current line rates, devices use specializedtechnologies for transmission, switching, control,and protocol stack. There is a need for standardsto be followed within the architecture of a plat-form so that MSAs can be adhered to andupgrades of technology can be “soft” rather than“hard.” This would tremendously save CAPEXas well as work well for the vendor who wouldhave a lock-in with the operator across genera-tions of features and upgrades.

SCALING AND EXPANDINGIP/MPLS NETWORKS

The vast deployment of IP/MPLS technology isan efficient mechanism to route packets betweensource-destination pairs. This one-stop technolo-gy is critical in providing complete routing, man-agement, and networking functionality. It comesat a price: high initial CAPEX, difficult to scalewithout significant periodic CAPEX infusionand control plane scalability issues. A biggerproblem with IP/MPLS systems is the difficultyin mapping revenue bearing services to theIP/MPLS network. Long Term Evolution (LTE)backhaul services are optimized for IP/MPLS,and their coexistence with other best effort ser-vices in the IP domain implies non-carrier-classsupport in the backhaul. In an era where uncon-trolled CDN traffic can create havoc on aprovider’s infrastructure [4], the limitations ofIP/MPLS technology specifically pertaining tosupport of carrier-class managed pipes becomescritical. While almost the entire IETF and large

Individual technology

standards such as

the Metro Ethernet

Forum for Ethernet

services exist. How-

ever, there is an

absolute lack of

standards that focus

on multiple tech-

nologies across

multiple domains for

the same service.

This creates QoE

discrepancy and

leads to

non-optimized

service performance.

GUMASTE_LAYOUT_Layout 10/28/13 2:25 PM Page 107

IEEE Communications Magazine • November 2013108

parts of the ITU are devoted to work on variousaspects of IP/MPLS technology, these are areaswhere new work is required.

Specifically, the scalability of IP/MPLS plat-forms is not well discussed in the standards. Itcan be argued that this is a vendor-specific issue.However, such an argument does not stand whenwe consider a holistic network view with the needto support scalable platforms at diverse locationsfrom different vendors. As line rates move from10 Gb/s to 40 Gb/s and to 100 Gb/s, there is aneed to scale the crossconnect fabric to supportlarger platforms. In an ideal world, for differentline rates, line cards and crossconnect fabricsshould be interchangeable provided the forward-ing plane adheres to the IETF RFCs. One wouldexpect the degree of scalability across vendors tobe such that operators are able to design a net-work with enough dynamic features. In currentsettings, it is difficult to plan a network, and onceprovisioned, it is difficult to replace gear easily.For example, if we have three vendors, X, Y, andZ, one problem is that not all of Y’s gear can beplaced anywhere. Similarly, Z’s gear may not beable to be used in conjunction with that of X toovercome its scalability deficiencies with Y. Thecontrol plane of Z may not be interoperable withX and Y, but its forwarding plane may be superbin a standalone network. The aforementionedissues are not just technology impairments; theyare due to the unavailability of standards for ven-dors to design platforms.

PACKET TRANSPORT: PBB-TE AND MPLS-TP

The high CAPEX and OPEX in SONET/SDHnetworks along with the inability to support sta-tistical packet multiplexing has accelerated CEstandards in the IETF, ITU with MPLS-TP, andIEEE with PBB-TE. Using packets for transportseemed the logical step in order to support vastdata traffic needs at low price points. The cur-

rent deployment of CE platforms, however, isquite the opposite. Even though it is expectedthat deployments will go up in the near future,these would be nowhere near what was expected.Partially this is due to lack of CAPEX budgetsfor operators. However, this is also due to a lackof clear vision for the technology itself. We bringto the fore certain issues that are addressed onlypartially through CE standardization efforts.

The CE standards (both PBB-TE and MPLS-TP) do not consider the legacy network. Forexample, several million T1s and E1s exist acrossthe globe; would it not make sense to migratethese to an all-Ethernet infrastructure? Whilesuch Ethernet-centric migration has happened,the standards do not focus on these. As a result,almost every provider deploying CE for end-to-end connectivity has a unique solution to offer.Both flavors of CE use service identifiers to mappayloads that forwarding elements can differen-tiate to process the data flows. To set these iden-tifiers (backbone virtual LAN [VLAN] identifiersor BVIDs in PBB-TE and labels in MPLS-TP)requires a control plane. While both standardsdwell on the control aspects of the protocol,there is little or no mention of identifier selec-tion, which is instructive for multi-vendor com-patibility. As a result, an operator deploying gearfrom a particular vendor can expect only for-warding plane interoperability between vendors.When we come to the more popular flavor ofCE (i.e. MPLS-TP), there is significant ambigui-ty in the relevant RFCs. RFCs 5317 and 5860discuss basic MPLS-TP and OAM aspects. Theambiguity on implementation and lack of com-pleteness are causes of implementation con-cerns. The implementation ambiguity allowsvendors to subscribe to their proprietary mecha-nisms. When compared to SONET/SDH, whichachieved good interoperability, CE interoper-ability is poor. The architectural problems due tointeroperability issues are immense: equipmentfrom vendor A claiming to be MPLS-TP-compli-ant is similar to vendor B’s MPLS equipmentwith some minor changes. The tragedy of thestandardization process is the conformance ofequipment to conflicting standards. Vendor D’sequipment, for example, conforms to a paradoxof carrier-class as well as a best effort packetstandard (PBB-TE and PBB). Theoretically, it ispossible to support both, but when implementedin a large network, it would create havoc —PBB’s Multiple Spanning Tree Protocol (MSTP)convergence would destroy all notions of carrier-class behavior in the network, leading to confu-sion, toggling, and crashing of the entirenetwork. While none of these have actually hap-pened in those select networks that deploy ven-dor D’s gear, it is just a question of time beforesome or all of the catastrophic effects occur.

THE OPTICAL TRANSPORT NETWORKAs Ethernet continued to reinvent itself fromcarrier sense multiple access with collision detec-tion (CSMA/CD)-based LANs to switched Ether-net, to Ethernet in the air (WiFi), and now toCE in WANs, there was a similar push inSONET/SDH toward being more packet-friendly.The first sign of such adoption was the facilita-

Figure 1. Multi-layer optimized network.

Carrier Ethernet transport

WDM transport

IP/MPLS overlay

GUMASTE_LAYOUT_Layout 10/28/13 2:25 PM Page 108

IEEE Communications Magazine • November 2013 109

tion of packet mapping using techniques such asthe link channel adjustment scheme (LCAS) andvirtual concatenation (VCAT). The generic fram-ing procedure (GFP, ITU-T 7041) was proposedas a better way to provide packet-over-SONET(POS) capabilities for variable data rates. POSled to next-generation SONET/SDH (NG-SONET), which had rich packet transport fea-tures including efficient traffic grooming andcontrol, but maintained high price points. Opera-tors that adopted these standards often did sodue to compatibility with legacy gear. In addition,the three decades of operational expertise withSONET/SDH implied a trained workforce. Tosupport SONET/SDH-like performance whiletaking advantage of WDM technologies acrossmultiple-provider domains, the ITU standardizedthe optical transport network. Apart from beingreadily able to map and encapsulate packetizedtraffic and provide SONET/SDH like OAM envi-ronment, OTN has two other major advantages:• The ability to include forward error correc-

tion (FEC) enhances reach to 40/100 Gb/s.• Providing tandem channel monitoring

(TCM) enables a user/operator to track asignal through multiple domains.

The OTN hierarchy and its use cases have beendescribed in [5]. Optical data unit (ODU) switch-ing and ODU-flex are two popular techniquesthat facilitate enhanced usage of the OTN hier-archy to map services to the OTN layer as wellas allow for efficient grooming. OTN standardshave focused on the architecture (ITU-T.G.872), protection, and signaling among otherfeatures. There are two aspects of the OTNstandards portfolio that require more work foroperator networks: • Making good use of OTN switching. Given

the redundancy of switching across multiplelayers — IP, Ethernet, OTN, and lambda— the role of OTN switching and how itwould impact the services portfolio leadingto business are unclear to most operators.Some questions we attempt to answer inour simulation study includes: –At what service granularity can ODUswitching best work?–Would ODU switching work better thanpacket switching for certain services, andhow do we define these services?

• Impact on network equipment and transla-tion of OTN information across layers.Router vendors are using OTN chipsets toencapsulate Ethernet frames within OTNs.The question operators ask is whether thisutilizes the full spectrum of OTN capabili-ties. If the idea is only to utilize FEC andhence enhance reach, the “OTN-izing” ofthe router interface has limited value. Fromthe perspective of standards, unless theaforementioned issues are standardized, itis difficult to interoperate OTN equipment,especially from the perspective of a largenetwork. Vendor O’s equipment may sup-port a smaller ODU-crossconnect than ven-dor P’s. What type and quantity ofequipment must an operator deploy?Unless the switching architecture and guar-antees are standardized, it is impossible fora designer to plan for future growth.

TOWARD SPECTRALLY EFFICIENTFLEXIBLE OPTICAL NETWORKING

With the commercialization of coherent optics aswell as extremely tightly spaced multiplexer tech-nology, flexible spacing oriented optical network-ing is becoming commercially feasible. Theapproaches of component vendors are quitediverse, and there is a critical lack of standardiza-tion as to how metro or core equipment wouldbe designed to conform to the super-tight specifi-cations juxtaposed by the flexible/gridless opticalnetwork. The goal of being able to provide awavelength or a spectral slice [9] to a user can beachieved in multiple ways. In one embodiment, asingle wavelength [6] consisting of many sub-wavelengths formed by OFDM techniques is sentto a group of users, each of which will extracttheir relevant data (on a subchannel). In anotherembodiment, the spacing between channels is

Figure 2. Flexible (gridless) optical network architecture.

Core IP / MPLS router

Flexible optics ROADM(grid-less support)

Table 1. Unit cost metric.

Equipment Cost for1 Gb/s

Cost for10 Gb/s

Cost for40 Gb/s

Cost for100 Gb/s

IP/MPLS 4 10 18 36

OTN 1 2.5 5 10

CE 0.25 2 6 12

Gridless TX/RX 0.5 1.5 4 8

ROADM TX/RX 0.25 0.5 1.8 4

ROADM crossconnect 4 4 8 16

GUMASTE_LAYOUT_Layout 10/28/13 2:25 PM Page 109

IEEE Communications Magazine • November 2013110

made extremely narrow, and an arrayed waveg-uide based multiplexing system is used to createthe composite signal. Some of the spectrally effi-cient approaches consider spanning across access,metro, and regional networks using coherentoptics to achieve reach [7]. Such a network hasthe potential of giving a completely new meaningto metro and access networks with good reachand efficient spectral utilization. The differentia-tion between the access and metro domains canpotentially be negated. The network has thepotential of reducing the amount of electronicequipment across the network. In this manner,bypassing the crossconnect at the metro-accessinterconnection as well as providing for IP bypassby using optical channels from the core to theedge of the network are possible. The flexiblespacing oriented optical network can manifest inmultiple architectures. Figure 2 shows the con-cept of a wavelength or slice-per-user architec-ture. The ROADM architecture required at theedge of the network is now replaced by a flexiblegrid supporting ROADM that not only providesoptical add-drop capabilities, but also absolvesthe need for edge routers/switches. Several ques-tions emerge: the optical architecture that spansthe unified network is undefined as to how tosupport such large number of channels. We must

also standardize the equipment at which to ter-minate the channels in the metro. This termina-tion equipment can be a collection of IP routersor CE switches, or some combination of both.

VERIFICATIONTo better understand the merits of a packet-opti-cal network, we built a multi-layer simulationmodel in Matlab. The model assumes a 120-nodenetwork with 24 core nodes and the remainingmetro nodes. Each node supports up to 100,000broadband users (using a distribution network).Each core node supports a cluster of metro nodesin an interconnected ring topology, while the corenodes form a mesh network. Our goal is to com-pute the CAPEX impact using network equipmentas well as optimized standards that may be avail-able in the future. For the purpose of cost compu-tation, we assume the costs in Table 1. Note thatthis table is derived from discussions with vendorsin North America as well as Asia, and costs areaveraged and then normalized [10]. The costs forflexible optical technology are only target costs assuch equipment is currently not available.

Based on the table, we performed simulationsby assuming user demands to arrive according toPoisson distribution and with exponential hold-ing times. Four levels of quality of service (QoS)were assumed. One hundredth of traffic wasassumed to be dedicated to network control.

In Fig. 3 we showcase the impact of a net-work with IP/MPLS, CE, and optical bypass(ROADMs). We compare the use of opticalbypass with select IP/MPLS nodes as well asoptical bypass with CE switches, and selectIP/MPLS nodes. We observe that with opticalbypass there is a direct cost reduction in themesh network, validating our argument of keep-ing the data in the lower layers of the stack. Afurther justification is seen when we add CEnodes to the simulation model. The cost furtherreduces, as those paths that could not be set uppurely with the optical bypass can now be set upusing CE. The role of IP/MPLS is restricted tothe core of the network. In fact, a side result ofthis view graph is that the adoption of largerswitch fabrics for IP/MPLS routers can bedelayed by the optimized CE+ROADM net-work in conjunction with the IP/MPLS network.

Shown in Fig. 4 is the impact of OTN switch-ing. It is observed that OTN switching has noimpact on a network that already deploys CEand IP/MPLS. However, when used individually(i.e. IP+OTN or CE+OTN), there is a costadvantage. While the latter (CE+OTN) solutionis less expensive, it is difficult to scale, especiallywhen requests are dynamic and require rapidprovisioning, which happens through anIP/MPLS network.

Shown in Fig. 5 is the impact of flexible opti-cal networking on CAPEX, and comparison withcable and xGPON options. Flexible optics signif-icantly reduces end-to-end CAPEX by eliminat-ing the need for aggregation/edge routers andreplacing these with spectrally efficient opticalgear. In fact, a spectrally optimized optical net-work is lower in cost than a cable or gigabit pas-sive optical network (xGPON) network, becausein spite of the higher target cost of optics, there is

Figure 3. Impact of IP/MPLS networks with standardized bypass and CE.

Network load30000

CA

PEX

in U

S$1

mill

ion

200

400

600

800

1000

1200

90006000 12,000 15,000 18,000 21,000 24,000 27,000 30,000

Unoptimized IP/MPLSOptical bypassOptical bypass+CE

Figure 4. Impact of OTN switching.

Load in 1000 requests per day30000

Cos

t in

US$

1 m

illio

n

200

400

600

1000

800

1200

1400

90006000 12,000 15,000 18,000 21,000 24,000 27,000 30,000

IP+CE+OTNIP+OTNCE+OTN

GUMASTE_LAYOUT_Layout 10/28/13 2:25 PM Page 110

IEEE Communications Magazine • November 2013 111

significant reduction of the core and metro equip-ment. What we have observed is that 37 percentof connections in the network can be provisionedin point-to-point optical channels (as spectralslices), removing the need for higher-layer (andmore expensive) equipment. We further observethat 29 percent of connections are provisionedsuch that the aggregation happens only in thecore, thus implying the metro part of the networkis largely free of aggregation equipment.

CONCLUSIONWe have presented the vision of a packet-opticalintegrated network architecture. Specifically, wehave focused on issues pertaining to network scala-bility and service delivery across IP/MPLS, CE,OTN, and optical domains. The qualitative andquantitative impact of the standards is also shown.Opportunities for new standardization approachesare discussed from the perspective of an operator.A key observation from this article is that stan-dards must impact both network architecture andplanning as well as product architecture and shouldperhaps be extended to service delivery. Thiswould result in benefiting the entire ecosystem.

REFERENCES[1] IEEE802.3 Industry Connections Ethernet Bandwidth

Assessment Ad Hoc Group Report, http://www.ieee802.org/3/ad_hoc/bwa/BWA_Report.pdf.

[2] “A Framework for the Control of Wavelength SwitchedOptical Networks (WSONs) with Impairments,” draft-ietf-ccamp-wson-impairments-10, http://tools.ietf.org/html/draft-ietf-ccamp-wson-impairments-10.

[3] “A Framework for Control of Flex Grid Networks,”draft-syed-ccamp-flexgrid-framework-ext-01.txt,http://tools.ietf.org/html/draft-syed-ccamp-flexgrid-framework-ext-01

[4] H. Chang, S. Jamin, and W. Willinger, “To Peer or Notto Peer: Modeling the Evolution of the Internet’s AS-Level Topology,” INFOCOM 2006.

[5] A. Gumaste and N. Krishnaswamy, “Proliferation of ITU709 - Optical Transport Network (OTN) — A Use CaseBased Study,” IEEE Commun. Mag., Sept. 2010.

[6] J. Armstrong, “OFDM for Optical Communications,” J.Ligthwave Tech. , vol. 27, no. 3, Feb. 2009, pp.189–204.

[7] H. Rohde et al, “Coherent Optical Access Networks,”Proc. OFC, Mar. 2011, San Diego, CA.

[8]Content Delivery Networks Interconnection, http://data-tracker.ietf.org/wg/cdni/.

[9] M. Jinno et al., “Spectrum-Efficient and Scalable ElasticOptical Path Network: Architecture, Benefits, andEnabling Technologies,” IEEE Commun. Mag., vol. 47,no 11, Nov. 2009, pp. 66–73.

[10] A. Gumaste et al., “Using Global Content Balancing toSolve the Broadband Penetration Problem in the Devel-oping World: Case Study — India,” IEEE Commun.Mag., May 2012, vol. 50, no. 5, pp. 74–81.

BIOGRAPHIESASHWIN GUMASTE (ashwing@ieee.org) is currently the Insti-tute Chair Associate Professor in the Department of Com-puter Science and Engineering at the Indian Institute ofTechnology (IIT) Bombay. From 2008 to 2012 he was alsothe J. R. Isaac Chair Assistant Professor. He was a visitingscientist with the Massachusetts Institute of Technology,Cambridge, Massachusetts, in the Research Laboratory forElectronics from 2008 to 2010. He was previously withFujitsu Laboratories (USA) Inc. in the Photonics NetworkingLaboratory (2001–2005). He also worked in Fujitsu NetworkCommunications R&D, Richardson, Texas, and prior to thatwith Cisco Systems in the Optical Networking Group(ONG), and has been a consultant to Nokia Siemens Net-

works, Munich, Germany, working on next generationaccess standards. His work on light-trails has been widelyreferred, deployed, and recognized by both industry andacademia. His recent work on omnipresent Ethernet hasbeen adopted by tier 1 service providers and also resultedin the largest ever acquisition between any IIT and indus-try. This has led to a family of transport products. He has20 granted U.S. patents and over 30 pending patent appli-cations. He has published about 150 papers in referredconferences and journals. He has also authored threebooks in broadband networks: DWDM Network Designsand Engineering Solutions (a networking bestseller), First-Mile Access Networks and Enabling Technologies, andBroadband Services: User Needs, Business Models andTechnologies (Wiley). Due to his many research achieve-ments and contributions, he was awarded the Governmentof India’s DAE-SRC Outstanding Research InvestigatorAward in 2010 as well as the Indian National Academy ofEngineering’s (INAE) Young Engineer Award (2010). Hewas also the recipient of the Vikram Sarabhai ResearchAward in 2012. He was the IBM Faculty Award winner for2012. He has served as Program Chair, Co-Chair, PublicityChair, and Workshop Chair for IEEE conferences and as aProgram Committee member for IEEE ICC, GLOBECOM,OFC, ICCCN, Gridnets, and others. He has also been aguest editor for IEEE Communications Magazine and IEEENetwork, and the founding Editor of the IEEE ComSocOptical Networks Technical Committee newsletter Prism. Heis Chair of the IEEE Communication Society’s TechnicalCommittee on High Speed Networks (TCHSN), 2011–2013.He has been with IIT Bombay since 2005, where he con-venes the Gigabit Networking Laboratory (GNL), www.cse.iitb.ac.in/gnl. The GNL has secured over US$11 million infunding since its inception and has been involved in fourmajor technology transfers to industry.

SHAMIM AKHTAR (shamim_akhtar@cable.comcast.com) isresponsible for driving the network technology platformand architecture roadmap for Comcast’s truly convergednational IP/optical backbone, metro, edge, and access net-work. His technology innovation and operations leadership,both inside and outside Comcast, has brought tremendousmomentum in the area of vendor agnostic network scalingto support triple play residential, MEF-based business ser-vices, and mobile backhaul services over one of the largestconverged IP/optical networks in the world. He is a fre-quent speaker in the packet optical converged core andaccess space. He is instrumental in aligning system-levelthought processes in scaling speeds beyond 100 Gb/s to 1Tb/s in North America by collaborating with carriers, tech-nology partners, and research organizations. He has beeninvolved in critical technology acquisition and investmentdecisions in IP/optical industry, assisting with his experienceand insight on the length and breadth of network tech-nologies, and their operational models across core, edgeand access network. He is an IIT graduate with workingknowledge of MSO/carrier network in North America,Europe, and APAC through his current and prior experi-ences in Philips, VPISystems, and Internet Photonics/Ciena.

Figure 5. Impact of a flexible optical network on CAPEX.

Network load in 1000 users3000

0

CA

PEX

in U

S$1

mill

ion

200

400

600

1000

800

1600

1200

1400

1800

9000 12,000 15,000 18,000 21,000 24,000 27,000 30,0006000

IP/MPLS+ROADM+CableIP/MPLS+ROADM+PONIP/MPLS+Flexible-Optical network

GUMASTE_LAYOUT_Layout 10/28/13 2:25 PM Page 111