MAca Protocol

8/12/2019 MAca Protocol

http://slidepdf.com/reader/full/maca-protocol 1/6

Software Dened Networking: Meeting CarrierGrade Requirements

(invited paper)

Dimitri Staessens, Sachin Sharma, Didier Colle, Mario Pickavet and Piet DemeesterDepartment of Information Technology

Ghent University - IBBTGhent, Belgium B-9050

Email: rstname.lastname@intec.ugent.be

Abstract —Software Dened Networking is a networkingparadigm which allows network operators to manage networkingelements using software running on an external server. Thisis accomplished by a split in the architecture between theforwarding element and the control element. Two technologies

which allow this split for packet networks are ForCES andOpenow. We present energy efciency and resilience aspectsof carrier grade networks which can be met by Openow. Weimplement ow restoration and run extensive experiments in anemulated carrier grade network. We show that Openow canrestore trafc quite fast, but its dependency on a centralizedcontroller means that it will be hard to achieve 50 ms restorationin large networks serving many ows. In order to achieve 50 msrecovery, protection will be required in carrier grade networks.

Index Terms —Openow, Restoration, Energy Efciency

I. INTRODUCTION

The goal of Software Dened Networking is to provide

open user-controlled management of the forwarding hardwareof a network element. Openow was designed particularlyto deploy and test experimental protocols in the productionquality campus network Stanford uses every day, instead of ina separated lab environment [1]. If operators want to be ableto program the behaviour of high speed networking elementssuch as IP routers or Ethernet switches for their custom needs,they require direct programming of the forwarding hardware.Modern routers/switches contain a proprietary FIB (Forward-ing Information Base), which is implemented in hardwareusing TCAMs (Ternary Content Addressable Memory).

Openow provides control of forwarding hardware by pro-viding a standardized abstraction of it called a Flowtable.An Openow switch is a network element implementing aninstance of the (abstract) Flowtable, and has a secure channelto the Openow controller, which manages the Openowswitches using this Openow protocol. The Openow protocolsupports messages to add, delete and modify ow entries inthe Flowtable. A ow entry consists of (1) a packet header which denes the ow, (2) an action which denes how thepacket should be processed, and (3) statistics which keep track of the number of packets per ow, the number of bytes perow, and the time since the last packet matched per ow.

The controller installs these ow entries in the Flowtablesof the Openow switches. Incoming packets processed bythe Openow switches are compared against the ow entriesin the Flowtable. If a matching ow entry is found, the

predened actions for that entry are performed on the matchedpacket. If no match is found, the packet is forwarded to thecontroller over the secure channel (PACKET IN message).The controller is responsible to determine how the packetshould be handled; either by returning this specic packet tothe switch and stating which port it should be forwarded to(PACKET OUT message) or by adding valid ow entries inthe Openow switches (FLOW MOD message).

In the access/aggregation domains of most carriers, anEthernet-based aggregation is used to provide services forresidential and business customers. Implementing a split archi-tecture between control and forwarding in these areas createsthe opportunity for network operators to use commodity hard-

ware under the control of centralized software to perform theintricate functions of specialized aggregation domain network elements at much a lower hardware cost. Moreover, Openowallows virtualization and thereby supports service separation.As mobility is essential, the infrastructure to provide wirelessservices is a must and the aggregation network is also used toguarantee mobile backhauling.

The term carrier grade [2] describes a set of functionalitiesand requirements that architectures should support in order tofulll the operational part of network operators. The require-ments are (1) Scalability (2) Reliability (3) Quality of Service(QoS) and (4) Service Management. In order to be applied tocarrier grade networks, Openow must be able to meet theserequirements. In this paper, we focus on reliability, and we alsoshow how Openow can enable energy-efciency improvingstrategies to be deployed in the network. Reducing network power consumption can lead to improved hardware scalabilityand reduces the carbon footprint.

I I . E NERGY EFFICIENCY IN O PENFLOW NETWORKS

An important goal for future networking is the reductionof its carbon footprint. The attention for climate change isinuencing the ICT sector. ICT accounts for 2 to 4% of

8/12/2019 MAca Protocol

http://slidepdf.com/reader/full/maca-protocol 2/6

the worldwide carbon emissions [3]. About 40 to 60% of these emissions can be attributed to energy consumption inthe user phase, whereas the remainder originates in otherlife cycle phases (material extraction, production, transport,end-of-life). By 2020 the share of ICT in the worldwidecarbon emissions is estimated to double in a business asusual scenario. Since optical signals consume less power thanelectrical signals, optical technologies could enable higherenergy efciency. There is a worldwide increase in researchefforts in the last years, with initiatives such as the current EUfunded Network of Excellence TREND [4], COST action 804[5] , the GreenTouch consortium [6] and the CANARIE fundedGreenStar Network [7] which also has European members.

III . E NERGY SAVING STRATEGIES

When it comes to solutions to save power in carriernetworks, different strategies are possible. On the highestlevel one can investigate if optimizations are possible in thenetwork topology. Currently, networks are designed to handlepeak loads. This means that when the loads are lower an

overcapacity is present in the network. At night time the trafcload can be 25% to 50% of the load during day time. Thislower load could allow a more simplied network topologyat night which in turn allows certain links to be switched off.Additionally, the switching off of these links allows for linecards to be switched off and thus leads to reduced node powerconsumption. An example which implements this principle ismultilayer trafc engineering. The MLTE (Fig. 1 [8]) approachcan lead to power savings of 50% during low load periods.

Fig. 1. Multilayer Trafng Engineering (MLTE)

The Openow architecture allows us to implement MLTEoperations as an application in the Openow controller. Inorder to have the maximum benet, the controller should beable to power up/down parts of the switch on demand, as afunction of the energy-efcient algorithms in the application.Openow currently has limited support for the control of power management in the switches. For enabling efcientpower management, we should allow the burst and adaptivelink modes in the switches and advertise them to the controller.On the controller side, it should be extended to allow control of such energy efcient features. Since the Openow architectureremoves the control software part from the switches andmoves it to a central location, we could expect a reductionin power consumption in the switches at the cost of the powerconsumption of the controller.

Since access networks are organized in tree structures,shutting down links is not a feasible option, which meansdynamic topology optimization cannot be applied in an accessnetwork. On a given topology further optimizations can beachieved by using adaptive link rates and burst mode operation[9]. Adaptive link rate is based on the principle that lowerlink rates lead to lower power consumption in the network equipment. By estimating the average link rate required ona line and adapting the link rate to this level power savingbecomes possible. Another possibility is burst mode operationwhere packets are buffered in a network node and then sentover the link at the maximal rate. In between the bursts the linecan be powered down. These strategies can be mainly useful inaccess networks due to the burstiness of the trafc. However,the difference in power consumption between different link rates is mainly manifested at the higher bit rates. Secondly,burst mode operation works on very small time scales sothe number of components which can be switched off islimited. Finally, both approaches require larger packet bufferswhich also need powering. Hence it is yet unclear whether the

strategies in reality can lead to signicant power optimization.

Fig. 2. Power management

To enable energy efcient networking applications to runon the Openow controller, some extra messages should beadded to the Openow specication which not only indicatethe status of a port on the switch, but also allows us to controlthe individual ports.

The following features should be controllable: portpower up/down, burst mode, adaptive line rate. Firstof all the controller needs to be aware of the capa-bilities of the switch. So we would add the energy-efciency features to the OFPT FEATURES REQUEST andOFPT FEATURES REPLY messages. In order to controlthe functionality, we need some extra protocol messages.These can be done by adding the capabilities to theOFPT PORT MOD message. All these messages can usethe ofp port structure to advertise features and to changethem as required. Currently the ofp port structure has a bitOFPPC PORT DOWN in the cong eld, which indicateswhether a port is administratively down. This can also beused to power on/off a specic port on the switch using theOFPT PORT MOD message. We can implement adaptive line

8/12/2019 MAca Protocol

http://slidepdf.com/reader/full/maca-protocol 3/6

rate using the curr speed and max speed elds. For the otherfeatures, it has to be investigated where they must be addedto the ofp port structure in the cong eld, the features eldand/or the advertised elds. We would also need to add theOFPC BURST MODE.

Fig. 3. Energy consumption of a core router [10]

Consolidation of the control software and hardware outside

of the switches means a reduction in switch power consump-tion offset by the power consumption of a new element, thecontroller. As we can see from Fig. 3, most of the powerconsumption stems from the Forwarding Engine (32%) andthe Cooling/Power Suply (35%). Only a small fraction (11%)of the power is consumed by the control plane, of whichonly 5% in the routing engine [11]. Openow allows us toreduce this part by moving the routing tables (RIB)/routingengine and control plane functionality to the controller andkeeping only the forwarding engine (FIB) in the switch with asmaller Openow software component and extra hardware forperforming communication with the controller. The controllerwill consume more power due to power supply inefciency

than the reduction in power in the switches, so we think the general Openow architecture will be similar in powerconsumption compared to conventional network architectures.

IV. D ATA PLANE RESILIENCY

Carrier grade networks should be able to detect and recoverfrom incidents without impacting users. Hence a requirementis added in the carrier grade network so that it should recoverfrom failure within 50 ms sub interval [12]. Resilience mech-anisms [13] can be classied as restoration and protection.In case of protection, the paths are preplanned and reservedbefore a failure occurs. When a failure occurs, no additionalsignaling is needed to establish the protection path. In caseof restoration, the recovery paths can be either preplannedor dynamically allocated, but resources are not reserved untilfailure occurs. When a failure occurs additional signalingis needed to establish the restoration path. Protection is aproactive strategy while restoration is a reactive strategy.

Data plane recovery in Openow networks can be donein two essentially different ways. One is to support therecovery mechanisms of a specic implemented protocol intoan Openow application. An example is supporting MPLS-TE[14] [15] which has restoration functionality in its own control

plane. The other approach is to build resilience into Openow,supporting the recovery of arbitrary ows, regardless of thetype of trafc they are carrying. This is the option we explorein this section.

A. Data plane restoration

Fast ow restoration in the Openow data network requiresan immediate action of the controller after notication of a link status change event thrown by the Openow switchdetecting the failure. The recovery is performed by removingthe incorrect ow entries and installing new entries in allaffected Openow switches as fast as possible following thenotication of the link failure. We use the following algorithmto restore trafc: after the controller gets notication of a link failure, a list is made of all affected paths. For all these affectedpaths, a restoration path is calculated using a shortest pathalgorithm on the remaining topology. For affected switcheswhich are on both the working and the restoration path, theow entry is modied. For the other switches, there are 2possibilities. If the switches are only on the failed path, the

entries are delete. If they are only on the restoration path, newentries are added.

B. Data plane protection

In order to further reduce packet loss resulting from restora-tion actions in an Openow network, we can turn to protection.Protection removes the need of the Openow switches to con-tact the controller for the deletion, modication and additionoperations required to to establishment the restoration path.This is accomplished by precomputing the protection pathand establishing it together with the original (i.e. working)path. This means adding the ow entries for the protectionpath in the switches. Advantages of this are faster recovery,

but the disadvantage is a larger Flowtable, which increasescosts (expensive TCAM equipment) and may slightly impedeforwarding performance.

Path protection requires end-to-end monitoring of the pathto enable the quick switchover. The recovery mechanismdepends on Bidirectional Forwarding Detection (BFD) sessionto declare the fault in path. BFD is a network protocol used todetect faults between two end-points. It provides low-overheaddetection of faults even on physical media that don’t supportfailure detection of any kind, such as Ethernet, virtual circuits,tunnels and MPLS Label Switched Paths.

V. C ONTROL PLANE RESILIENCY

Because Openow is a centralized architecture, relying onthe controller to take action when a new ow is introduced inthe network, reliability of the control plane is of very highimportance. The controller should also be resilient againsttargeted attacks. There are multiple options for control planeresiliency. One can provide two controllers, each in a separatecontrol network and when connection to one controller islost, the switch switches over to the backup network. Thisis a very expensive solution. Another option is to try torestore the connection to the controller by routing the control

8/12/2019 MAca Protocol

http://slidepdf.com/reader/full/maca-protocol 4/6

trafc over the data network. When a switch loses connectionto the Openow controller, it sends its control trafc toa neighboring switch, which will require the controller todetect such messages and establish ow entries for routingthe control trafc through this neighbour switch. This through-the-data-plane solution is an intermediate step towards full in-band control. An effective scheme for carrier grade networksmay be to implement out-of-band control in the failure freescenario, switching to in-band control for switches who losethe controller connection after a failure. In-band control issupported in the Openow specication.

VI . C ASE STUDY : RESTORATION IN A NATIONWIDE

TOPOLOGY

Fig. 4. German backbone network topology

In this section we evaluate ow restoration in Openownetworks. The topology of the experiment is depicted in Fig.4. It contains 14 nodes and 21 links.

We emulated this topology on our iLab.t Virtual Walltestbed. The iLab.t Virtual Wall facility is a generic testenvironment for advanced network, distributed software andservice evaluation, based on emulab [16]. The virtual wallfacilities consist of 100 (linux) nodes (dual processor, dualcore servers, 6x1 Gb/s interfaces per node) interconnected viaa non-blocking 1.5 Tb/s VLAN Ethernet switch, and connectedto a display wall (20 monitors) for experiment visualization.Each node is connected with 4 or 6 Gigabit Ethernet links tothe switch.

Each of the 14 Openow nodes is connected to a servernode (not shown) and also has a dedicated interface to aswitched ethernet LAN which establishes connection to theNOX controller (i.e. out-of-band control). All links are 1GbEthernet links. We implemented the restoration algorithm inthe NOX Openow controller [17]. Switches are running OpenvSwitch [18]. To evaluate the restoration time, we generate

packets using the linux kernel module pktgen with 3 msintervals. In total there are 182 ows in the network. We alsoloop trafc from each server through its connected switchto verify pktgen generation consistency. All server nodeshave manually congured routing tables which send all trafcthrough the interface connected to the Openow network.

Fig. 5. Trafc received at NOX

We will use the incoming trafc intensity at the NOXcontroller as captured using tcpdump to illustrate the experi-ment setup. This is shown in Fig. 5. At the beginning of theexperiment, the Openow switches connect to the controllerwhich generate the low spikes in the rst seconds of theexperiment. Then we start sending the pktgen trafc, waitingone second between each server to allow the NOX to installthe ow entries. These are the 14 large spikes 9 s to 23 s inthe experiment. The number of packets depends on the pathlengths as each switch requests ow entries to the controller.The one second interval is to make sure we don’t accidentally

overload the NOX by trying to establish too many ows ina short timespan. After the ows are installed, we see smallspikes at 5s intervals. These are ECHO REQUEST messageswhich are sent to check liveness of the controller link. Atroughly 44 .5 s into the experiment we break the link Berlin-Hamburg by giving an eth down command in the Berlinswitch, which will trigger a PORT STATUS message sent tothe controller when the Berlin switch detects its Ethernet portis down. At this point, the controller starts recovery (largespike at around 44 .5 s) and all trafc is restored.

Fig. 6 shows trafc on the link Berlin-Hamburg as capturedin Hamburg. Capturing with tcpdump started roughly 8 sec-onds into the experiment. We show both the total trafc on thelink, as the (unidirectional) trafc from Berlin to Hamburg. Atthe start we see a stepwise increase in trafc as each pktgen isstarted on each node. The large step at approximately 15 s iswhen the server at Hamburg starts sending trafc to all otherclients. By 23 s all trafc is set up in the network. Each owis roughly 300 packets per second, total trafc on the link isaround 6000 packets per second. At around 44 .5 s, the link isbroken and all trafc on the link is lost.

Fig. 7 shows the trafc on the link Bremen-Hamburg. Afterthe link Berlin-Hamburg is taken down, this is the only link

8/12/2019 MAca Protocol

http://slidepdf.com/reader/full/maca-protocol 5/6

Fig. 6. Trafc on affected link

Fig. 7. Trafc on restoration link

connecting Hamburg, so all trafc from and towards Hamburg

must now follow this link. Before the link failure, no trafcfrom Berlin to Hamburg is on this link. At the time of the link failure, we see a small drop in total trafc on the link. This isthe trafc which was coming from the link Berlin-Hamburgover Hamburg-Bremen which is now lost. The restorationon the controller reroutes the affected trafc, and we see anincrease in the trafc on this link. After restoration, the link serves all trafc to Hamburg, so it also contains the trafcfrom Berlin.

Fig. 8 shows the trafc from the viewpoint of theclient/server at Hamburg. After the initial setup of the pktgenin all servers, the trafc is stable until the link failure at 44 .5

s. Hamburg sees a drop in received trafc as some ows are

lost until the controller restores the trafc. The trafc owfrom Berlin shows that its trafc is completely lost over ashort interval, inspection of the tcpdump traces shows this tobe 289 .75 ms.

Fig. 9 shows a detail of the outgoing trafc from theNOX controller during the recovery phase, captured in 1 msintervals. At approximately 44 .498 s, NOX starts sending theFLOW MOD (modify/delete/add) messages required to installthe new Flow entries in the affected switches. The tcpdumptrace shows that in total 118 FLOW MOD messages are sent

Fig. 8. Trafc at Hamburg

Fig. 9. NOX outbound trafc during recovery

to restore 19 affected ows in 102 ms, including the pathcalculation times. The 3 (last) packets in the graph at times45 .61 − 45 .63 s are TCP ACK messages and need not beincluded, recovery is nised at time 45 .6 s.

Fig. 10. Restoration times

We did this experiment for a number of link failures inthe given experimental topology. The results are depicted inFig. 10. The x-axis shows the broken link, the y-axis shows,in milliseconds, the minimum restoration time (or the time it

8/12/2019 MAca Protocol

http://slidepdf.com/reader/full/maca-protocol 6/6

took to restore one connection), the maximum restoration time(or the time it took to restore all connections) and the averagerestoration time (the expected time for any ow to be restoredafter a failure). The links are ordered from left to right in thenumber of ows that are affected if this link fails, shown inbrackets in the axis labels.

The gure shows that the rst ows are restored roughly180 ms after the failure, while total recovery takes anywherebetween 260 and 310 ms. Also, there seems to be only minordependence on the number of ows which had to be restored.In fact, recovery time will depend on the number of ows tobe restored, the average path length of each ow, the averagepath length of the restoration paths and the average number of nodes which are on both the working and the restoration path.Furthermore, there are some unknown random factors, suchas momentary load of the NOX cpu and trafc bursts in thecontrol network. Such factors make drawing conclusions froma single experiment difcult. Therefore we will perform moreexperiments to allow statistical analysis, which will allow usto see how the recovery time scales with the number of ows

to be restored.The recovery times in Fig. 10 are substantially longer thanthe 100 to 105 milliseconds it takes for the NOX to calculatethe new paths and send out the FLOW MODS to the switches.Actually, the tcpdump traces show that the rst paths arerestored within 1 to 6 ms after the NOX starts sending thecorrect ow entries. The reason for the difference in delayis the failure detection time, i.e. the time it takes the linuxkernel to detect/declare that the ethernet link is down afterthe ”eth down” command effectively took the interface down.This seems to be roughly 170 ms in our experiments. So asignicant part of the recovery time in Fig. 10 is currently indetecting the failure. However, and this is an important point,

even with instant detection, while the recovery of a single owwould take on the order of 10 - 20 milliseconds [19], to recoverall ows in the network would take at least the time neededby NOX to perform all recovery actions. For our experimentsthis was roughly 80 − 130 ms. In carrier grade networks, thenumber of ows to restore will be orders of magnitude higher,requiring many more ows to be restored, which will severelystress the control network and controller hardware for the shorttimespan of recovery. In normal operation, the control network load is generally orders of magnitude lower. Implementing ahigh speed control network only for restoration will probablynot make sense. Implementing protection mechanisms in theswitches will be more cost-efcient, slightly increasing thebandwidth requirement at ow setup time due to extra protec-tion information to be sent to the switch, but highly decreasingthe bandwidth requirements during failures by allowing theswitch to perform the protection switching without controllerinterference.

VII. C ONCLUSIONS AND FUTURE WORK

We have presented two aspects of carrier grade networkswhich can be met by Openow, being improved scalabilityby reducing the energy consumption and performing recovery

in case of network failures. While the Openow architecturemay not be able to signicantly reduce energy consumptionby consolidating the control hardware/software in a singlemachine, it shows signicant promise by facilitating network-wide energy efciency solutions such as MLTE in combinationwith local power saving options such as controlled adaptiveline rates in the Openow switches. Second, we give anindication how Openow can handle both data plane andcontrol plane failures. We have implemented a ow restorationscheme in an open source controller (NOX) and ran extensiveexperiments in an emulated carrier grade topology. We showthat Openow can restore trafc, but its dependency on acentralized controller means that it will be hard to achieve50 ms restoration in large networks.

In future work, we will implement ow protection to beable to recover under 50 ms. Also we will experiment withhybrid in-band and out-band control for dealing with failuresin the controller network.

ACKNOWLEDGMENTS

This work was partially funded by the European Com-mission under the 7th Framework ICT research Programmeprojects SPARC [20], OFELIA [21] and Network of Excel-lence TREND [4].

R EFERENCES

[1] N. McKeown et al., Openow: Enabling innovation in campus networks ,SIGCOMM, Rev. 38(2), 69-74, 2008.

[2] http://metroethernetforum.org/index.php[3] M. Pickavet et al., Worldwide energy needs for ICT: The rise of power-

aware networking , in Proc. 2nd International Symposium on AdvancedNetworks and Telecommunication Systems (ANTS), 2008.

[4] http://www.fp7-trend.eu/, Towards Real Energy-efcient Network De-sign

[5] http://www.irit.fr/cost804/, Energy Efciency in Large Scale DistributedSystems

[6] http://www.greentouch.org/ [7] http://www.greenstarnetwork.com/ [8] B. Puype et al., Multilayer trafc engineering for energy efciency ,

Photonic Network Communications, vol. 21, no. 2, 127-140, 2011.[9] B. Nordman et al., Reducing the Energy Consumption of Networked

Devices , IEEE 802.3 tutorial, 2005[10] R. Tucker et al., Evolution of WDM Optical IP Networks: A Cost and

Energy Perspective , Journal of Lightwave Technology, vol. 27, no. 3,243-251, 2009.

[11] W. Vereecken et al., Power consumption in telecommunication networks:overview and reduction strategies , IEEE Communications Magazine,vol. 49, no 6, 62-69, 2011.

[12] B. Niven-Jenkins et al., MPLS-TP requirements , RFC 5654, IETF 2009[13] J. P. Vasseur et al., Network recovery: protection and restoration of

optical, SONET-SDH, IP and MPLS , Morgan Kaufmann, 2004.[14] A. R. Sharafat et al., MPLS-TE and MPLS VPNs with Openow ,

SIGCOMM 2011.[15] A. Kern et al., MPLS-Openow based access/aggregation network , iPOP

2011.[16] http://www.emulab.net/, Emulab Network Emulation Testbed[17] N. Gude et al., NOX: Towards an Operating System for networks ,

SIGCOMM 2008.[18] http://openvswitch.org/, An Open Virtual Switch.[19] S. Sharma et al., Enabling Fast Failure Recovery in Openow Networks ,

DRCN 2011.[20] http://www.fp7-sparc.eu/, Split Architecture Carrier Grade Networks.[21] http://www.fp7-ofelia.eu/, OpenFlow in Europe, Linking Infrastructure

and Applications.