00898740

8/13/2019 00898740

1/14

8/13/2019 00898740

2/14

2604 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATION, VOL. 18, NO. 12, DECEMBER 2000

Fig. 1. Proposed multilayer QoS routing scheme.

The rest of this paper is organized as follows. Section IIdescribes the hierarchical multilayer QoS routing system withdynamic management of the SLA, and discusses several of the components required. Section III describes the prototypesystem, and Section IV presents and discusses the results of the performance evaluation. Section V concludes this paper bysummarizing it briefly.

II. HIERARCHICAL MULTILAYER QOS ROUTING SYSTEM

A. Overview

Fig. 1 depicts theproposed multilayer QoS routing systemforan IP core network interconnecting several ISPs. It lets them usethe core IP network as a traditional best-effort IP routing net-work, and also lets them use it as a QoS-guaranteed network by

establishing virtual leased lines (VLLs) over thenetwork. We as-sume that these VLLs can be created dynamically and that theirQoS parameters (e.g., bandwidth and delay) can be changed ondemand according to the actual or anticipated traffic load. Themain reason for introducing the dynamic VLLs is to ensure theend-to-end QoS, such as packet delay and loss, specified in SLAbetween the ISP and its users. This SLA is the user-SLA, and theSLA between the ISP and the core IP network is the ISP-SLA inFig. 1. While the user-SLA for each DiffServ class is essentiallystatic, the ISP-SLA is dynamic and is affected by the number of active users of the specified class. By making use of such dy-namicVLLservices,ISPscanreplacetheexistingleasedlineser-viceswith VLLs, andcan significantly reducethe totalcost of the

leased line.To support such services, we propose a hierarchical mul-

tilayer QoS routing system consisting of 1) dynamic SLAmanagement of ISP-SLA (policy server), 2) hierarchical QoSreservation for aggregated IP flows (hierarchical CR-LDPsignaling), and 3) hierarchical QoS routing (hierarchicalQoS-enabled OSPF (QOSPF)). We assume that the IP core net-work is administrated by a single carrier as a single autonomoussystem (AS), in which the QOSPF is used for intradomain pathcomputation, and the border gateway protocol (BGP) [ 16] isused for interdomain path computation.

The overall procedures of the proposed system are illustratedin Fig. 1. Basically, the BGP runs on each ingress router to es-

tablish external BGP sessions (e-BGP), based on the network administrators policy, for exchanging interdomain reachabilityinformation with surrounding ISP routers. Internal BGP ses-sions (i-BGP), in which ingress and egress routers pass thisreachability information through the core IP network, are alsoestablished.Thus, each ingress routercan know theegress routeraddress associated with destination reachable addresses.

The policy server manages each ISP-SLA, and the negotiatedSLA parameters (IP src-dst addresses, bandwidth, etc.) aredownloaded to the policy agent on the ingress router of theIP core network. The policy agent conveys the parameters tothe CR-LDP module [ 23] so that it can establish the MPLSlabel-switched path (LSP) from the ingress to egress router andreserve resources along the path. The address of this egressrouter is resolved by using the BGP reachability information,described in preceding paragraph. Although TE-RSVP sig-naling [ 23] could be used on this system, we use CR-LDP,because its implementation is easier while having the samefunctionality as TE-RSVP, and because it uses hard-state-basedLSP maintenance, which requires significantly less control

packets than does soft-state-based TE-RSVP.Path computation for the LSP is performed by the QOSPF

module. Integrated IS-IS [ 18] could also be used as an alter-native routing protocol, but we used OSPF for our system be-cause it is more widely used in the IP network environment.Theproposedextensionsof QOSPF areQoS parameteraggrega-tion, hierarchical QoS link state advertisement (LSA), and hier-archical QoS path computation. The proposed path computationuses a combination of multiple static-link-metric-based pre-computations and on-demand computations for reducing botha blocking probability and an average delay of finding a QoSavailable path. These static multiple precomputed paths are alsoused for QoS aggregation for reducing the computational over-

head of updating QoS summary LSAs.The proposed extension of CR-LDP is hierarchical source

routing with crankback capability. This extension can be veryuseful for finding an alternative route on demand if the pathspecified by the QOSPF is unavailable due to the mismatch of local and global QoS information. The QOSPF link state infor-mationstoredat each routeris not so accurate actually because itis not so updated frequently and because the propagation delayfor link state update messages is not negligible [ 12], [13]. Inhierarchical networks, the aggregation of QoS information canalso cause such mismatches more often.

B. Dynamic SLA Management: Policy Server

When an IP core network is given a new set of service featuresor functions, it is important that the changes on the user side(i.e., these on the ISP side) should be minimized. The routersof the ISP should be used just as they were before, even if thereare many changes in the core network. In this context, it seemsto be good to take a centralized approach in which a centralpolicy server provides a user interface, which can exchange thedynamic SLA negotiation parameters with a secured commu-nication channel, and in which it performs a centralized QoSpath computation and controls the routers inside the IP core net-work. As pointed out earlier, however, this approach will leadto performance bottleneck problems when the network size be-
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-

8/13/2019 00898740

3/14

8/13/2019 00898740

4/14


into the signaling message the ID of the blocked link or node.The ingress border router receiving the signaling message withthe crankback TLV computes an alternative path within the areaby temporarily pruning the unavailable link or node from itstopology database. If no new path can be found at an ABR (or if it is blocked again), however, the signaling message is returnedto the ingress router of the previous area in order to trigger an

alternative routing there.

D. QoS-Enabled OSPF Routing: QOSPF

QoS-enabled OSPF (QOSPF) protocol is being standardizedin IETF by extending the OSPF protocol to support QoS link state parameters. It is designed to collect and maintain the QoStopology map used for QoS path computation. This subsectionbriefly describes previous QOSPF approaches [ 10], [26] andthen explainsour proposed extension of link state advertisement(LSA), resource LSA, and discusses the proposed QoS pathcomputation scheme using this extension. These proposals areexplained for both flat and hierarchical networks.

1) Existing Problems of Previous QOSPF Approaches: Twoencoding schemes for OSPF QoS extensions have been pro-posed: type-of-service(TOS)-metrics-based encoding [ 10] andopaque-LSA (link state advertisement) encoding [ 19]. Althoughthe TOS-metrics-based encoding can support backward compat-ibility, it restricts the encoding of extended parameters and doesnot have sufficient flexibility to accommodate future possibleextensions (e.g., other QoS extensions and traffic engineeringextensions) [ 20], [26]. We therefore chose the opaque LSA en-coding for the proposed QoS LSA, resource LSA [ 21].

Recent papers on QoS path computation [ 9], [10] have pro-posed a QoS-based precomputation scheme: the precomputa-

tion module computes new optimized (or load-balanced) pathsand updates the routing table whenever it receives new QoSlink state information. To reduce the number of LSAs, theyhave also proposed to use a triggering threshold and hold-downtimers. As the intervals between LSAs becomes longer, how-ever, the advertised QoS information becomes more inaccu-rate and thus may not be useful for QoS path computation.As we already mentioned, we therefore use a combination of multiple static-link-metric-based precomputations and on-de-mand computations to address this problem. The combinationof the two computations can significantly reduce the computa-tional load of the QOSPF module while providing a small LSPblocking probability.

2) LSA Extension for Basic QoS Routing: The proposedresource LSA represents the maximum link bandwidth( ), the reserved link bandwidth ( ), the avail-able link bandwidth ( ), and the link metric ( )corresponding to each physical link. Although other parame-ters, such as delay and delay jitter, could also be represented,since the current DiffServ-based traffic management does notinclude delay and delay jitter parameters, in the work describedin this paper, we only use these link metrics and these band-width parameters. The path computation is basically to finda bandwidth-cost constrained path [ 5], [8], which representsa minimized cost path satisfying the requested bandwidth.The can be either a physical link bandwidth or

Fig. 5. QOSPF: QoS parameter aggregation.

the bandwidth allocated for a logical channel such as anATM virtual channel connection (VCC), The is abandwidth reserved by the CR-LDP signaling for establishinga QoS-guaranteed LSP, and the is the current residualbandwidth calculated by subtracting from the thebandwidth of the flows currently active on the link.

3) LSA Extension for Hierarchical QoS Routing: Fig. 5shows the proposed QoS parameter aggregation behavior in ahierarchical network. The proposed resource LSA also supports

, , , and for a logical linkspanning across areas toward the IP destination. We define alogical link as multiple candidate paths from an ABR to thedestination IP summary addresses, which are conventionally

carried by standard summary LSA and AS external LSA [ 17].These multiple candidate paths are precomputed initially bythe sequential Dijkstra algorithm, for example, using a staticadministrative link metric as a cost function.

As shown in Fig. 5, for example, there are in the egress area(C.*) two ABRs: ABR2 and ABR3. Each ABR initially calcu-lates multiple static precomputed paths from itself to the egressrouter. Each ABR monitors the maximum bottleneck bandwidthamong these precomputed paths (i.e., for ABR2,

for ABR3) whenever a new resource LSA is re-ceived, and if a change is larger than a specified threshold value(i.e., if the change is significant), the ABR creates a summa-rized resource LSA that specifies destination IP summary ad-

dresses, the maximum bandwidth available in the precomputedpaths, and theaccumulated link metric costs for thechosen path.It then advertises a new resource LSA to backbone area (B.*).The processing load for this is small, since the maximum bottle-neck bandwidth of only dedicated multiple precomputed pathsis monitored.

When ABR1 receives these resource LSAs from ABR2 andABR3, it also verifies whether or not the change of maximumbottleneck bandwidth (calculated as ( ,

), ( , ) ) of the pre-computed paths (from itself to egress router through eitherABR2 or ABR3) is significant. If it is, then ABR1 advertisesan updated resource LSA to the ingress area (A.*). This QoS
http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-http://-/?-

8/13/2019 00898740

5/14

8/13/2019 00898740

6/14


Fig. 7. Prototype system.

Fig. 8. Policy server: Topology/LSP window.

in the Unix kernel. We modified the kernel to provide MPLSlabel-switching and also modified the ALTQ package to provideDiffServ-based absolute/differentiated packet scheduling forhandling MPLS-labeled packets as well as pure IP packets. Thismodification enables the FreeBSD machine to serve as either aningress, intermediate, or egress MPLS label-switching router(LSR). The implementedLSRis a frame-based LSR[ 24], whichspecifiesthe label in a shim header[ 24] infrontof an IPpacket.

Fig. 9. Topologies evaluated in the prototype system.

Fig. 10. CR-LDP signaling: average processing delay for each router.

IV. PERFORMANCE EVALUATION

For performance evaluation, we used three environments:1) the prototype system, 2) a virtual network simulator, and3) a QoS routing algorithm simulator. The prototype systemwas used for measurements of the basic performance (i.e.,processing delay) of the CR-LDP signaling. Since the numbersof routers and links are limited in the prototype system, webuilt a virtual network simulator that can emulate a large-scaleMPLS network. The virtual network simulator can be used asan integrated simulator that can integrate the simulator withthe real prototype system in a virtual large network and thatcan generate a large number of routing and signaling controlpackets into the prototype system according to the emulated

topology. The scalability performance (e.g., CPU load and pathcomputation delay as the number of routers increases) of theimplemented software was evaluated by controlling the numberof nodes and topologies. Fig. 12 shows the structure of thisnetwork simulator. Using the measured results as basic param-eters, we further evaluate the performance of the proposed QoSrouting algorithms more extensively with our specialized QoSrouting algorithm simulator.

A. Performance Measured with the Prototype System and Virtual Network Simulator

1) Network Topology for Measurement: Fig. 9(a) and (b)shows two kinds of physical topologies used for measurement:
http://-/?-http://-/?-http://-/?-http://-/?-

8/13/2019 00898740

7/14

IWATA AND FUJITA: A HIERARCHICAL MULTILAYER QoS ROUTING SYSTEM WITH DYNAMIC SLA MANAGEMENT 2609

Fig. 11. CR-LDP signaling: end-to-end LSP setup delay with and withoutcrankbacks.

Fig. 12. Virtual network simulator: the connectivity between real router andsimulator.

a straight-line topology and a concatenated meshed-block topology. Both topologies are used for measuring the perfor-mance of CR-LDP signaling (we define the processing delayat each hop as a performance unit) for a normal routing and acrankback routing. This performance unit is used for a basicparameter on the QoS routing algorithm simulator. The link bandwidth for both topologies was 155 Mb/s.

Fig. 13 shows a large network topology created on the virtualnetwork simulator for measurement of QOSPF performance.The size of the network is controlled by parameter rangingfrom 2 to 6, where the total number of routers is and thetotal n umber o f links i s . A lthough t he n etwork

size, , evaluated in this section is a medium size for anISP network [ 7], we can extrapolate its performance behavior tolarger networks as well. In order to compare the performance of our algorithm with flat (nonhierarchical) routing, we comparethe routing performance on the aggregated five areas with therouting performance on a single flat area, consisting of the samefive areas. We locate the real prototype system at the location(A) or (B), to which the virtual network simulator is connectedfor emulating all the other nodes than the node located in (A) or(B), respectively. The performance is measured on this proto-type system connected with the simulator. The location (A) and(B) is for evaluating the load of the standard router and the areaborder router (ABR), respectively. The performance of the two

Fig. 13. Evaluated topologies on the virtual network simulator.

is expected to be different, since ABR has to handle multiplelink state databases from different areas, and looks interestingto examine, as discussed later.

2) Processing Delay of CR-LDP Signaling: Processingdelay of the CR-LDP signaling was measured using thetopologies shown in Fig. 9. The average processing delay of each MPLS hop (ingress, intermediate, and egress) for thelabel-request and the label-mapping processing was measuredusing the topology in Fig. 9(a), the end-to-end delay forestablishing the LSP and allocating the resources along theLSP in a two-level hierarchy was measured using the topology

shown in Fig. 9(b).As shown in Fig. 10, most of the processing delay in the labelrequest message occurs at the ingress router. This delay consistsof (i) query-response delay for QoS path computation betweentheCR-LDP and the QOSPF module, (ii) thedelay for reservingthe ingress selector [ 24], label, and bandwidth resources; and(iii) other transmission delay of signaling packets. Element (i)depends on the hop counts due to the QoS path computationalgorithm (all links of the path has to pass the feasibility check,which takes more time for longer paths) and is the largest partof the processing delay.

Fig. 11 shows the average end-to-end CR-LDP processingdelay measured, with and without crankback routing for a

two-level hierarchy, where each of ingress, backbone, andegress areas have , , and blocked units inside (plotted byalong the horizontal axis), and for the flat network

with routers. Here, the blocked unit is defined asshown in Fig. 9(b). As the size of the network grows (i.e.,

to ), in the case of having a crankback routingonce or twice in any area, there is less delay in the hierarchicalnetwork than there is in the flat network. Since the crankback routing requires the time to return to the ingress router to findanother path for rerouting, the distance of cranking back affectsthe end-to-end delay performance.

3) CPU Load and Path Computation Delay of QOSPF Routing: Performance of QOSPF routing is measured on

8/13/2019 00898740

8/14


Fig. 14. QOSPF: CPU load against number of routers (flooding interval: 5 s).

Fig. 15. QOSPF: CPU load versus average flooding interval ( , 176routers).

Fig. 13 by connecting the real QOSPF router with the virtualnetwork simulator.

a) CPU Load: The CPU load required, under the5-second flooding interval, for LSA processing and path pre-computation is plotted in Fig. 14, as a function of the numberof routers. The CPU load, for 176 routers ( ), is plotted inFig. 15 for flooding intervals from 5 to 20 seconds. Both loadsare evaluated on flat and hierarchical networks of various sizes( 6) and with two different precomputation schemes; the

QoS-based precomputation [ 9], [10] (Case 1) and the proposedstatic-link-metric-based precomputation (Case 2). Location(A) and (B) in each figure means the position in which the realrouter is located for the performance measurement.

As shown in Fig. 14, the CPU load in a hierarchical network can, with either precomputation method, be kept within aboutone-third of the load in a flat network. This 3 : 1 ratio is almostconstant regardless the number of routers. The load in the loca-tion (A) is smaller than that in the location (B), due to the factthe router in the location (B) has to update routing informationfor two directly connected areas, Area 0 and Area 2, while therouter in the location (A) maintains only Area 2 information.As shown in Fig. 15, the CPU load in a hierarchical network

can also be kept to about one-third to one-half of the load ina flat network by using either precomputation method. As theflooding interval becomes smaller, the difference of this ratiobecomes much more significant.

Interesting results in both figures are that the CPU load due toQoS-based precomputation (Case 1) is greater than that due tothe proposed precomputation (Case 2). This difference becomes

more significant as either the number of routers becomes larger,or as theflooding interval becomes smaller. The QoS-based pre-computation requires a lot of CPU cycles because it calculatesthe best available QoS path whenever new LSAs are received,whereas the proposed precomputation does not require any newcalculation unless the network topology or the link metrics arechanged.

b) Path Computation Delay: Fig. 16(a) shows the pathcomputation delay of the QOSPF in the flat and the hierarchicalnetworks (176 routers, ), for precomputation and foron-demand computation, plotted against the LSP path lengthfrom the ingress router to the egress router. Fig. 16(b) and (c)shows the corresponding accumulated path computation delayalong the LSP and its improvement due to the use of an areascope on-demand computation (described later in this subsec-tion).

As shown in Fig. 16(a), since the precomputation incurs pro-cessing delay only for checking the QoS availability along theprecomputed path, the delay in either the flat or hierarchical net-work is small and almost independent of LSP length. The delaydue to on-demand computation, on the other hand, is quite largeand depends on the LSP length significantly. As expected, com-paring the performance in the flat network [flat:location (A)]and the hierarchical network [hierarchy:location (A)], the delayin the flat network keeps growing as the LSP length grows, andthe delay in the hierarchical network grows up to five hops (i.e.,a distance between ingress and ABR router) and remains con-stant afterwards. This is because on-demand computation of thehierarchical network is only performed within the local area.

However, measuring another delay at the location (B) in thehierarchical network [hierarchy:location (B)], we observed anunexpected result, where its performance is four times as muchas that at the location (A), and is also worse than that in eventhe flat network with LSP length of 12 hops. The router locatedat (B) is ABR, and it maintains multiple area topological infor-mation, Area 0 and Area 2, which is two times as much as thatat the location (A), and we find this causes a processing bot-tleneck. We then evaluate the average accumulated QoS path

computation delay on ingress and two intermediate ABRs, andplot the result in Fig. 16(b), where precomputation*M + on-de-mand computation*N ( ) means a combination of M times precomputation and N times on-demand computation.Because of the on-demand computation processing bottleneck at intermediate ABRs, the average processing delay of on-de-mand*3, which uses on-demand computation on every router,is even worse than that of the flat network.

To solve this problem, we propose an area scope on-demand(AS-OD) computation method, which chooses candidates forthe transit areas and performs on-demand computation only forthe reduced transit areas to which candidate ABRs are attached.The candidates for transit areas toward destinations are stored

8/13/2019 00898740

9/14


(a)

(b)

(c)

Fig. 16. QOSPF: QoS path computation delay measured with different QoSpath computation schemes. (a) Path computation delay versus LSP path length( , 176 routers). (b) Accumulated path computation delay versus numberof routers. (c) Improved path computation delay with the AS-OD scheme.

on an area scope table, which can help to improve the on-de-mand computation delay. Since ABRs are normally attachedto multiple local areas, reducing (or scoping) the number of areas which on-demand computation is performed can improvethe performance significantly. The improved delay performancebased on the AS-OD scheme is shown in Fig. 18(c). Even when

Fig. 17. Flat topologies evaluated in QoS routing algorithm simulations.

Fig. 18. Hierarchical topologies evaluated in QoS routing algorithmsimulations ( , 176 routers).

on-demand computation is used three times at the ingress andintermediate ABRs (plotted by on-demand*3), it can be kept toa one-third delay of that of the flat network. Although we canreduce the on-demand computation delay by using the AS-ODscheme, theprecomputationschemeincurs much less delay thanthe on-demand computation. It is therefore important that thenumber of on-demand computations be reduced by using mul-tiple precomputation tables.

B. Performance Evaluated Using the QoS Routing AlgorithmSimulator

The previous section mentioned the basic performance of theproposed mechanisms. Using these results, we have evaluatedthe LSP blocking probability and the average LSP setup delay,for different QoS path computation methods and different net-work topologies, by using the QoS routing algorithm simulator.

1) Simulation Model: Three kinds of flat networks evalu-ated are shown in Fig. 17; (a) square mesh (36 routers, ),(b) full mesh (15 routers), and (c) typical ISP topologies [ 7] (19routers), and the hierarchical network (176 routers) is shown inFig. 18. We simulated two types of traffic, best effort (BE) andexpedited forwarding (EF) classes, and we assumed a uniformdistribution of the load between the two classes: each traffic type
http://-/?-http://-/?-

8/13/2019 00898740

10/14


consisted of one-half of the total traffic flow. The traffic is ran-domly sent among the routers. Both the LSP holding time andbandwidth requirements for each LSP were simulated as expo-nentially distributedrandom values. Thisarrangement generatesLSPs of longer duration less frequently than it does LSPs of shorter duration. LSPs requiring less bandwidth occur more fre-quently than those requiring morebandwidth. In this simulation,

the average bandwidth requirement was 25 Mb/s and the av-erage LSP holding times was 5 minutes. LSP interarrival timeswere simulated as exponentially distributed random values. Bychoosing the appropriate average interarrival times, we simu-lated different network loads or link utilizations. All LSPs wereclassified into 11 bandwidth slots ranging from 05 Mb/s to9095 Mb/s, and the aggregated blocking probability for eachrange is plotted in the figures in Sections IV-B-2 and IV-B-3.

2) Performance of Different QoS Path Computations: Weevaluated different QoS path computation schemes in theflat networks shown in Fig. 17: i) one precomputed path(1-PRE), ii) one precomputed path and one on-demand com-putation (1-PRE+1-OD), iii) four sets of precomputed paths(4-PRE(XX), XX: two different selection algorithms explainedbelow), and iv) four sets of precomputed paths and one on-de-mand computation (4-PRE(XX)+1-OD). The four routes of iii)and iv) are selected in order to produce diverse routes. Evenif four diverse routes are not found, multiple routes (less thanfour) are also used with the same path selection mechanisms.The reason of choosing four as multiple precomputed paths isthat the neighbor link connectivity of the current typical ISP [ 7]is around three to four, and it is usually difficult to obtain morethan four diverse routes. Note that on-demand computationincludes the case of crankback routing.

For choosing four precomputed paths, we use either a

heuristic algorithm (XX=HE), or a node-disjoint-path routingalgorithm (XX=ND), and compare the performance of eachalgorithm. If the network connectivity is dense enough to havemultiple disjoint paths, ND algorithm is expected to performwell. However, in order to adapt the network, where there issparse connectivity, not to have enough disjoint paths, the HEalgorithm is expected to perform well to find multiple paths thatcan be well load-balanced. The heuristic algorithm performsthe following steps: 1) the first route is the minimum link metricpath chosen by the Dijkstra algorithm; and 2)4) the second,third, and fourth route is the minimum link metric path on thereduced network, where the first link of the first route is pruned,where the second link of the first route is pruned, and where

the first link of the first route and the second link of the secondroutes are pruned, respectively. These steps are sequentiallyperformed until the total number of candidate paths reachesfour. If multiple paths with an equivalent cost are found in eachstep, those paths are given precedence over the following steps.The node-disjoint-path algorithm, on the other hand, performsthe following steps: 1) the first route is the minimum link metric path; and 2) the th route is the minimum link metricpath on the reduced network, where the router and link alongthe 1st route, 2nd route, th route are pruned exceptsource and destination router. Examples of both computationsare shown in Fig. 19. Thus we compared the performance of sixdifferent schemes with three different flat network topologies.

Fig. 19. QOSPF: example of four precomputed paths.

(a)

(b)

(c)

Fig. 20. QOSPF: average LSP blocking probabilities for flat networks withdifferentQoS path computation schemes. (a)Flat squaremesh network( ,

2 , ave. link load 0.22). (b) Flat full mesh network (15 routers, ave. link load 0.17). (c) Typical flat ISP network (19 routers, ave. link load 0.24).

Fig. 20 shows the LSP blocking probability for the square-mesh, full-mesh, and typical ISP topologies. The blocking prob-
http://-/?-http://-/?-

8/13/2019 00898740

11/14


ability of each scheme increases as the requested bandwidthincreases. One precomputation scheme (1-PRE) has a blockingprobability quite a bit worse than the other schemes do. Thefour sets of precomputation schemes [4-PRE(HE/ND)] resultin lower blocking probabilities than 1-PRE does. The otherthree schemes with on-demand computation, 1-PRE+1-ODand 4-PRE(HE/ND)+1-OD, can significantly improve the per-

formance. Although they have almost the same improvement,4-PRE(HE/ND)+1-OD can reduce the number of on-demandcomputations more than 1-PRE+1-OD does. In Fig. 20(a),for example, when the requested bandwidth is 70 Mb/s,4-PRE(ND)+1-OD can reduce the on-demand computationto as little as one-third of 1-PRE+1-OD scheme. Comparingthe performances of 4-PRE(HE) and 4-PRE(ND), we foundthat their performances depend on the network topology. Thenode-disjoint-path routing algorithm, 4-PRE(ND), performswell in dense connectivity networks (i.e., square-mesh andfull-mesh topologies), in which there are enough candidatepaths. The heuristic algorithm, 4-PRE(HE), on the other hand,works well in sparse connectivity networks (i.e., ISP topology).

Fig. 21 shows the average QoS path computation delayof each scheme, for square-mesh, full-mesh, and typicalISP topologies. The path computation delay uses the actualmeasured delay in Fig. 16(a) and the LSP blocking proba-bility in Fig. 20(a)(c). As each part of this figure shows,delays are smallest for the precomputation methods, 1-PREand 4-PRE(HE/ND). As the requested bandwidth increases,on-demand approaches, 4-PRE(HE/ND)+1-OD, increase theaverage delay, which, however, can be kept much lower than1-PRE+1-OD scheme. Therefore, the scheme combiningmultiple precomputed paths and on-demand computation canprovide the most beneficial solution to reduce both the LSPblocking probability and the processing delay.

Fig. 22 shows another interesting LSP blocking probabilityresult obtained when simulating the full-mesh network, whenthe network load was three times higher than in the case of Fig. 20(b). For requested bandwidth below 65 Mb/s, the similarcall blocking behavior can be seen in this figure. For bandwidthabove 65 Mb/s, the 1-PRE scheme performs the best, and the4-PRE(HE/ND) performs the next best, and other three on-de-mand schemes performs the worst. This is because choosingthe shortest path for larger requested bandwidth is the best wayto save the network resources in such a heavy load situation.Fig. 23 depicts the throughput performance of Fig. 22, whichcan be calculated by multiplying the average number of suc-

ceeded calls with their average requested bandwidth. As shownin this figure, 1-PRE+1-OD and 4-PRE(HE/ND)+1-OD havethe highest throughput at lower requested bandwidth, and havethe lowest throughput at higher bandwidth. The performance of the 1-PRE exhibits the reverse behavior.

Fig. 24 shows the average total throughput of Fig. 23 for eachpath computation scheme. It shows that the total throughput isalmost the same for each scheme, although the 1-PRE+1-ODand 4-PRE(HE/ND)+1-OD schemes provide slightly higherthroughput. Analyzing the average link utilization of eachscheme, however, we can observe an interesting situation.The average link utilization is shown in Fig. 25. It shows that1-PRE+1-OD and 4-PRE(HE/ND)+1-OD spend a higher link

(a)

(b)

(c)

Fig. 21. QOSPF: average computation delay for flat networks with differentQoS path computation schemes. (a) Flat square mesh network ( , 2 ,ave. link load 0.22). (b) Flat full mesh network (15 routers, ave. link load 0.17). (c) Typical flat ISP network (19 routers, ave. link load 0.24).

utilization, whereas the average total throughput is the sameas other schemes. 4-PRE(HE/ND) scheme has reasonablylower link utilization. This is because on-demand computationpossibly finds a longer hop path that would then increaseaverage link utilization. Thus, if one of the optimization goalsis to keep the link utilization relatively small under the heavyload condition (while keeping the call blocking probabilitysmall), we should choose the multiple precomputation methods(4-PRE(HE/ND)) for good load balancing.

3) Performance of QoS Aggregation in a Hierarchical Net-work: We evaluated the performance of the proposed QoS ag-gregation schemes in a hierarchical network in Fig. 18(a) andcompared it to the performance in a flat network in Fig. 18(b).

8/13/2019 00898740

12/14


Fig. 22. QOSPF: average LSP blocking probabilities for flat full-meshnetwork under high load.

Fig. 23. QOSPF: average throughput against requested bandwidth for the flatfull-mesh network under high load.

Fig. 24. QOSPF: average total throughput for various QoS path computationalgorithms used in the flat full-mesh network under high load.

Fig. 26(a) and (b) shows the LSP blocking probability for eachcomputation scheme as a function of requested bandwidth in ahierarchical network (depicted by AGG:) and the flat network (depicted by FLAT:). This probability only shows the blockingprobability of inter-area traffic among Area 1, 2, 3, and 4 acrossArea 0, by removing the intra-area LSP blocking probability ineach area.

Fig. 26(a) shows the call blocking probability for the 1-PREschemes and 4-PRE (HE/ND) schemes. Both 1-PRE schemesresult in high blocking probabilities. The 4-PRE schemes re-sult in lower blocking probabilities, because they can use di-verse routes for path selection. Comparing the performance of each 1-PRE scheme and each 4-PRE scheme between the hier-

Fig. 25. QOSPF: average link utilization for various QoS path computationalgorithms used in the flat full-mesh network under high load.

(a)

(b)

(c)

Fig. 26. QOSPF: performance comparison between hierarchical network andflat network ( , ave. link load 0.10). (a) LSP blocking probabilityfor precomputation schemes. (b) LSP blocking probability for on-demandcomputation schemes. (c) Average path computation delay for each scheme.

archical network and the flat network, we can observe an inter-esting result: they work slightly better in the hierarchical net-work. It is a bit counterintuitive, since the QoS aggregation

8/13/2019 00898740

13/14


scheme (AGG:) only advertises the aggregated internal network topology and bandwidth information to the outside and the flatrouting scheme (FLAT:) should provide more exact QoS infor-mation to help to reduce the blocking probability.

The reason for this behavior has been analyzed as follows. Inthe hierarchical network, each area has two ABRs toward desti-nation routers. Within each area, each router hasone or four pre-

computed paths to all intra-area routers, including ABRs. Wheneach router performs inter-area path computation in the hierar-chicalnetwork, it hastwo or eight precomputation paths throughtwo ABRs. Thus, doubling the precomputed paths helps to im-prove the call blocking probability. If we increase number of ABRs, we can expect the more performance improvement inthe hierarchical network.

Fig. 26(b) shows the call blocking probability for the1-PRE+OD schemes and 4-PRE(HE/ND)+1-OD schemes.The performances of all on-demand related schemes arealmost the same, except that the AGG:1-PRE+1-OD andthe AGG:4-PRE(ND)+1-OD schemes have slightly higherblocking probabilities than the others do. This is also an unex-

pected result where the performance of the hierarchical routingis as good as that of the flat routing. This is mainly becausedoubling the number of precomputed paths and using hierar-chical crankback routing can increase the probability of findinga path even when the QoS routing information is inaccurate.Since the crankback routing can get accurate QoS informationon-demand and feed it back to the ingress router for a newpath computation, it can decrease the blocking probabilitysignificantly, even when the topology and QoS information areavailable in accurate. Examining the performance differenceof AGG:4-PRE(HE)+1-OD and AGG:4-PRE(ND)+1-OD indetail, the former one has better performance than the latterone in this hierarchical network topology. This is because the

former one has more precomputed paths than the latter one,causing the performance difference.

Fig. 26(c) shows the average QoS path computation delayfor each computation scheme. Since inter-area traffic across thebackbone area requires three separate path computations, thedelay result is accumulated along thehops. As this figure shows,the average delay for each scheme is lower in the hierarchicalnetwork than in the flat network. This is because, as the path hoplengthsbecome longer,the on-demand pathcomputationdelay intheflatnetworkbecomesmuch larger thanthat in thehierarchicalnetwork,even though thecallblockingprobabilityis thesame.

V. CONCLUSION

This paper proposed a hierarchical multilayer QoS routingsystem with dynamic SLA management for large-scale IP net-works and introduced three augmented components: a policyserver, hierarchical CR-LDP signaling, and hierarchical QOSPFrouting. We implemented theprototypesystemanda virtual net-work simulator so that we could evaluate the performance of thesystem and performance bottlenecks. We also developed a sim-ulator for evaluating the performance of the routing algorithmitself. We found that the proposed hierarchical QoS routing andsignaling were proved to be a scalable solution delivering betterperformance than previously proposed approaches. This solu-tion reduces the LSP blocking probability and significantly re-

duces the control overhead. The applicability and the motiva-tions of using the policy server is also explained and proved aquite useful feature for the proposed QoS-enabled network. Wealso plan to investigate the proposed system and its path com-putation algorithm more intensively in various kinds of network topologies and with various QoS service scenarios.

ACKNOWLEDGMENTThe authors would like to thank R. Izmailov at NEC USA,

Inc. for providing many valuable comments and suggestions forimproving this paper.

REFERENCES[1] X. Xiao et al. , Internet QoS: A big picture, IEEE Network Mag. , pp.

818, Mar. 1999.[2] S. Blake et al. , An architecture for differentiated services, IETF RFC

2475, Dec. 1998.[3] S. Chen et al. , An overview of quality of service routing of next-gen-

eration high speed networks: Problems and solutions, IEEE Network Mag. , pp. 6479, Nov. 1998.

[4] Z. Wang, Quality-of-service routing for supporting multimedia appli-

cations, IEEE J. Select. Areas Commun. , vol. 14, Sept. 1996.[5] A. Iwata et al. , ATM routing algorithms with multiple QoS require-

ments for multimedia Internetworking, IEICE Trans. Commun. , vol.E79-B, no. 8, pp. 9991007, Aug. 1996.

[6] Q. Ma et al. , Quality-of-service routing with performance guarantees,in Proc. IFIP IWQoS Workshop , May 1997.

[7] , On pathselection for traffic withbandwidthguarantees, in Proc. IEEE Int. Conf. Network Protocols , 1997, pp. 191202.

[8] S. Chen et al. , On finding multi-constrained paths, in Proc. IEEE ICC98 , June 1998.

[9] G. Apostolopoulos et al. , Quality of service based routing: A perfor-mance perspective, in Proc. ACM SIGCOMM , 1998.

[10] G. Apostolopoulos et al. , Implementation and performance measure-ments of QoS routing extensions to OSPF, in Proc. Infocom99 , Apr.1999, pp. 680688.

[11] ATM Forum, Private network network interface (PNNI), v1.0 specifi-cation, May 1996.

[12] R. Guerin et al. , QoS routing in networks with inaccurate information:Theory and algorithms, IEEE/ACM Trans. Networking , vol. 7, no. 3,pp. 350364, June 1999.

[13] A. Orda, Routing with end-to-end QoS guarantees in broadband net-works, IEEE/ACM Trans. Networking , vol. 7, no. 3, pp. 365374, June1999.

[14] A. Iwata et al. , QoS aggregation algorithms in hierarchical ATM net-works, in Proc. IEEE ICC98 , vol. 1, June 1998, pp. 243248.

[15] S. Nelakuditi et al. , Quality-of-service routing without global infor-mation exchange, in Proc. IEEE IWQoS Workshop99 , Mar. 1999, pp.129131.

[16] B. Halabi, Internet Routing Architectures : Cisco Press, 1997.[17] J. Moy, OSPF: Anatomy of an Internet Routing Protocol . Reading,

MA: Addison-Wesley, 1998.[18] ISO, Intermediate system to intermediate system intra-domain routing

exchange protocol for use in conjunction with the protocol for providingthe connectionless-mode network service (ISO 8473), ISO DP 10589,

Feb. 1990.[19] R. Colton, The OSPF Opaque LSA option, IETF RFC, RFC 2370,July 1998.

[20] D. Katz et al. , Traffic engineering extensions to OSPF, IETF InternetDraft, draft-katz-yeung-ospf-traffic-00.txt, Apr. 1999.

[21] N. Fujita and A. Iwata, Traffic engineering extensions to OSPF sum-mary LSA, IETF Internet Draft, draft-fujita-ospf-te-summary-00.txt,Mar. 2000.

[22] D. Awduche, MPLS and traffic engineering in IP networks, IEEE Commun. Mag. , pp. 4247, Dec. 1999.

[23] A. Ghanwani et al. , Traffic engineering standards in IP networks usingMPLS, IEEE Commun. Mag. , pp. 4953, Dec. 1999.

[24] G. Swallow, MPLSadvantages for traffic engineering, IEEE Commun. Mag. , pp. 5457, Dec. 1999.

[25] N. Fujita and A. Iwata, Crankback routing extensions for CR-LDP,IETF Internet Draft, draft-fujita-mpls-crldp-crankback-00.txt, Mar.2000.

8/13/2019 00898740

14/14

Documents

00898740