L5a 731 WFQ Slides (Updated) 2pp

1

Weighted Fair Queuing (WFQ)

Prof. T. Szymanski McMaster University

Prof. T Szymanski 4DN4 - WFQ, pg 2

Overview

! Internet Flows & Fairness ! Fluid-Flow Generalized Processor Sharing (GPS) ! Packet-Flow Weighted Fair Queuing (WFQ)

! Definition ! Computing Finishing Times ! Examples

! Jitter (Inter-Departure Time) Distribution ! Commercial Implementations ! Other Scheduling Methods


Internet Flow/Session

" Definition: A flow is a sequence of packets sent from one IP source to one IP destination [text, p594]

" the source and destination addresses are defined by an IP address and a port number

Example of the bandwidth requirement of a traffic flow. [1, Fig 7.52, p551]

Bandwidth


Flow of Packets [text, p539]

" An IP router = a node where IP packets arrive, are segmented into fixed-size cells, queued at the input queues, scheduled and transmitted through the switch, reassembled and buffered at the output queues, queued at the output fiber awaiting transmission, and eventually transmitted onto the outgoing fiber .

" An IP router will contain Input Queues (IQs) and Output Queues (OQs)

" A typical backbone router with 40 Gbps links will buffer typically up to 5 million IP packets per port

" Packets from competing traffic flows compete for buffer space and transmission bandwidth along the outgoing transmission line

" How does one define fairness?


Flow of Packets Through Routers

Consider the end-to-end QoS of a packet traversing N switches [1, Fig 7.40, p539]

Each packet traverses N queueing systems, and experiences delay and jitter at each queueing system.


Fairness

" Intuitively

" Competing flows should be guaranteed the bandwidth they have reserved (assuming they can reserve bandwidth)

" Any excess bandwidth should be equally shared amongst the flows.

" Protection against Starvation

" Fairness prevents Starvation or Deadlock, where flows are deprived of access to buffers/bandwidth [text, p540]

" Fair scheduling

" Automatically isolates heavy users, to avoid resource hogging

" Provides dynamic sharing of unused bandwidth allocations.


Requirements of Fair Scheduling

" An ideal scheduler: " is easy to implement " is totally fair in resource sharing " provides protection against starvation " provides guaranteed performance bounds " offers different flows different QoS; each flow given a

weight, ie a guaranteed fraction of resource usage

Interactive video

Packet voice

Email

r

Flows entering the same queue are given weights, w1, w2, and w3, and share the bandwidth r accordingly.

Q

w1

w2

w3


Problem with Round-Robin Service (pg 544, text)

" Suppose 4 flows arrive at an IP router. Why not service the packets in round-robin order ?

" Suppose packets in flow 1 are large, packets in flows 2 and 3 are small, and packets in flow 4 are very small.

" Servicing queues in round-robin order would give flow 1 much more service than the other flows, since it has much larger packets. Similarly, flow 4 would receive very little bandwidth.


Generalized Processor Sharing(GPS)

" GPS is an idealized Fluid Model, developed at MIT, which provides an upper bound on Fairness [1, p542] " Each flow has its own buffer. " Transmission bandwidth is divided among the flows

according to flow weights " Conceptually, packets in buffers like a fluid that is drained

continuously. " How it works (assume all flow weights are equal for now) :

" Assume initially that we service individual bits from each flow, in a round-robin order

" Given n flows in the system and transmission capacity = C bits/sec, then each flow is guaranteed transmission rate = (C/n) bits/sec

" A flow could experience a higher rate when the other flows are not using their allocation of bandwidth.


Fluid (Bit-by-Bit) Round-Robin Service in GPS

let R(t) denote the current round of service at time t; R(t) is also called Virtual Time. Virtual time increases quickly when few buffers are active. Virtual time increases slowly when many buffers are active.


" One round consists of a round-robin service cycle in which all n queues are offered service (1 bit of service each, for now).

" Therefore, the amount of real time elapsed per round = (number of active queues) * (time serving one bit from each active queue)

" nactive(t) queues = number of active (non-empty) queues at time t

" In round of service, all nactive(t) queues (i.e. non-empty queues) are serviced (1 bit for now).

" The Round Number R(t) = the number of rounds of service completed by the GPS round-robin server at the current time t

Definition of a GPS Round (text, Fig. 7.46, pg. 544)


Packet Finishing Time

" The Fluid-Based GPS system effectively computes the ideal finishing time of a packet in a perfectly fair system

" The finishing time of a packet is easily expressed in terms of rounds (Virtual-Time), rather than real time (i.e. a packet with 100 bits will finish service after 100 rounds have expired)

" In fluid-based GPS, packets are said to depart the system when they reach their finishing times => Packets with smallest finishing times are transmitted first.

" The GPS scheduler maintains several state variables: 1) The current round number R(t) or the current Virtual Time 2) The finishing time (in rounds) for every packet in the system,

3) The last finishing time (in rounds) of the last packet in every queue.


Finishing time: Empty Queue Case (text, pg 545)

" GPS Finishing times are expressed in rounds of service.

" Let F(f,k) denote the finishing time of packet k from flow f in a queue " Suppose packet k from flow f arrives at an empty queue at real time t

(f,k) and round (ie virtual time) R( t(f,k) ). " The Packet has length P(f,k) bits. Therefore, it will complete its

transmission after P(f,k) rounds of additional service (since each round services one bit)

" Assuming all flows request equal service: " The finishing time, expressed in rounds, is clearly:

F(f,k) = R( t(f,k) ) + P(f,k) (1)


Finishing time: NonEmpty Queue Case

" Suppose packet k from flow f arrives at a nonempty queue " It must wait for all the packets ahead of it (and in particular packet

k-1 which is directly ahead of it) to be serviced before it is serviced. " The finishing time of the packet ahead of it (packet k-1) in the queue

is F(f, k-1). " Assuming all flows request equal service:

" The current packet needs P(f,k) additional rounds of service, therefore the finishing time of this packet is clearly:

F(f,k) = F(f,k-1) + P(f,k) (2)


Finishing time: Empty and NonEmpty Queues

From the last slides we have:

" F(f,k) = R( t(f,k) )+ P(f,k) - Empty Queue Case (1) " F(f,k) = F(f,k-1) + P(f,k) - NonEmpty Queue Case (2)

Combining equations (1) and (2) we get

F(f,k) = max{F(f,k-1), R( t(f,k) )} + P(f,k)

This is all great, but how does it help us? We can now compute the ideal departure time of packets in a perfectly fair GPS

system, expressed in rounds, when each flow requests equal service. However, can we really build IP routers which service packets in round-robin order

at the bit-level (ie one bit of service per round) ? No, but we do not need to. We only need to know the ideal finishing time of each packet in a GPS system.

Also, what if flows request different levels of service?


Packet-oriented GPS == WFQ (pg 544) Let a Weighted Fair Queueing (WFQ) Server run in parallel with an ideal GPS server. The WFQ server only services whole IP packets (it doesnt split them into bits).

Given variable length IP packets, suppose the WFQ server will service entire packets in the same order as the perfectly fair service order of a fluid-flow GPS system, i.e., the next packet to be transmitted over a transmission line by the WFQ server is the packet with the smallest GPS Virtual Finishing time amongst all packets waiting for that transmission line

Call this system Packet-oriented GPS, to distinguish it from the Fluid-flow GPS. It is also called Weighted Fair Queueing. This scheme is implemented in CISCO routers.


Packet-by-Packet WFQ vs Fluid-Flow GPS

" Following slides will illustrate the differences between ideal fluid flow GPS and Packet-by-Packet Weighted-Fair-Queuing.

" Assume buffers 1 and 2 each have one L-bit packet to transmit at time t = 0, and that no subsequent packets arrive

" Assume the transmission capacity is C = L bits/second, then we can transmit 1 packet/sec .

" The idealized fluid-flow GPS system transmits both packets simultaneously at a rate = ! packet/sec and both packets complete at time t = 2 seconds.

" The packet-by-packet WFQ system transmits (1) the packet from buffer 1 at rate = 1 packet/sec, then (2) the packet from buffer 2 at rate = 1 packet/sec, so the packet completion times are 1 and 2 seconds, respectively.


GPS vs Packet-by-Packet WFQ, (Fig 7.45, pg 543) Note: Y-axis = fraction of packet in buffer1 and buffer 2. Both buffers contain 1 L -bit packet initially.


Packets with Different Lengths (Fig 7.46, pg 546)


Adding Flow Weights to GPS or WFQ (pg 546) " In a real IP router, different flows will have different bandwidth requirements " Let each flow j have its own buffer and a weight Wj which determines its

relative share of bandwidth C (bits/sec) " In each service round, the server services Wj bits from flow j " When all flows are active, each flow j receives the fluid service at the rate C*

(Wj) /sum(Wj == guaranteed service rate for each flow " When some flows are idle, the excess bandwidth is shared amongst active flows

according to the weight of the active flows

" the next slide illustrates an example of Weighted Fair Queueing - WFQ. Let flow 1 have weight W1=(1/4), and flow 2 have weight W2 = (3/4)

" Both flows have packet size = 1. The time to complete each packet is (L bits)/(Wj * C bits/sec), where Wj is expressed as a fraction. At t=0, based upon the 2 active flows, the time to complete flow #1 = L bits/(1/4 * C bits/sec) = 4 L/C seconds. Time to complete flow #2 = L bits/(3/4 * C bits/sec) = (4/3) L/C seconds. In this case (L/C) = 1. Once flow 2 is finished it service, since there are no other active flows, flow 1 starts receiving service at the rate L bits/sec. (When making such graphs, one difficulty is keeping track of the discontinuities, when the service rates change.)


Weighted GPS and Packet Fair Queueing (Fig. 7.48, pg 546)


GPS Finishing Time: Empty Queue

" Finishing times are expressed in rounds of service.

" Suppose packet k from flow f arrives at an empty queue at time t(f,k) and round R( t(f,k) )=>

" the packet arrives when R( t(f,k) ) rounds of service have already finished.

" The Packet has length P(f, k) bits. Therefore, it will complete its transmission after P(f,k)/Wf rounds of additional service (Wf expressed as an integer), since each round services Wf bits

" The finishing time is clearly: F(f,k)= R( t(f,k) ) + P(f,k)/Wf (1)


GPS Finishing time: Non-Empty Queue

" If packet k from flow f arrives at a nonempty queue, it must wait for all packets ahead of it (in particular packet k-1 from flow f) to be serviced before it is serviced.

" The finishing time of the packet k-1 ahead if it is F(f, k-1). " The current packet needs P(f,k)/Wf additional rounds of service,

therefore, the finishing time of this packet is clearly:

F(f,k) = F(f,k-1) + P(f,k)/Wf (2)


Comparing Both Finishing Times:

" Recall the finishing times using GPS fluid-flow fair queuing, when all flows have the same weight:

F(f,k) = max{F(f,k-1), R( t(f,k) } + P(f,k)

" When each flow has its own weight, the equation for the finishing times using GPQ fluid-flow fair queueing becomes:

F(f,k) = max{F(f,k-1), R( t(f,k) } + P(f,k)/Wf

" Each flows weight determines the flows share of the bandwidth.


Example " Consider 3 flows being serviced by a link with capacity C = 1 Mbps (ie 1 usec

per bit). Assume each flow has its own queue with weights 5, 10, and 20 (bits/round).

" Flow 1: pkt with 100 bits arrives at time t = 0 usec, weight = 5 " Flow 2: pkt with 300 bits arrives at time t = 50 usec, weight = 10 " Flow 3: pkt with 200 bits arrives at time t = 100 usec, weight = 20 " " (a) Plot the # Queued bits versus the round, for each queue in a fluid-flow

GPS queueing system. (A real router does not need plot this. We ask this question to make sure you understand why the VFT equations are valid.)

" (b) Plot the # Queued bits versus the real-time, for each queue in a fluid-flow GPS queueing system. (A real router does not need to plot this. We ask this question to make sure you understand why the VFT equations are valid.)

" (c) Label the slope of each piecewise linear segment. Label the (x,y) coordinates of each end point of a piece-wise linear segment


Example

" (d) What are the departure times of the last bit in each flow, first expressed in Rounds (Virtual-Time), and then expressed in Real-Time ?

" (e) Now consider a packet-by-packet WFQ approximation of the fluid-flow GPS system. Only one whole (entire) packet is transmitted at a time.

" Find the start time and end time for each packet in each flow, expressed in real-time. (A real router needs to know these numbers.)


Solution Summary

Flow

Bits Weight Arr. Time * (usec)

Arr. Time (round)

Exit Time (round)

ExitTime* (usec)

1 100 5 0 us 0 30 320

2 300 10 50 us 10 40 600

3 200 20 100 us 14 24 440

4 Nill

Assume new flows arrive at the beginning of a round, for simplicity.


Bits vs Virtual Time (Round)

This slide helps explain the GPS equations.

The bits in a GPS queue drain at a smooth rate, when the x-axis is the round of service


Bits vs Real Time

This slide helps explain the GPS equations.

The bits in a GPS queue drain at a variable rate, when the x-axis is real-time.

That is why we use the virtual time in the GPS equations It is easier to compute the virtual finishing times using virtual time (round), than real-time.


Queued Bits vs Real-Time, WFQ

A real WFQ server in a router implements this behaviour:

The green packet arrives at time 100 usec, just as the red packet finishes transmission. It has a smaller VFT than the blue packet (see slide 28), so the WFQ server sends the green packet before the blue packet, even though the blue packet arrived before the green packet.

Prof. T Szymanski

What Does a Real Router Do?

4DN4 - WFQ, pg 31

" A real router must implement the 4 GPS equations, to compute the virtual finishing time of each packet that arrives at the router.

" A router must have 2 clocks, a real-time clock, and a virtual-time clock (i.e., the current round of service).

" Every time a packet arrives at the router, the router must compute the packets VFT in rounds (virtual time).

" A real router has a WFQ server, which transmits outgoing packets in the order of increasing packet VFTs, so that packets leave the real router in the same order as if a perfectly fair fluid-flow GPS server was used.


Jitter Comparison, TCP vs WFQ

Compare: TCP vs WFQ traffic sources, generating packets at a average rate of 50 Mbp, for a 1 Gbps transmission line.

A TCP source could generate bursts of packets, at the full transmission rate of 1 Gbps, every time a sequence of ACKs is received (roughly every round-trip time): bursts could occur 1/20 of the time, idle periods could occupy 19/20 of the time, and if the network is not too loaded, the bursts will travel without much dispersion => potentially overloading queues


Inter-Departure Time Distribution, TCP vs WFQ

" Assume packets are fixed-size; the TCP departures vs. Time will be bursty:

" An unconstrained TCP source produces bursts of packets(red), consuming the full 1Gbps bandwidth for the duration of the burst.

" Bursts also be caused by the uneven delays in the reception of ACKs.

" TCP can cause instantaneous congestion at intermediate nodes , causing potential dropping of packets when Random Early Detection (RED) is used, introducing wasted bandwidth and resource usage.


Departure Time Distribution , TCP vs WFQ

! The WFQ jitter is much less bursty ! With WFQ, each flow has a fair share of the 1 Gbps bandwidth. At a

50 Mb/sec sustained rate, WFQ packet departures are evenly spaced, with one packet in every 20 packet time-slots, followed by 19 empty time-slots.

! The traffic is much less bursty and will reduce/eliminate the instantaneous congestion at intermediate routers that TCP bursts may experience


Commercial Implementations

! Consider a national network: ! WFQ can be used per individual flow. ! Very expensive when used for individual flows, since we may have over

1 billion flows for a national backbone network.

http://nsssc.superb.net/img/algx-usmap.gif


Class-Based Weighted Fair Queuing [2]

! A commercial implementation of WFQ is Class Based Weighted Fair Queuing (CBWFQ) , used by Cisco Systems

! CBWFQ extends the WFQ functionality by providing support for user-defined traffic classes instead of individual flows.

! IP doesnt have any easy way to specify flows. Therefore, classes are defined by a match criteria on packet headers which include:

! Protocols (TCP, UDP, RTP, etc) ! IP addresses ! TCP and UDP ports

! A queue is reserved for each traffic class. Incoming packets are filtered and IP packets that meet the match criteria for a class are placed in their corresponding class queue. Network management software allows for IP routers to specify class-match criteria (class filters)


Class-Based Weighted Fair Queuing

! After defining a class according to its filter, the characteristics of the class must be assigned.

! Class characteristics include: ! Bandwidth guaranteed amount to be delivered to a class

determined by class weight.

! Maximum Packet limit maximum amount of packets allowed to accumulate in a queue for that class.

! The weight for each packet is derived from class weight and CBWFQ uses the class weights to ensure that each class is serviced fairly.

! More information available at Cisco Systems website: www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120t/120t5/cbwfq.htm


Other scheduling methods

" There are many other proposed fair queue scheduling algorithms, and many more variations of Weighted Fair Queueing, including:

" Deficit Round Robin

" Self-Clocked Fair Queuing " Rate Controlled Service Discipline

" Worst case fair queuing

" GPS and WFQ define the ideal fair bandwidth-sharing performance. In practice, one cannot implement fluid-flow GPS, so an algorithm which approximates the ideal scheme, such as packet-based WFQ, can be used.


Appendix

1. Communication Networks: Fundamental Concepts and Key Architectures Second Edition. Alberto Leon-Garcia and Indra Widjaja. McGraw-Hill.

2. Class-Based Weighted Fair Queuing. Cisco Systems Inc. www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120t/120t5/cbwfq.htm

3. A. Parekh and R.G. Gallager, A Generalized Processor Sharing Approach to Flow Control in Integrated Services Networks: The Single-Node Case IEEE/ACM Trans. Networking, vol. 1, no. 3, pp. 344-357, June 1993. (Based upon A. Parekhs PhD Thesis at MIT).


Example (Ignore in 2013)

Prof. T Szymanski

Some Fun With Matlab

! Lets use Matlab to implement WFQ as a real router would ! Most Internet routers implement WFQ in real time to achieve some

type of fairness, processing millions of packets per second

! We generally have around a microsecond per packet to compute the fair GPS virtual finishing times

! Lets (1) generate one flow = random stream of arriving packets ! Lets (2) plot the flow so we can visualize the packets and their

sizes

! Lets (3) generate 4 flows ! Lets (4) plot the 4 flows on one graph, so we can see the

competition for bandwidth

! Let (5) figure out how to implement WFQ in a router !

4DN4 - WFQ, pg 45

Prof. T Szymanski

Generate One Flow

4DN4 - WFQ, pg 46

Prof. T Szymanski

Plot the Flow on One Graph

4DN4 - WFQ, pg 47

Prof. T Szymanski

The Main program : Generate 1 Flow

4DN4 - WFQ, pg 48

Prof. T Szymanski

The Figure : One Flow (Packet Stream)

4DN4 - WFQ, pg 49

Prof. T Szymanski

Generate 4 Flows (Packet Streams)

4DN4- WFQ, pg 50

Prof. T Szymanski

Plot All 4 Flows on One Graph

4DN4 - WFQ, pg 51

Prof. T Szymanski

The Result: 4 Arriving Flows

4DN4 - WFQ, pg 52

Now lets use WFQ to achieve fairness in a router.

Prof. T Szymanski

Next: In Class Example for WFQ

4DN4 - WFQ, pg 53

! Ill ask the class for ideas on how to use Matlab to implement WFQ as a real router would

! Lets experiment with the Matlab, and see what we can achieve ! Assignment #2 will cover WFQ

Documents

L5a 731 WFQ Slides (Updated) 2pp