Traffic Behavior and Queuing Traffic Behavior and Queuing in in QoSQoS Environment Environment

Hamid R. RabieeMostafa Salehi, Fatemeh Dabiran, Hoda Ayatollahi

Spring 2011

Objectives

² Provide some basic understanding of queuing phenomena

² Explain the available solution approaches and associated

trade-offs

² Give guidelines on how to match applications and solutions

Outline

² Basic concepts

² Source models

² Service models (demo)

² Single-queue systems

² Priority/shared service systems

² Networks of queues

Outline

² Basic concepts² Performance measures² Solution methodologies² Queuing system concepts² Stability and steady-state² Causes of delay and bottlenecks

² Source models² Service models(demo)² Single-queue systems ² Priority/shared service systems ² Networks of queues

Performance Measures

² Delay

² Delay variation (jitter)

² Packet loss

² Efficient sharing of bandwidth

² Relative importance depends on traffic type (audio/video, file transfer,

interactive)

² Challenge: Provide adequate performance for (possibly) heterogeneous

traffic

Queuing System Concepts: Arrival Rate, Occupancy, Time in the System

² Queuing system

² Data network where packets arrive, wait in various queues, receive service at

various points, and exit after some time

² Arrival rate

² Long-term number of arrivals per unit time

² Occupancy

² Number of packets in the system (averaged over a long time)

² Time in the system (delay)

² Time from packet entry to exit (averaged over many packets)

Stability and Steady-State

² A single queue system is stable ifpacket arrival rate < system transmission capacity

² For a single queue, the ratio

packet arrival rate / system transmission capacity

is called the utilization factor² Describes the loading of a queue

² In an unstable system packets accumulate in various queues and/or get dropped

² For unstable systems with large buffers some packet delays become very large² Flow/admission control may be used to limit the packet arrival rate

² Prioritization of flows keeps delays bounded for the important traffic

² Stable systems with time-stationary arrival traffic approach a steady-state

Little’s Law

² For a given arrival rate, the time in the system is proportional to packet occupancy

N = λ T

where

N: average number of packets in the system

λ: packet arrival rate (packets per unit time)

T: average delay (time in the system) per packet

² Examples:² On rainy days, streets and highways are more crowded

² Fast food restaurants need a smaller dining room than regular restaurants with the same customer

arrival rate

² Large buffering may cause large delays

Explanation of Little’s Law

² Amusement park analogy: people arrive, spend time at various sites, and

leave

² They pay $1 per unit time in the park

² The rate at which the park earns is $N per unit time

(N: average number of people in the park)

² The rate at which people pay is $λ T per unit time

(λ: traffic arrival rate; T: time per person)

² Over a long horizon:

Rate of park earnings = Rate of people’s payment

or

N = λ T

Delay is Caused by Packet Interference

² If arrivals are regular or sufficiently spaced apart, no queuing delay occurs

Regular Traffic

Irregular but Spaced Apart Traffic

Time

Arrival Times

Departure Times

1 3 42

1 3 42

Time

Arrival Times

Departure Times

1 3 42

1 3 42

Burstiness Causes Interference

² Note that the departures are less bursty

Time

Queuing Delays

Bursty Traffic

1 2 3 4

1 2 3 4

Burstiness ExampleDifferent Burstiness Levels at Same Packet Rate

Source: Fei Xue and S. J. Ben Yoo, UCDavis, “On the Generation and Shaping Self-similar Traffic in Optical Packet-switched Networks”, OPNETWORK 2002

Packet Length Variation Causes Interference

Regular arrivals, irregular packet lengths

Time

Queuing Delays

High Utilization Exacerbates Interference

As the work arrival rate:

(packet arrival rate * packet length)

increases, the opportunity for interference increases

Time

Queuing Delays

Bottlenecks

² Types of bottlenecks

² At access points (flow control, prioritization, QoS enforcement needed)

² At points within the network core

² Isolated (can be analyzed in isolation)

² Interrelated (network or chain analysis needed)

² Bottlenecks result from overloads caused by:

² High load sessions, or

² Convergence of sufficient number of moderate load sessions at the same queue

Bottlenecks Cause Shaping

² The departure traffic from a bottleneck is more regular than the arrival

traffic

² The inter-departure time between two packets is at least as large as the

transmission time of the 2nd packet

Time

Bottlenecks Cause Shaping

Bottleneck

90% utilization

Transmission time

sec

Exponentialinterarrivals

sec

# of packets# of packets

Incoming trafficInterarrival times

Outgoing trafficInterdeparture times

Fixed packetlength

Bottleneck

90% utilization

Outgoing trafficInterdeparture times

Incoming trafficInterarrival times

Large

Medium

Smallsec

# of packets

Packet Trains

Histogram of inter-departure times for small packets

sec

# of packets

Outline

² Basic concepts

² Source models

² Poisson traffic

² Batch arrivals

² Example applications – voice, video, file transfer

² Service models (demo)

² Single-queue systems

² Priority/shared service systems

² Networks of queues

Poisson Process with Rate λ

² Interarrival times are independent and exponentially

distributed

² Models well the accumulated traffic of many

independent sources

² The average interarrival time is 1/ λ (secs/packet), so

λ is the arrival rate (packets/sec)

Time

Interarrival Times

Batch Arrivals

² Some sources transmit in packet bursts

² May be better modeled by a batch arrival process

(e.g., bursts of packets arriving according to a Poisson process)

² The case for a batch model is weaker at queues after the first, because of

shaping

Time

Interarrival Times

Markov Modulated Rate Process (MMRP)

² Extension: Models with more than two states and/or stochastic

transmission process

Stay in each state an exponentially distributed time, Transmit according to a deterministic processat each state

State 0 State 1

OFF ON

Source Types

² Voice sources

² Video sources

² File transfers

² Web traffic

² Interactive traffic

² Different application types have different QoS requirements, e.g., delay,

jitter, loss, throughput, etc.

Source Type Properties

Characteristics QoS Requirements

Model

Voice * Alternating talk-spurts and silence intervals.

* Talk-spurts produce constant packet-rate traffic

Delay < ~150 msJitter < ~30 msPacket loss < ~1%

* Two-state (on-off) Markov Modulated Rate Process (MMRP)

* Exponentially distributed time at each state

Video * Highly bursty traffic (when encoded)

* Long range dependencies

Delay < ~ 400 msJitter < ~ 30 msPacket loss < ~1%

K-state (on-off) Markov Modulated Rate Process (MMRP)

DataFTP

telnetweb

* Poisson type * Sometimes batch-

arrivals, or bursty, or sometimes on-off

Zero or near-zero packet loss Delay may be important

Poisson, Poisson with batch arrivals, Two-state MMRP

Typical Voice Source Behavior

MPEG1 Video Source Model

Diagram Source: Mark W. Garrett and Walter Willinger, “Analysis, Modeling, and Generation of Self-Similar VBR Video Traffic, BELLCORE, 1994

² The MPEG1 MMRP model can be extremely bursty, and has “long range

dependency” behavior due to the deterministic frame sequence

Outline

² Basic concepts

² Source models

² Service models

² Single vs. multiple-servers

² FIFO, priority, and shared servers

² Demo

² Single-queue systems

² Priority/shared service systems

² Networks of queues

Device Queuing Mechanisms

² Common queue examples for IP routers

² FIFO: First In First Out

² PQ: Priority Queuing

² WFQ: Weighted Fair Queuing

² Combinations of the above

² Service types from a queuing theory standpoint

² Single server (one queue - one transmission line)

² Multiple server (one queue - several transmission lines)

² Priority server (several queues with hard priorities - one transmission line)

² Shared server (several queues with soft priorities - one transmission line)

Single Server FIFO

² Single transmission line serving packets on a FIFO (First-In-First-Out)

basis

² Each packet must wait for all packets found in the system to complete

transmission, before starting transmission

² Departure Time = Arrival Time + Workload Found in the System +

Transmission time

² Packets arriving to a full buffer are dropped

ArrivalsTransmission

Line

Multiple Servers

² Multiple packets are transmitted simultaneously on multiple lines/servers

² Head of the line service: packets wait in a FIFO queue, and when a server

becomes free, the first packet goes into service

Arrivals

Transmission Lines

Priority Servers

² Packets form priority classes (each may have several flows)

² There is a separate FIFO queue for each priority class

² Packets of lower priority start transmission only if no higher priority packet is waiting

² Priority types:² Non-preemptive (high priority packet must wait for a lower priority packet found under transmission

upon arrival)

² Preemptive (high priority packet does not have to wait …)

Class 1 Arrivals High Priority

Transmission Line

Class 3 Arrivals Low Priority

Class 2 Arrivals Interm. Priority

Priority Queuing

² Packets are classified into separate queues

² E.g., based on source/destination IP address, source/destination TCP port, etc.

² All packets in a higher priority queue are served before a lower priority queue is served

² Typically in routers, if a higher priority packet arrives while a lower priority packet is

being transmitted, it waits until the lower priority packet completes

Shared Servers

² Again we have multiple classes/queues, but they are served with a “soft” priority

scheme

² Round-robin

² Weighted fair queuing

Class 1 Arrivals Weight 10

Transmission Line

Class 3 Arrivals Weight 1

Class 2 Arrivals Weight 3

Round-Robin/Cyclic Service

² Round-robin serves each queue in sequence

² A queue that is empty is skipped

² Each queue when served may have limited service (at most k packets

transmitted with k = 1 or k > 1)

² Round-robin is fair for all queues (in terms of packet transmission rate)

² Round-robin cannot be used to enforce bandwidth allocation among the

queues.

Fair Queuing

² This scheduling method is inspired by the “most fair” of methods:

² Transmit one bit from each queue in cyclic order (bit-by-bit round robin)

² Skip queues that are empty

² To approximate the bit-by-bit processing behavior, for each packet² We calculate upon arrival its “finish time under bit-by-bit round robin” and we

transmit by FIFO within each queue

² Transmit next the packet with the minimum finish time

² Important properties:

² Priority is given to short packets

² Equal bandwidth is allocated to all queues that are continuously busy

Finish Time of Packet i

i-1Arrival times

Departure times

i

ii-1

Finish Time of Packet i

i-1Arrival times

Departure times

i

ii-1

Weighted Fair Queuing

² Fair queuing cannot be used to implement bandwidth allocation and soft priorities

² Weighted fair queuing is a variation that corrects this deficiency² Let wk be the weight of the kth queue² Think of round-robin with queue k transmitting wk bits upon its turn² If all queues have always something to send, the kth queue

receives bandwidth equal to a fraction wk / Σi wi of the total bandwidth

² Fair queuing corresponds to wk = 1² Priority queuing corresponds to the weights being very high as we

move to higher priorities² Implementation: For each packet² Calculate its “finish time” (under the weighted bit-by-bit round

robin scheme)² Transmit the packet with the minimum finish time

Weighted Fair Queuing Illustration

Weights:Queue 1 = 3Queue 2 = 1Queue 3 = 1

A Practical Combination (e.g. Cisco)

² Example – voice in PQ, guaranteed b/w traffic in WFQs

(all at middle priority), and best effort traffic in low priority queue

Demo: Comparing FIFO, WFQ and PQ

² Two traffic streams mixing on a common interface² Video² FTP

² Apply different service schemes² FIFO² PQ² WFQ

² Run simulation and compare queuing delays

Demo: FIFO

FIFOBottleneck 90% utilization

Demo: FIFO Queuing Delay

Applications have different requirements

§ Video§ delay, jitter

§ FTP§ packet loss

Control beyond “best effort” needed

§ Priority Queuing (PQ)§ Weighted Fair Queuing

(WFQ)

Demo: Priority Queuing (PQ)

PQBottleneck 90% utilization

Demo: PQ Queuing Delays

FIFO

PQ Video

PQ FTP

Demo: Weighted Fair Queuing (WFQ)

WFQBottleneck 90% utilization

Demo: WFQ Queuing Delays

FIFO

WFQ/PQ Video

PQ FTP

WFQ FTP

Queuing: Summary Points

² Choice of queuing mechanism can have a profound effect on performance² To achieve desired service differentiation, appropriate queuing

mechanisms can be used² Complex queuing mechanisms may require simulation techniques to

analyze behavior ² Improper configuration (e.g., queuing mechanism selection or weights)

may impact performance of low priority traffic

Outline

² Basic concepts

² Source models

² Service models (demo)

² Single-queue systems

² M/M/1……M/M/m/k

² M/G/1……G/G/1

² Demo: Analytics vs. simulation

² Priority/shared service systems

² Networks of queues

M/M/1 System

§ Nomenclature: M stands for “Memoryless” (a property of the exponential distribution)§M/M/1 stands for Poisson arrival process (which is

“probabilistically memoryless”) §M/M/1 stands for exponentially distributed transmission times

§ Assumptions:§ Arrival process is Poisson with rate λ packets/sec§ Packet transmission times are exponentially distributed with mean

1/µ§One server§ Independent interarrival times and packet transmission times

§ Transmission time is proportional to packet length§ Note 1/µ is secs/packet so µ is packets/sec (packet transmission rate of the

queue)§ Utilization factor: ρ = λ/µ (stable system if ρ < 1)

Delay Calculation

² Let

Q = Average time spent waiting in queue

T = Average packet delay (transmission plus queuing)

² Note that T = 1/µ + Q

² Also by Little’s law

N = λ T and Nq = λ Q

where

Nq = Average number waiting in queue

² These quantities can be calculated with formulas derived by Markov chain

analysis (see references)

² The analysis gives the steady-state probabilities of number of

packets in queue or transmission

² P{n packets} = ρn(1-ρ) where ρ = λ/µ

² From this we can get the averages:

N = ρ/(1 - ρ)

T = N/λ = ρ/λ(1 - ρ) = 1/(µ - λ)

ρ

N

10 λ

T

µ0

1/µ

M/M/1 Results

Example: How Delay Scales with Bandwidth

² Occupancy and delay formulas

N = ρ/(1 - ρ) T = 1/(µ - λ) ρ = λ/µ

² Assume:

² Traffic arrival rate λ is doubled

² System transmission capacity µ is doubled

² Then:

² Queue sizes stay at the same level (ρ stays the same)

² Packet delay is cut in half (µ and λ are doubled)

² A conclusion: In high speed networks

² propagation delay increases in importance relative to transmission and queuing delays

² buffer size and packet loss may still be a problem

M/M/m, M/M/∞ System

²Same as M/M/1, but it has m (or ∞) servers

²In M/M/m, the packet at the head of the queue

moves to service when a server becomes free

²Qualitative result

²Delay increases to ∞ as ρ = λ/mµ approaches 1

²There are analytical formulas for the occupancy

probabilities and average delay of these systems

Finite Buffer Systems: M/M/m/k

² The M/M/m/k system

² Same as M/M/m, but there is buffer space for at most k

packets. Packets arriving at a full buffer are dropped

² There are formulas for average delay, steady-state

occupancy probabilities, and loss probability

² The M/M/m/m system is used widely to size telephone or

circuit switching systems

Characteristics of M/M/. Systems

² Advantage: Simple analytical formulas

² Disadvantages:

² The Poisson assumption may be violated

² The exponential transmission time distribution is an

approximation at best

² Interarrival and packet transmission times may be

dependent (particularly in the network core)

² Head-of-the-line assumption precludes heterogeneous input

traffic with priorities (hard or soft)

M/G/1 System

² Same as M/M/1 but the packet transmission time distribution is

general, with given mean 1/µ and variance σ2

² Utilization factor ρ = λ /µ

² Pollaczek-Kinchine formula for

Average time in queue = λ(σ2 + 1/µ2)/2(1- ρ)

Average delay = 1/µ + λ(σ2 + 1/µ2)/2(1- ρ)

² The formulas for the steady-state occupancy probabilities are more

complicated

² Insight: As σ2 increases, delay increases

G/G/1 System

² Same as M/G/1 but now the packet interarrival time

distribution is also general, with mean λ and variance γ2

² We still assume FIFO, independent interarrival times and

packet transmission times

² Heavy traffic approximation:

Average time in queue ~ λ(σ2 + γ2)/2(1- ρ)

² Becomes increasingly accurate as ρ→1

Demo: M/G/1

Packet inter-arrival times

exponential (0.02) sec

Capacity1 Mbps

Packet size 1250 bytes(10000 bits)

Packet size distribution:exponential

constantlognormal

What is the average delay and queue size ?§ Compare analytical formulas with simulation results

Demo: M/G/1 Analytical Results

Packet Size Distribution Delay T (sec) Queue Size (packets)

Exponentialmean = 10000

variance = 1.0 *1080.02 1.0

Constantmean = 10000variance = 0.0

0.015 0.75

Lognormalmean = 10000

variance = 9.0 *1080.06 3.0

Demo: M/G/1 Simulation Results

Average Delay (sec) Average Queue Size (packets)

Demo: M/G/1 Limitations

Application traffic mix not memoryless

§ Video § constant packet inter-arrivals

§ Http§ bursty traffic

Delay

P-K formula

Simulation

Outline

² Basic concepts

² Source models

² Service models (demo)

² Single-queue systems

² Priority/shared service systems

² Preemptive vs. non-preemptive

² Cyclic, WFQ, PQ systems

² Demo: Simulation results

² Networks of queues

² Hybrid simulation (demo)

Non-preemptive Priority Systems² We distinguish between different classes of traffic (flows)

² Non-preemptive priority: packet under transmission is not preempted

by a packet of higher priority

² P-K formula for delay generalizes

Class 1 Arrivals High Priority

Transmission Line

Class 3 Arrivals Low Priority

Class 2 Arrivals Interm. Priority

Cyclic Service Systems

² Multiple flows, each with its own queue

² Fair system: Each flow gets access to the transmission line in turn

² Several possible assumptions about how many packets each flow can

transmit when it gets access

² Formulas for delay under M/G/1 type assumptions are available

Class 1 Arrivals

Transmission Line

Class 3 Arrivals

Class 2 Arrivals

Weighted Fair Queuing

² A combination of priority and cyclic service

² No exact analytical formulas are available

Outline

² Basic concepts

² Source models

² Service models (demo)

² Single-queue systems

² Priority/shared service systems

² Networks of queues

² Violation of M/M/. assumptions

² Effects on delays and traffic shaping

² Analytical approximations

Two Queues in Series

² First queue shapes the traffic into second queue

² Arrival times and packet lengths are correlated

² M/G/1 formulas yield significant error for second queue

² How about M/M/1 if packet lengths are exponential?

Time

First Queue

Time

Second Queue

Two bottlenecks in series

Bottleneck

Exponentialinter-arrivals

Bottleneck

No queuing delayDelay

Spaced-apart inter-arrivals

Spaced-apart departures

Approximations

² Kleinrock independence approximation

² Perform a delay calculation in each queue independently of other queues

² Add the results (including propagation delay)

² Note: In the preceding example, the Kleinrock independence

approximation overestimates the queuing delay by 100%

² Tends to be more accurate in networks with “lots of traffic mixing”, e.g.,

nodes serving many relatively small flows from several different locations

References

² Networking² Bertsekas and Gallager, Data Networks, Prentice-Hall, 1992

² Device Queuing Implementations² Vegesna, IP Quality of Service, Ciscopress.com, 2001² http://www.juniper.net/techcenter/techpapers/200020.pdf

² Probability and Queuing Models² Bertsekas and Tsitsiklis, Introduction to Probability, Athena Scientific, 2002,

http://www.athenasc.com/probbook.html² Cohen, The Single Server Queue, North-Holland, 1992² Takagi, Queuing Analysis: A Foundation of Performance Evaluation. (3

Volumes), North-Holland, 1991² Gross and Harris, Fundamentals of Queuing Theory, Wiley, 1985² Cooper, Introduction to Queuing Theory, CEEPress, 1981

² OPNET Hybrid Simulation and Micro Simulation² See Case Studies papers in

http://secure.opnet.com/services/muc/mtdlogis_cse_stdies_81.html