Student: Shih-Chiang TsaoAdvisor: Ying-Dar LinDate: 2007/12/12

DissertationDissertation

Fairness Controls forTCP-equivalence at Endpoint and

Request-Response Scheduling at Gateway

Where are bottlenecks? What kind of fairness? How and where to control it?

2 /482007/12/12

Bottlenecks for the Internet Traffic

EDU2

ISP2

ISP1

EDU1

ERER

ER

ER

S

Internet

D2

D1

ER

ER

GIGUGU

H1H1

HnHn

Intranet

Public Fairness

PrivateFairness

Issue Promoted concept Criterion Schemes

Public Fairness

TCP-friendliness(TCP-compatibility)

<= TCP’s bandwidth AIMD in TCP

Private Fairness

Class-based weighted fairness

Bw1:Bw2=Weight1:Weight

2

Packet scheduling

R

ER: edge routerGI: ISP-side gatewayGU:User-side gateway Hi, S, D1, D2: End Points

3 /482007/12/12

Public Fairness Control at End Point

Motivation: New rate control for Streaming

Objective: Smoothness Fairness with TCP

R1S1

InternetR2S2

E

EE

R RData

Streaming

TCP

???

Trans. Rate Trans. Direction Instance Tolerable Start-up latency

Unconstrained Simplex On-demand Video <5 sec

Constrained

Simplex Live broadcast <5 sec

Interactive Video conference <0.5 sec

Classification of Internet Streaming

Seq. No.

TimeStart-up Latency

received playing

Late packets Because of the oscillatory rat

e of TCP

4 /482007/12/12

Private Fairness Control at Gateway

InternetGUGI

W1

W2GG

W3

user-side access gateway

H1

Hn

Queuing packets

ISP-side edge gateway

Motivation:Scheduling packets fails to allocate downlink bandwidth at GI

Scheduling requests to manage the responses

Objectives: Weighted fairnessShared bandwidthFull link utilizationShort transmission time

Uplink requests ->

<- Downlink responses

the IP addresses of Hi’s are hidden in packetsFavorable place by

enterprises

5 /482007/12/12

Related Work and Research Road Map

Fairness

Private Fairness

- Weighted Fairness [PG93]- Class-based [FJ95]

Packet Scheduling

-WFQ [PG93]-WRR/ DRR [SV96]-SCFQ [GOL94]-SFQ [GVC96]

Pre-order DRR (Computer Networks)

Public Fairness

- TCP-friendliness [FF99, BCC98]- TCP throughput eq [PFT98, AAB05]- Internet Conditions [ZDP01][JID04]

TCP-friendly Schemes

- GAIMD [YL00]- TFRC [FHP00]- TEAR [ROY00]- SQRT, IIAD [BB01]- SIMD [JGM03]2. WARC (To be submitted to IEEE Trans. on Computer)

Evaluation

- Survey [WDM01]- Dynamic Cond. [BBF01]- TFRC’s analysis [VB05]1. Taxonomy & Evaluation(IEEE Network, Nov. 2007)

Request Scheduling

- web server[PBB98, BBK00, CP99]- web-side gateway[CCC02, CC01, LGC01]- user-side gateway3. Minimum-service first request scheduling(Submitted to Computer Networks)

Utilization of high BW*delay path

High-speed TCP

Bandwidth-HS-TCP [SF03]

-FAST [JLH07,TWH05]

-XCP [KHR02]

-VCP [XSS05]

DCCP [KHF06]

Protocol

Versions of TCP

-Vegas [BP95]-Improvement [YCC06]

Wireless 802.11e, 802.11s

Load Balance

On-the-fly TCP Path Selection(Computer Comm.)

TCP-friendly AQM

-Survey [CLB04]-WARD [YCC07]

Taxonomy and Evaluation of TCP-friendly Rate-Control Schemeson Fairness, Aggressiveness, and Responsiveness

Why do TCP-friendly Schemes have throughput unequal to TCP’s?

7 /482007/12/12

3 Criteria for 8 TCP-friendly Schemes

Criterion Premise

Proper behaviors for a scheme

Steady-state Transient-state

Fairness Aggressiveness Responsiveness

TCP-compatibility Identicalnetwork

conditions

Less bw Don’t care As fast as TCP

TCP-equivalenceEqual bw As fast as TCP

TCP equal-shareIdentical

bottleneck

Taxonomy

EvaluationScheme Full Name Parameters Ref.

GAIMD General additive inc./multiplicative-dec. α=0.2, β=0.125 [YL00]

IIAD Inverse-inc./additive-dec. α=1.0, β=0.67, k=1, l=0 [BB01]

SQRT Square-root inc./dec. α=1.0, β=0.67, k=0.5, l=0.5 [BB01]

SIMD Square-inc./multiplicative-dec. β=0.0625, k=-0.5, l=1 [JGM03]

AIAD/H Additive inc./dec. with history β=0.25, k=0, l=0 [JGM03]

TFRCP TCP-friendly rate control protocol Interval=5 seconds [PKT99]

TFRC TCP-friendly rate control The number of samples=8 [FHP00]

TEAR TCP-emulation at receiver The number of samples=8 [ROY00]

More realistic

Influenced by AQM

8 /482007/12/12

Fairness Policy In steady-state how a scheme control a flow

to use the equivalent bandwidth as a TCP flow?

Rate=1/(the time between packets) Estimates the recent TCP throughput dur

ing the connection. Repeatedly adjust the sending rate by the

estimation TFRC,TEAR, TFRCP

Rate

Time (s)x xx

Rate-based (RB)

short-termTCP’s mean rate

x

Update CWND by a set of control parameters

Specific relations between the parameters

GAIMD, SQRT, IIAD, SIMD, AIAD/H Long-term Fair

E[TWB]=f(p, RTT, a,b,k,l..) E[TTCP]=f(p, RTT,α=1, β=0.5) E[TWB]=E[TTCP]

Window-based (WB)

Cong. Window (CWND)

Time (RTT)xx xxx

packet loss event

x xx x x

long-term TCP’s mean rate

9 /482007/12/12

Aggressiveness Policy How a scheme increases the throughput of a flow

before encountering the next lossbehavior between two losses

Sub-linear Linear Super-linear

Step/proportion of each inc.

Non-historical GAIMD TFRCP

Historical SQRT, IIAD AIAD/HTEAR, TFRC

SIMD

Tradeoff between aggr. & smoothness

Aggressive but not smooth

Smooth but not aggressive

CWND

Time

Tradeoff CWND

Time

Slow at beginning and Fast if no loss

occurs for long time

Sub-linear CWND

Time

Super-linear

10 /482007/12/12

Responsiveness Policy How a a scheme decreases the throughput of a flow

when the loss condition becomes severe

Non-history History

Fixed Variable

IIAD, AIAD/H TFRC, TFRCP, TEAR GAIMD, SQRT, SIMD

-Historical scheme is adaptive for wider network conditions -Fixed-history scheme have fast responsive behavior

Tradeoff between resp. & smoothness

Tradeoff between resp. & smoothness Discard

Out-of-bound historyDiscard

Out-of-bound history

Responsive but not smooth

Smooth but not responsive

CWND

Time

Tradeoff

Loss rate

Time

Tx. Rate

Time

Fixed

Loss rate

Tx. Rate

Time

Time

Variable

11 /482007/12/12

EvaluationsFairness Aggressiveness/

Responsiveness

TCP-equivalence (Identical loss condition, artificial loss link)

- Change loss rate

- Change loss variance

Two-states

TCP equal-share (Identical bottleneck, dumbbell topology)

RTT- heterogeneous On/Off CBR

S DR1 R2

100Mbps2ms

100Mbps2ms

100Mbps30 ms

Discarding packets by math model

TimeXX X

Inter-loss

loss loss loss

R1

S1

Sn

R2

Drop-Tailn TCP-friendlysenders

2n Mbps10ms S’1

S’n

n TCPsenders

D1

Dn

D’1

D’n

100 Mbps20ms on average

100 Mbps20ms on average

Artificial loss link Dumbbell topology

12 /482007/12/12

Losses in the Internet

Losses in the Internet

Different trend

Different trend

Different trend

Different trend

Losses in the Internet [ZDP01]

Losses in the Internet [ZDP01]

TCP-equivalence as CV[T]=0

TCP-equivalence as CV[T]=0

T: the time between two lossesCV[T]: the coefficient-of-variance of T

Fairness Test for TCP-Equivalence:Different Variances of Inter-loss Time

Observation 1: Non-periodic losses should be considered in adopting WB/RB fairness policies

13 /482007/12/12

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

CV[RTT]=0 AVG(CV[RTT]>0.25)

Loss

(T

FC

C)

/Los

s(T

CP

)

SIMDGAIMDAIAD/HIIADSQRTTFRCPTFRCTEAR

Fairness Test for TCP Equal-share:Low-multiplexing Traffic

RTT-heterogeneity matters

for TCP equal-share

RTT-heterogeneity matters

for TCP equal-share

(b) n = 8SQRT

TFRCP, TFRC, TEAR

SIMD

IIAD

GAIMD Rate-based fairness policy

wins

Rate-based fairness policy

wins

Drop-Tail, N=8

n=8

Observation 2: RB fairness policy wins and RTT-heterogeneity matters for TCP equal-share

14 /482007/12/12

Aggr. and Resp. Test For TCP Equal-share

1

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

SQRT GAIMD SIMD TFRC TEAR

norm

aliz

ed lo

ss r

atio

No

rma

lize

d L

oss

Ra

tio

(a)

TCPSIMD

TEAR

IIAD

GAIMDTFRC

SQRT

TFRCP

AIAD/H

Non-history aggressive policy

Non-history aggressive policy

History/super-linear aggressive pol

icy

History/super-linear aggressive pol

icyFixed-history responsiveness poli

cy: fewer losses

Fixed-history responsiveness poli

cy: fewer losses

Observation 3: Historical/super-linearly aggressive and fixed-history responsive policies are satisfactory

15 /482007/12/12

Summary: Strategies in Eight SchemesTCP-compatibility is not enoughFairness at steady-state and fast agg/resp at transient-state

Policy Fairness Aggressiveness Responsiveness

Aspect throughput adjusting

step/proportionof each inc.

curve type life cycle ofloss statistics

GAIMD Window-based Non-historical Linear Variable-history

IIAD Window-based Historical Sub-linear Non-historical

SQRT Window-based Historical Sub-linear Variable-history

SIMD Window-based Historical Super-linear Variable-history

AIAD/H Window-based Historical Linear Non-historical

TFRCP Rate-based Non-historical Super-linear Fixed-history

TFRC Rate-based Historical Linear Fixed-history

TEAR Rate-based Historical Linear Fixed-history

16 /482007/12/12

Summary: Comparison among Schemes

Rate-based fairness policy Historical/super-linear

aggressiveness policy

Fixed history responsiveness policy

Behavior Fairness Aggressiveness Responsiveness

CriterionTCP-eq

(TCP-comp)TCP equal-share

TCP-eq(TCP-comp)

TCP eq-share

TCP-eq(TCP-comp)

TCP eq-share

ScenarioHeavy Losses

Non-periodicLosses

Low-multiplexingTwo-states

LossesBursty Losses

Two-states Losses

Bursty LossesHomogeneo

us RTTsHeterogeneous RTTs

GAIMD X (O) Δ(O) X X Δ (O) Δ Δ (Δ) Δ

IIAD X (O) X(O) Δ X X (O) X X (X) X

SQRT O (O) X(O) Δ X O (O) Δ O (O) O

SIMD X (O) Δ(O) X X O (O) O Δ (Δ) X

AIAD/H X (O) X(O) Δ X X (O) X X (X) X

TFRCP Δ (O) X(X) Δ O Δ (O) X O (O) O

TFRC Δ (O) Δ(O) Δ O Δ (O) Δ O (O) O

TEAR X (X) X(O) Δ O X (O) X O (O) O

O: Satisfactory Δ: Acceptable X: Unacceptable

A Fast-Converging TCP-Equivalent Window-Averaging

Rate Control Scheme

Perform better in terms of fairness, smoothness, aggressiveness, and responsivness

18 /482007/12/12

Window-Averaging Rate Control

Major rate control: Real-time estimation (RTE) control model

(Rate-based fairness policy) Fairness even under non-periodic losses (variant inter-loss time) Faster aggressiveness

Three complemental rate controls: History-reset (HR) mechanism: (Fixed-history responsiveness polic

y) For fast responsiveness

Fluid-based timeout mechanism: For fairness under heavy-losses

One-RTT reduction procedure: For FIFO-managed link

19 /482007/12/12

How to Calculate TCP’s Mean Rate?

Time

TEAR and TFRC: Fixed # of Epoches

CWND

Epoch, inter-loss time

Real-Time Estimation (RTE) Control Model

Lower rate under variant int

er-loss time

•WARC adjusts the rate per RTT.

•WARC averages the latest s CWNDs of a potential TCP flow.

min( , )

1

1( , ) ( )

min( , )

s t

i

R t m W t is t

CWND

TimeFixed # of CWNDs

WARC: Fixed # of CWNDs

20 /482007/12/12

Fast Aggressiveness

0

-1

-2-3-4

-5

-6-7-8

TFRC,TEAR WARC

21 /482007/12/12

History-reset Mechanism for Fast Responsiveness

If then

remove CWNDs before the nHRth last loss from rate computing

Rat

e (p

kt/R

TT

)

Rounds

R(t,s)

s roundsT

X(-1)X(-2)X(-N)X(-N+1)

the last loss

T-S(N)

the 2th last lossthe 9th last lossCWND

1( ) ( , )TCP KR N R t s

3 12 1

( ) ( )N

TCP N jR N X j

22 /482007/12/12

Analysis of Fairness

[Definition]in the steady state a scheme can control a flow to have the same mean rate as TCP does when both perceived the same network conditions

Periodic-losses Expo-losses Stationary-loss

WARC = = =TFRC* = < TCP < TCPTFRCP = > TCP > TCPTEAR = < TCP < TCP

Scheme

Loss conditions

23 /482007/12/12

Analysis on Aggressiveness

1/Aggr(m)= the time taken by a scheme to increase its rate with a factor of m.

timeLast loss

ratem

1

???

2 3 4 5m

200

400

600

800

1000

1/Aggressiveness(RTTs)

WARC(160) SIMD(1/16)

GAIMD(

1/5,1/

8)

TCP

IIAD(1,2/3)

Fast as the most aggressive scheme

[Definition] [JGM03]

24 /482007/12/12

Better Tradeoff between Smoothness and Aggressiveness

0.02 0.04 0.06 0.08

500

1000

1500

2000

2500

3000

3500

SIMD

Smoothness (CV[w])

1/Aggressiveness (RTTs)

GAIMD

IIAD

WARC

WARC(160)SIMD(1/16)GAIMD(1/5,1/8)IIAD(1,2/3)

0.02 0.04 0.06 0.08

500

1000

1500

2000

2500

3000

3500

SIMD

Smoothness (CV[w])

1/Aggressiveness (RTTs)

GAIMD

IIAD

WARC

WARC(160)SIMD(1/16)GAIMD(1/5,1/8)IIAD(1,2/3)

445

150

1080

25 /482007/12/12

Analysis on Responsiveness[Definition] [JGM03]1/Resp(m) = the number of loss events required by a scheme

to decrease the rate with a factor of m.

.

Smoothness (CV[w])

0.02 0.04 0.06 0.08 0.1 0.12 0.14

10

20

30

40

50

0.02 0.04 0.06 0.08 0.1 0.12 0.14

10

20

30

40

501/Responsiveness

(#loss events)

WARC(160)SIMD(1/16)GAIMD(1/5,1/8)IIAD(1,2/3)

WARC

1/Responsiveness

(#loss events)

2 4 6 8m

10

20

30

40

50

WARC(160)K=3,N=12

GAIMD(1/5,1/8)IIAD(1,2/3)

SIMD(1/1

6)

TCP=GAIMD(1,1/2)

1/Responsiveness

(#loss events)

2 4 6 8m

10

20

30

40

50

WARC(160)K=3,N=12

GAIMD(1/5,1/8)IIAD(1,2/3)

SIMD(1/1

6)

TCP=GAIMD(1,1/2)

2 4 6 8m

10

20

30

40

50

WARC(160)K=3,N=12

GAIMD(1/5,1/8)IIAD(1,2/3)

SIMD(1/1

6)

TCP=GAIMD(1,1/2)

more losses for smoothness

26 /482007/12/12

Probability of False-Positive Enabling HR

AssumeX(-j) is an i.i.d. exponential distribution

forms a gamma distribution (n, λ) 1( )

n

jX j

Invoked when the mean of inter-loss time does not change

P=10-3 ->False Positive per 1000 losses35 mins when W=5~30, RTT=50~300ms [JID04]

2.2 2.4 2.6 2.8 3 3.2 3.4k

0.002

0.004

0.006

0.008

0.01

N=16 N=12

N=10N=8

Prob.

K

10-3

2.2 2.4 2.6 2.8 3 3.2 3.4k

0.002

0.004

0.006

0.008

0.01

N=16 N=12

N=10N=8

Prob.

K

10-3

0.3 30

0.05 6

1 11000 2100

0.3 0.05 30 6 1.5

WRTT dW dRTT

1

1 1

( , ) ( ,1)

( ) ( , )

( ) [ ] ( )

( ) ( ),

TCP K

N N

j j

gamma N gamma N

P R N R t s

N NP X j EX P X j

K KN N

F FK K

27 /482007/12/12

Fairness Test for TCP-Equivalence: Under the Variant-Losses Network

WARC:Average fixed #

of CWND

WARC:Average fixed #

of CWND

28 /482007/12/12

Fairness Test for TCP Equal-Share15 Mbps-link

60 Mbps-link

TimeoutMechanism

TimeoutMechanism

Equal share

Equal share

WARC

WARC

TEAR

GAIMDSIMD

29 /482007/12/12

Fast Aggressiveness & Responsiveness

1.0

1.2

1.4

1.6

1.8

2.0

2.2

TFRC TEAR WARC GAIMD SIMD WARCw/o HR

norm

. #

of

loss

es

WARC decreases rate with fewer losses

WARC decreases rate with fewer losses

Fast aggressiveness:WARC and SIMD

TFRC

TEAR

WARC w/oOne-RTT reduction

GAIMD TCP

20sec

30 /482007/12/12

Smoothness over Different Time Scale

WARC is smooth as TFRC

WARC is smooth as TFRC

Smoother rate than

TCP

Smoother rate than

TCP

SIMD

GAIMD

SQRT

IIAD

Better smoothness

Better smoothness

TEAR

(0.1 sec)

31 /482007/12/12

Low Start-up Latency for Constrained Streaming(e.g. video conference)

late packets

WARC has low ratio of late packets

WARC

TCP

TCP

WARC

32 /482007/12/12

Applicability of TCP-equivalent Smooth Rate Control

s

IP

UDP

Socket

APP

RTP/RTCP

IP

UDP

Socket

APP

RTP/RTCP Rate Control

User-layer Solution

(IETF Draft)

LiveMedia Library (LGPL), DirectShow RTP Filter

IP

TCP

Socket

APP

Rate Control

Kernel-layer Solution

(RFC4340, S. Floyd)

IP

DCCP

Socket

APP Layered/Base Protocols

Supported in Linux Kernel

A possible solution in MS Windows

Datagram CongestionControl Protocol (DCCP)

33 /482007/12/12

Summary WARC

RTE control model + Fixed number of CWNDs Fairness, Aggressivness,

History-reset mechanism Responsiveness

TCP-equivalence and TCP equal-share Fairness under stationary loss condition.

For non-periodic loss conditions Fast Aggr. & Rspo. for drastic change

Smoothness

Problems on Applying Fair Queuing Discipline to Schedule Requests at Access Gateway

for Downlink Differential QoS

No-monthly fee solution for downlink differential service

35 /482007/12/12

Where to Schedule Packets?

InternetGUGI

W1

W2GG

W3

User-side gateway

H1

Hn

ISP-side gateway

access link

Uplink requests ->

<- Downlink responses

User-side gateway (GU) or ISP-side gateway (GI) ? GU is bought by the user’s specification and easy to be managed GI is owned by ISP. Additional charge may requrie. Packets are not queued at GU GI cannot see the IPs of H1~Hn

Scheduling uplink requests at GU

to managing downlink responses Class-based Fair Queuing

Queuing packets

36 /482007/12/12

monopolizes the link bandwidth

sending one-by-one

Responses share the downlink neither is appropriate

sending reqs one-by-one sending a request right after g

etting a response

1. Time to Release the Next ...Packet Request

monopolize

packets

simultaneous

responsesS

Srequests

37 /482007/12/12

Selecting in the order of service-completion time Known packet size

Fairness should rely on response size

Response size is unknown until it returns

2. From Which Queue to Release the Next ..

Packet Request

SS

requests

8 7 4

9 5 3

6 2 1Q1

Q2

Q3

packets

1i

i i LF F

known packet length

? ? ?

? ? ?

? ? ?Q1

Q2

Q3

requests

response

response size is onlyavailable in 1st

packet of response

38 /482007/12/12

3. User-based Weighted Fairness Class-based

Between different types of traffic: e.g. voice or ftp

Admission Control User-level Differentiation

High-class users get more bandwidth than low-class users

39 /482007/12/12

Cr

Q2

Qn

Cq

Requests

Response

requestselector

SC1

SCn

W

End of RI

U

Wmax

Minimum-Service First Request Scheduling (MSF-RS)

End of Rsp.

Minimum-service order arbiter (MOA)Q1

A changes BA BA is referenced by BA B

A A is a variable

Data flow

Window-basedrate controller

(WRC)

requestreceiver

UC1 UCn

requestreleaser

w1 w3

Minimum-Service First Request Scheduling

SC: Service CounterUC: User Counterw: Weight

InternetGUGI

H1

Hn

40 /482007/12/12

Minimum-service Order Arbiter (MOA)

Cr

Q2

Qn

Cq

Requests

Response

requestselector

SC1

SCn

W

End of RI

U

Wmax

End of Rsp.

Minimum-service order arbiter (MOA)

Q1

Window-basedrate controller (WRC)

requestreceiver

UC1 UCn

requestreleaser

w1 wn

2. Select from the class with the min SC

2. Select from the class with the min SC

1. Log the amount of received service

1. Log the amount of received service

1

kk k ii i

i i

LSC SC

w UC1

kk k ii i

i i

LSC SC

w UCthe length of the

received response k in bytes

41 /482007/12/12

Q2

Qn

Cqrequestselector

SC1

SCn

Minimum-service order arbiter (MOA)

Q1

requestreceiver

UC1 UCn

w1 w3

Window-based Rate Controller (WRC)

Requests

ResponseCr

W

End of RI

U

W+

End of Rsp.

Window-basedrate controller (WRC)

requestreleaser

Release requests if W<W+

Release requests if W<W+

W : the number of outstanding requestsT : the time interval between two updatesSi : the responses in bytes received during TC : the link capacity. K: a constant. Ui: the link utilization. U+: upper bound of U

1 min{ , }i ii

UW K W

U

1 min{ , }i i

i

UW K W

U

/ T,

Ci

i

SU

/ T,

Ci

i

SU

42 /482007/12/12

Analysis of User-perceived LatencyLong queuing time of request+ Short transmission time of response

= Short user-perceived latency

Client send

request

Gateway

getrequest

Gateway

sendrequest

Gatewayget

response

Clientget

response

User-perceived latency

Tq Ts

Ta=Tq+Ts

Time

0 1 2

(4*1+4*2)/8=1.5

1111

2222

Example20 40 60 80 100

m

0.5

0.6

0.7

0.8

0.9

W+=10W+=20

W+=40

W+=80

TaMSF-RS/Ta

ordinary

43 /482007/12/12

S

Qi

Qj

sublink1

sublinkW+

Rspi,1

Rspi,W+

+ji

i j

LW L

w w

Time

Normalized Service

Class i

Class j

t0 t1 t2

0

i

i

W L

w

Analysis for Worst-case Fairness of MSF-RS:

+ji

i j

LW L

w w

1 21 2( , )( , ) ji

i j

D t tD t t

w w

Di(t1,t2): the responses receved by Class i in bytes between t1 and t2

wi: the weight of Class iW+ : # of sub-linksL+ : Resp. of max. size

Fairness Parameter defined by Golestani for analysis of SCFQ

44 /482007/12/12

Weighted Fairness and Sharing

0

200

400

600

800

1000

1200

1400

1600

1800

2000

1 2 3 4 Phase

BW(kbps)

Class 1

Class 2

Class 3

All

0

50

100

150

200

250

300

4 8 12 16 20 24

Hosts in Class 1

Ban

dwid

th (

Kbp

s)

Class 1

Class 2

Class 3

0

50

100

150

200

250

300

4 8 12 16 20 24

Hosts in Class 1

Ban

dwid

th (

Kbp

s)

Class 1

Class 2

Class 3

0

400

800

1200

1600

2000

0 100 200 300 400 500 600 700 800

Time (sec)

BW (kbps)

BW of class 1

BW of class 2

BW of class 2

Bandwidth Sharing

Bandwidth Sharing

Weighted FairnessWeighted Fairness

Ban

dwid

th p

er u

ser

(Kbp

s)

Class-basedUser-based

Users in Class 1 Users in Class 1

BW1<BW2 or BW3

45 /482007/12/12

2.95

6.78

11.91

1.34

1.68

1.50

6.768.83

0

4

8

12

16

20

Sec

transmission time

Q time

Class 1 Class 2 Class 3 Average No Scheduling

User-Perceived Latency Lower congestion Lower transmission time

Client send

request

Gateway

getrequest

Gateway

sendrequest

Gateway

getrespons

e

Clientget

response

Higher # of concurrent connectionsHigher loss rate

46 /482007/12/12

Experimental Results User-perceived Latency

Squid Squid with MSF-RS

ms/request 1686.1 1174.9

(includes queuing time 515.5)

Throughput

2 Mbps 10 Mbps

10 Classes MSF-RS Squid 22.4 28.17

100 Classes MSF-RS Squid 23.01 29.02

Original Squid 31.61 42.45

Lower CPU loading due to fewer concurrent transactions in MSF-RS

Percentage on CPU utilization

user space127.0.0.1:3128

kernel spaceLinux

MSF-RSSquid

Realistic Servers in Internet

eth0eth1

192.168.2.53

Port Redirect (iptables): iptables -t nat -A PREROUTING -i eth1 -s 192.168.2.0/24 -p tcp --dport 80 -j REDIRECT --to-port 3128

Rate Limiting by switch: Input 2Mbps / Output 2Mbps

switch

Avalanche(Clients)

user space127.0.0.1:3128

kernel spaceLinux

MSF-RSSquid

Realistic Servers in Internet

eth0eth1

192.168.2.53

Port Redirect (iptables): iptables -t nat -A PREROUTING -i eth1 -s 192.168.2.0/24 -p tcp --dport 80 -j REDIRECT --to-port 3128

Rate Limiting by switch: Input 2Mbps / Output 2Mbps

switch

Avalanche(Clients)

47 /482007/12/12

Summary for MSF-RS Scheduling uplink requests -> Control Downlink Responses MSF-RS= Minimum-service Order Arbiter (MOA) +

Window-based Rate Control (WRC)

User-based weighted fairness Bandwidth Sharing among classes Reduce 20~30% of user-perceived Latency Reduce 25% of CPU loading

Low congestion Fewer concurrent transactions

48 /482007/12/12

Dissertation ConclusionsPublic Fairness:1. Taxonomy and evaluation of 8 TCP-friendly schemes

TCP-equivalence and TCP equal-share Rate-based fairness +

historical/super-linear aggressiveness + fixed history responsiveness TFRC: if meeting TCP-compatibility is the major concern SIMD: if fast aggressiveness is favorable

2. The design of WARC RTE control -> Non-periodic Fairness, Fast aggressiveness as SIMD History-reset procedure -> Fast Responsiveness as TFRC Better Meeting TCP-equivalence and TCP equal-share Smoothness in short-term for interactive constrained streaming

Private Fairness: The design of MSF-RS

Scheduling Uplink Requests to Manage Downlink Responses User-based Weighted Fairness High utilization while reducing 30% of user-perceived latency Reducing 25% of CPU loading

49 /482007/12/12

References[FF99] Floyd, S., and Fall, K., “Promoting the Use of End-to-End Congestion Control in the Internet.” IEEE/ACM Transaction

s on Networking, August 1999.[PFT98] J. Padhye, V. Firoiu, D. Towsley, and J. Kurose, “Modeling TCP throughput: A simple model and its Empirical Validat

ion,” in Proc. of ACM SIGCOMM’98, Sep. 1998, pp. 303-314.[YL00] Y. Yang and S. Lam, “General AIMD Congestion Control,” in Proc. of IEEE ICNP 2000, Nov 2000, pp. 187-198.[FHP00] S. Floyd, M. Handley, J. Padhye, and J. Widmer, “Equation-based Congestion Control for Unicast Applications,” in

Proc. of ACM SIGCOMM’00, Aug 2000, pp. 43-56.[ROY00] I. Rhee, V. Ozdemir, and Y. Yi, “TEAR: TCP Emulation at Receivers-Flow Control for Multimedia Streaming,” Tech.

Rep., Department of Computer Science, NCSU, Apr. 2000.[JGM03] Shudong Jin, Liang Guo, Ibrahim Matta, and Azer Bestavros, "A Spectrum of TCP-friendly Window-based Congestion

Control Algorithms," IEEE/ACM Transactions on Networking, vol. 11, no. 3, pp. 341-355, June 2003.[BB01] D. Bansal and H. Balakrishnan., “Binomial Congestion Control Algorithms,” In Proc. of IEEE INFOCOM, April 2001,

pp. 631-640.[PKT99] J. Padhye, J. Kurose, D. Towsley, and R. Koodli., “A Model Based TCP-friendly Rate Control Protocol,” In Proc. of N

OSSDAV, June 1999.[BCC98] B. Braden, D. Clark and J. Crowcroft, “Recommendations on Queue Management and Congestion Avoidance in the Int

ernet,” RFC 2309, Apr 1998, Available at http://www.ietf.org. [BBF01] D. Bansal, H. Balakrishnan, S. Floyd, S. Shenker, “Dynamic Behavior of Slowly-responsive Congestion Control Algorithms,” in Proc. of ACM SIGCOMM’01, Aug. 2001, pp. 263-274.

[LT03] Lahanas A and Tsaoussidis V, “Exploiting the efficiency and fairness potential of AIMD-based congestion avoidance and control,” Computer Networks, vol 43, no 2, pp. 227-245, Oct. 7 2003.

[VB05] Milan Vojnovic and J.-Y. Le Boudec, "On the Long-Run Behavior of Equation-Based Rate Control," IEEE/ACM Transactions on Networking, vol. 13, no 3, pp. 568-581, June 2005.

[ZDP01] Y. Zhang, N. Duffield, V. Paxson, and S. Shenker, “On the Constancy of Internet Path Properties, “ in Proc. of ACM SIGCOMM Internet Measurement Workshop, Nov. 2001, pp 197-211.

[CLB04] Gwyn Chatranon, Miguel Labrador, and Sujata Banerjee, "Fairness of AQM Schemes for TCP-friendly Traffic" in Proc. of IEEE Globecom, Dallas, Dec. 2004, pp 721-731.

50 /482007/12/12

[AAB05] E. Altman, K. Avrachenkov, and C. Barakat, “A Stochastic Model of TCP/IP with Stationary Random Losses,” IEEE/ACM Transactions on Networking, vol. 13, no 2, pp. 356-369, April 2005.

[NS06] The Network Simulator - ns-2, 2.1b9, http://www.isi.edu/nsnam/ns/.[RHE99] R. Rejaie, M. Handley, and D. Estrin, “Rap: An End-to-end Rate-based Congestion Control Mechanism for Real-time Str

eams in the Internet,” In Proc. of IEEE INFOCOM, Mar. 1999, pp. 1337-1345.[SW00] D. Sisalem and A. Wolisz, “Lda+ tcp-friendly adaptation: A measurement and comparison study,” in Proc. of NOSSDAV,

June 2000.[WDM01] J. Widmer, R. Denda and M. Mauve, “A survey on TCP-friendly congestion control,” Special Issue of the

IEEE Network ``Control of Best Effort Traffic”, vol. 15, no 3, pp. 28-37, May/June 2001.[FJ95] Floyd, S., and Jacobson, V., “Link-sharing and Resource Management Models for Packet Networks,” IEEE/ACM Transa

ctions on Networking, Vol. 3 No. 4, pp. 365-386, August 1995.[WTL04] H. Y. Wei, S. C. Tsao, and Y. D. Lin, “Assessing and Improving TCP Rate Shaping Over Edge Gateways,” IEEE Trans.

on Computer, vol. 53, issue 3, pp. 259-275, March 2004.[CKD02] M. Conti, M. Kumar, S. K. Das, and B. A. Shirazi, “Quality of Service Issues in Internet Web Services,” IEEE Trans. on

Computers, vol. 51, no. 6, June 2002.[PBB98] R. Pandey, J. Fritz Barnes, and R. Fritz Barnes, “Supporting Quality of Service in HTTP Servers,” Proceedings of the Se

venteenth Annual ACM Symposium on Principles of Distributed Computing, pp. 247-256, 1998.[BBK00] N. Bhatti, A. Bouch, and A. Kuchinsky, “Integrating User-Perceived Quality into Web Server Design,” Proceedings of th

e 9th International World Wide Web Conference, 2000.[CP99] L. Cherkasova and P. Phaa, “Session Based Admission Control: a Mechanism for Web QoS,” Proceedings of the Internat

ional Workshop on Quality of Service, 1999.[CCC02] V. Cardellini, E. Casalicchio, M. Colajanni, and M. Mambelli, “Enhancing a Web-Server Cluster with Quality of Service

Mechanisms,” Proceedings of IEEE International Performance Computing and Communications Conference, 2002.[CC01] E. Casalicchio and M. Colajanni, “A Client-Aware Dispatching Algorithm for Web Clusters Providing Multiple Services,

” Proceedings of the 10th International World Wide Web Conference, 2001.[LGC01] C. Li, G. Peng, K. Gopalan, and T. Chiuch, “Performance Guarantee for Cluster-Based Internet Services,” State Universit

y of Stony Brook, May 2001. [PG93] A. K. Parekh and R. G. Gallager, “A Generalized Processor Sharing Approach to Flow Control in Integrated Services Net

works: The Single-Node case,” IEEE/ACM Trans. on Networking, pp. 344-357, June 1993.[SQI06] Squid Web Proxy Cache, http://www.squid-cache.org/.[SV96] M. Shreedhar and G. Varghese, “Efficient Fair Queuing Using Deficit Round-Robin,” IEEE/ACM transactions on networ

king vol. 4, no. 3, June 1996.[GOL94] J. Golestani, “A Self-Clocked Fair Queueing Scheme for Broadband Applications,” Proceedings of the IEEE INFOCOM,

Toronto, June 1994.

51 /482007/12/12

[GVC96] P. Goyal, H. Vin, and H. Chen, “Start-Time Fair Queueing: A Scheduling Algorithm for Integrated Services Packet Switching Networks,” Proceedings of the ACM SIGCOMM, August 1996.

[BC98] P. Barford and M. Crovella, “Generating Representative Web Workloads for Network and Server Performance Evaluation,” ACM SIGMETRICS Performance Evaluation Review, vol. 26, issue 1, pp. 151-160, June 1998.

[AVA06] Avalanche, http://www.spirentcom.com/analysis/product_line.cfm?PL =32.[NFT06] The netfilter/iptables project, http://www.netfilter.org/.[TL01] Shih-Chiang Tsao and Ying-Dar Lin, “Pre-order Deficit Round Robin: A New Scheduling Algorithm for Packet-switche

d Networks,” Computer Networks, Vol. 35(2-3), pp. 287-305, 2001. [TZ05] V. Tsaoussidis and C. Zhang, “The Dynamics of Responsiveness and Smoothness in Heterogeneous Networks,” IEEE J

SEL AREA COMM, vol. 23, no. 6, pp. 1178-1189, June 2005.[MMF96] M. Mathis, J. Mahdavi, S. Floyd, and A. Romanow, “TCP Selective Acknowledgment Options, “ RFC 2018, April 1996.[F00] S. Floyd, “Congestion Control Principles,” RFC 2914, Sept. 2000.[NK04] N. Itaya and S. Kasahara, “Dynamic Parameter Adjustment for Available-bandwidth Estimation of TCP in Wired-wireles

s Networks,” Computer Communications, vol. 27 no. 10, pp. 976-988, June 2004.[G04] S. Gorinsky, "Feedback Modeling in Internet Congestion Control", in Proc. of Next Generation Teletraffic and Wired/Wi

reless Advanced Networking (NEW2AN 2004), Feb. 2004, pp. 231-234.[BP95] L. Brakmo and L. Peterson, “TCP Vegas: End to End Congestion Avoidance on a Global Internet,” IEEE Journal on Sele

cted Areas in Communication, vol 13, no. 8, Oct. 1995, pp. 1465-1480.[ZT06] C. Zhang and V. Tsaoussidis, "TCP Smoothness and Window Adjustment Strategy", IEEE Transactions on Multimedia,

vol 8, no. 3, pp. 600-609, June 2006.[KHF06] E. Kohler, M. Handley, and S. Floyd, “Designing DCCP: congestion control without reliability,” ACM SIGCOMM Comp

uter Communication Review, vol. 36, no. 4, Sept. 2006.[GTC06] L. Guo, E. Tan, S. Chen, Z. Xiao, O. Spatscheck and X. Zhang, “Delving into Internet Streaming Media Delivery: A Qua

lity and Resource Utilization Perspective,” in Proc. of ACM IMC’06, Oct. 2006. [JID04] S. Jaiswal, G. Iannaccone, C. Diot, J. Kurose, Towsley, D., “Inferring TCP connection characteristics through passive me

asurements,” IEEE INFOCOM 2004, Mar. 2004, pp. 1582-1592.[RKL01] Y. R. Yang, M. S. Kim, and S. S. Lam, “Transient behavior of TCP friendly congestion control protocols,” in Proc. of IE

EE INFOCOM, Apr. 2001, pp. 1716-1725.[JLH07] D. Wei, C. Jin, S. H. Low, and S. Hegde, “FAST TCP: Motivation, Architecture, Algorithms, Performance, “ IEEE/ACM

Transactions on Networking, vol. 14, no. 6, pp. 1246-1259, 2007.[TWH05] K. A. Tang, J. Wang, S. Hegde, and S. H. Low, “Equilibrium and Fairness of Networks Shared by TCP Reno and FAST”,

Telecommunications Systems special issue on High Speed Transport Protocols, vol. 30, no. 4, pp. 417-439, Dec. 2005.

52 /482007/12/12

[LPW03] S. H. Low, F. Paganini, J. Wang, and J. Doyle, “Linear Stability of TCP/RED and a Scalable Control,” Computer Networks Journal, vol. 43, no. 5, pp. 633-647, Dec. 2003.

[KHR02] D. Katabi, M. Handley, and C. Rohrs, “Congestion Control for High Bandwidth-Delay Product Networks,” in Proc. of ACM SIGCOMM’02, Aug. 2002, pp. 89-102.

[XSS05] Y. Xia, L. Subramanian, I. Stoica, and S. Kalyanaraman, "One More Bit Is Enough", in Proc. of ACM SIGCOMM’05, Aug. 2005

[PG93] A. K. Parekh and R. G. Gallager, “A Generalized Processor Sharing Approach to Flow Control in Integrated Services Networks: The Single-Node case,” IEEE/ACM Trans. on Networking, pp. 344-357, June 1993.

[SF03] Sally Floyd, “HighSpeed TCP for Large Congestion Windows,” RFC 3649, Dec. 2003.[YCC07] Cheng-Yuan Ho, Yi-Cheng Chan, and Yaw-Chung Chen, “WARD: A Deterministic Fluid Model,” IET Communications,

Vol. 1, Issue 4, pp. 711-717, August 2007. [YCC06] Cheng-Yuan Ho, Yi-Cheng Chan, and Yaw-Chung Chen, “Gallop-Vegas: An Enhanced Slow-Start Mechanism for TCP

Vegas,” Journal of Communications and Networks, Vol. 8, No. 3, pp. 351-359, Sept. 2006.