CoDelpresent by Van Jacobson to the

IETF-84 Transport Area Open Meeting30 July 2012

Vancouver, Canada

���2

���3

Sender Receiver

���4

Sender Receiver

���5

Sender Receiver

• Queue forms at a bottleneck

• There’s probably just one bottleneck(each flow sees exactly one)

➡ Choices: can move the queue (by making a new bottleneck) or control it.

���5

Time

Que

ue le

ngth

Good Queue / Bad Queue

���6

Time

Que

ue le

ngth

Time

Que

ue le

ngth

Good Queue / Bad Queue

���6

Time

Que

ue le

ngth

Time

Que

ue le

ngth

Good Queue / Bad Queue

• Good queue goes away in an RTT, bad queue hangs around.

➡ queue length min() over a sliding window measures bad queue ...

➡ ... as long as window is at least an RTT wide.

���7

Time

Que

ue le

ngth

Time

Que

ue le

ngth

Good Queue / Bad Queue

• Good queue goes away in an RTT, bad queue hangs around.

➡ tracking min() in a sliding window gives bad queue ...

➡ ... as long as window is at least an RTT wide.

���8

Time

Que

ue le

ngth

Time

Que

ue le

ngth

Good Queue / Bad Queue

• Good queue goes away in an RTT, bad queue hangs around.

➡ tracking min() in a sliding window gives bad queue ...

➡ ... as long as window is at least an RTT wide.

���8

How big is the queue?

• Can measure size in bytes – interesting if worried about overflow – requires output bandwidth to compute delay

• Can measure packet’s sojourn time – direct measure of delay – easy (no enque/deque coupling so works with any packet pipeline).

���9

Sojourn Time

• Works with time-varying output bandwidth (e.g., wireless and shared links)

• Better behaved than queue length – no high frequency phase noise

• Includes everything that affects packet so works for multi-queue links

���10

24.5 25.0 25.5 26.0

01

23

45

6

Time (sec.)

Q s

ize (m

s.)

24.5 25.0 25.5 26.0

01

23

45

6

Time (sec.)

Q d

elay

(ms.

)Two views of a Queue

Top graph is sojourn time, bottom is queue size.

(one ftp + web traffic; 10Mbps bottleneck;

80ms RTT; TCP Reno)���11

24.5 25.0 25.5 26.0

01

23

45

6

Time (sec.)

Q s

ize (m

s.)

24.5 25.0 25.5 26.0

01

23

45

6

Time (sec.)

Q d

elay

(ms.

)Two views of a Queue

Top graph is sojourn time, bottom is queue size.

(one ftp + web traffic; 10Mbps bottleneck;

80ms RTT; TCP Reno)���11

Two views of a Queue

Top graph is sojourn time, bottom is queue size.

(one ftp + web traffic; 10Mbps bottleneck;

80ms RTT; TCP Reno)25.00 25.05 25.10 25.15 25.20

0.0

0.5

1.0

1.5

2.0

2.5

Time (sec.)

Q s

ize (m

s.)

25.00 25.05 25.10 25.15 25.20

0.0

0.5

1.0

1.5

2.0

Time (sec.)

Q d

elay

(ms.

)

���12

Multi-Queue behavior

���13

a)Measure what you’ve got

b)Decide what you want

c)If (a) isn’t (b), move it toward (b)

Controlling Queue

���14

a)Measure what you’ve got

b)Decide what you want

c)If (a) isn’t (b), move it toward (b)

Controlling Queue

- Estimator

- Setpoint

- Control loop

���15

How much ‘bad’ queue do we want?

• Can’t let the link go idle (need one or two MTU of backlog)

• More than this will give higher utilization at low degree of multiplexing (1-3 bulk xfers) at the cost of higher delay

• Can the trade-off be quantified?

���16

0 20 40 60 80 100

7580

8590

9510

0

Utilization vs. Target for a single Reno TCP

Target (% of RTT)

Bottl

enec

k Li

nk U

tiliz

atio

n (%

of m

ax)

���17

0 20 40 60 80 100

0.70

0.75

0.80

0.85

0.90

0.95

1.00

Power vs. Target for a Reno TCP

Target (as % of RTT)

Aver

age

Powe

r (Xp

ut/D

elay

)

���18

0 20 40 60 80 100

0.70

0.75

0.80

0.85

0.90

0.95

1.00

Power vs. Target for a Reno TCP

Target (as % of RTT)

Aver

age

Powe

r (Xp

ut/D

elay

)

���18

0 5 10 15 20 25 30

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1.00

Power vs. Target for a Reno TCP

Target (as % of RTT)

Aver

age

Powe

r (Xp

ut/D

elay

)

���19

0 5 10 15 20 25 30

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1.00

Power vs. Target for a Reno TCP

Target (as % of RTT)

Aver

age

Powe

r (Xp

ut/D

elay

)

���19

���20

• Setpoint target of 5% of nominal RTT (5ms for 100ms RTT) yields substantial utilization improvement for small added delay.

���20

• Setpoint target of 5% of nominal RTT (5ms for 100ms RTT) yields substantial utilization improvement for small added delay.

• Result holds independent of bandwidth and congestion control algorithm (tested with Reno, Cubic & Westwood).

���20

• Setpoint target of 5% of nominal RTT (5ms for 100ms RTT) yields substantial utilization improvement for small added delay.

• Result holds independent of bandwidth and congestion control algorithm (tested with Reno, Cubic & Westwood).

➡ CoDel has no free parameters: running-min window width determined by worst-case expected RTT and target is a fixed fraction of same RTT.

���20

Algorithm & Control Law

(see I-D, CACM paper and Linux kernels >= 3.5)

���21

• provides isolation - protects low rate CBR and web for a better user experience. Makes IW10 concerns a non-issue.

• gets rid of bottleneck bi-directional traffic problems (‘ack-compression’ burstiness)

• improves flow mixing for better network performance (reduce HoL blocking)

Eric Dumazet has combined CoDel with a simple SFQ (256-1024 buckets with RR service discipline). Cost in

state & cycles is small and improvement is big.

➡ Since we’re adding code, add fqcodel, not codel.���22

• thanks to Jim Gettys and the ACM, have dead-tree publication to protect ideas

• un-encumbered code (BSD/GPL dual-license) available for ns2, ns3 & linux

• in both simulation and real deployment, CoDel does no harm – it either does nothing or reduces delay without affecting xput.

Where are we?

���23

What needs to be done• Still looking at parts of the algorithm but

changes likely to be 2nd order.

• Would like to see CoDel on both ends of every home/small-office access link but:

- We need to know more about how traffic behaves on particular bottlenecks (wi-fi, 3G cellular, cable modem)

- There are system issues with deployment

���24

Deployment Issues

RTR/AP Cable Modem HeadendHome

Gateway

���25

Deployment Issues

RTR/AP Cable Modem HeadendHome

Gateway

Protocol stack

DeviceDriver DeviceLinux

kernel

���26

Deployment Issues

RTR/AP Cable Modem HeadendHome

Gateway

Protocol stack

DeviceDriver DeviceLinux

kernel

PhoneCPU

3G Modem RAN SGSN?Cellphone

���27

Our thanks to:

• Jim Gettys

• CoDel early experimenters, particularly Dave Taht

• Eric Dumazet

• ACM Queue

• Eben Moglen

���28