Not All Microseconds are Equal:Fine-Grained Per-Flow Measure-

ments with Reference Latency Inter-polation

Myungjin Lee†, Nick Duffield‡, Ramana Rao Kompella†

†Purdue University, ‡AT&T Labs–Research

2

Low-latency applications

Several new types of applications require ex-tremely low end-to-end latency Algorithmic trading applications in financial data

center networks High performance computing applications in data

center networks Storage applications

Low latency cut-through switches Arista 7100 series Woven EFX 1000 series

3

… … … …

ToR S/W

Edge Router

Core Router

…Need for high-fidelity measurements

At every router, high-fidelity measurements are critical to localize root causes

Once root cause localized, operators can fix by rerouting traffic, upgrade links or perform detailed diagnosis

Which router causes the prob-

lem??

1ms

Router

Measurement within a router is

necessary

4

Measurement solutions today

SNMP and NetFlow No latency measurements

Active probes Typically end-to-end, do not localize the root cause

Expensive high-fidelity measurement box Corvil boxes (£ 90,000): used by London Stock Exchange Cannot place these boxes ubiquitously

Lossy Difference Aggregator (LDA) [Kompella, SIG-COMM’09] Provides average latency and variance at high-fidelity

within a switch Provides a good start but may not be sufficient to diag-

nose flow-specific problems

5

Motivation for per-flow measurements

Key observation: Significant amount of difference in average latencies across flows at a router

Dela

y

Time

S/W

…

Queue

Average la-tency

Measurement period

Large de-lay

Small de-lay

6

Outline of the rest of talk

Measurement model

Alternative approaches

Intuition behind our approach: Delay locality

Our architecture: Reference Latency Interpola-tion (RLI)

Evaluation

7

Measurement model

Assumption: Time synchronization between router interfaces Constraint: Cannot modify regular packets to carry time-

stamps Intrusive changes to the routing forwarding path Extra bandwidth consumption up to 10% capacity

Router

Ingress I

Egress E

8

Naïve approach

For each flow key, Store timestamps for each packet at I and E After a flow stops sending, I sends the packet timestamps

to E E computes individual packet delays E aggregates average latency, variance, etc for each flow

Problem: High communication costs At 10Gbps, few million packets per second Sampling reduces communication, but also reduces accu-

racy

Ingress I Egress E

10

− =2023

2730

+151

318

−=

2232Avg. delay = 22/2

= 11Avg. delay = 32/2 = 16

−+ −

9

A (naïve) extension of LDA

Maintaining LDAs with many counters for flows of interest

Problem: (Potentially) high communication costs Proportional to the number of flows

Ingress I Egress E

LDA

LDA

LDA

LDA

LDA

LDA

28

15

2

1

Packet

count

Sum of time-

stamps

…

Coordina-tion

Per-flow la-tency

10

Key observation: Delay locality

LocaTrue mean delay = W(D1 + WD2 + WD3) / 3

Localized mean delay = (WD1 + WD2 + WD3) / 3

WD1 WD3WD2

How close is localized mean delay to

true mean delay as window size varies?

Dela

y

Time

D1

D2D3

11

Key observation: Delay locality

True Mean delay per key / ms

Loca

l m

ean

dela

y p

er

key /

ms

Global Mean

0.1ms: RMSRE=0.054

10ms: RMSRE=0.16

1s: RMSRE=1.72

Data sets from real router and synthetic queueing model

12

Exploiting delay locality

Reference packets are injected regularly at the ingress I Special packets carrying ingress timestamp Provide some reference delay samples Used to approximate the latencies of regular packets

Dela

y

Time

ReferencePacket

IngressTimestamp

13

RLI architecture

Component 1: Reference Packet generator Injects reference packets regularly

Component 2: Latency Estimator Estimates packet latencies and updates per-flow statis-

tics Estimates directly at the egress with no extra state main-

tained at ingress side (reduces storage and communica-tion overheads)

Egress E

Ingress I

1) ReferencePacket

Generator

2) LatencyEstimator

123 123

RL

IngressTimestamp

14

Component 1: Reference packet generator

Question: When to inject a reference packet ?

Idea 1: 1-in-n: Inject one reference packet every n packets Problem: low accuracy under low utilization

Idea 2: 1-in-τ: Inject one reference packet every τ seconds Problem: bad in case where short-term delay variance is

high Our approach: Dynamic injection based on utilization

High utilization low injection rate Low utilization high injection rate Adaptive scheme works better than fixed rate schemes

15

Component 2: Latency estimator

Question 1: How to estimate latencies using reference packets

Solution: Different estimators possible Use only the delay of a left reference packet (RLI-L) Use linear interpolation of left and right reference packets (RLI) Other non-linear estimators possible (e.g., shrinkage)

LInterpolated

delay

Dela

y

Time

Error indelay estimate

RegularPacket

ReferencePacket

Linear interpolationline

Arrival time is known

Arrival time and delay are

known

Estimateddelay

Error indelay estimate

R

16

Component 2: Latency estimatorFlowkey C1 C2 C3

8 11 39

2 3 6

Interpolation buffer

Estimate

10

20

80

3 4 7

Avg. latency = C2 / C1

RL

Right Reference Packet arrived

When a flow is ex-

ported

Question 2: How to compute per-flow latency statistics Solution: Maintain 3 counters per flow at the egress side

C1: Number of packets C2: Sum of packet delays C3: Sum of squares of packet delays (for estimating variance) To minimize state, can use any flow selection strategy to main-

tain counters for only a subset of flows

Flow Key4 51Delay

Square of de-

lay

16

25

1

Update

Any flow se-lection

strategy

Up-date

Selection

17

Experimental environment

Data sets No public data center traces with timestamps Real router traces with synthetic workloads: WISC Real backbone traces with synthetic queueing: CHIC

and SANJ

Simulation tool: Open source NetFlow software – YAF Supports reference packet injection mechanism Simulates a queueing model with RED active queue

management policy

Experiments with different link utilizations

18

Accuracy of RLI under high link utilization

Relative error

CD

F

Median relative erroris 10-12%

19

Comparison with other solutions

Utilization

Avera

ge r

ela

tive e

rror

Packet sampling rate = 0.1%

1-2 orders of magnitude dif-

ference

20

Overhead of RLI

Bandwidth overhead is low less than 0.2% of link capacity

Impact to packet loss is small Packet loss difference with and without RLI is at

most 0.001% at around 80% utilization

21

Summary

A scalable architecture to obtain high-fidelity per-flow latency measurements between router interfaces

Achieves a median relative error of 10-12%

Shows 1-2 orders of magnitude lower relative error compared to existing solutions

Measurements are obtained directly at the egress side

Future work: Per-packet diagnosis

22

Thank you! Questions?

23

Backup

24

Comparison with other solutions

Relative error

CD

F

25

Bandwidth overhead

Utilization

Band

wid

th c

onsu

mpti

on

26

Interference with regular traffic

Per-flow delay interference (seconds)

Cum

ula

tive f

ract

ion

27

Impact to packet lossesLo

ss r

ate

diff

ere

nce

Utilization