Towards Content Distribution Networks with Latency Guarantees

Chengdu Huang and Tarek F. Abdelzaher

University of Virginia

Outline Background Challenges Contributions

Formulation Architecture Evaluation

Conclusion

Overview Our goal is to guarantee a subsecond upper bound

on response time Latency bound formula specified in contract with content

provider Existing research: optimizing average response time

Client perceived latency consists of Latency from client to a CDN server Latency from a CDN server to some other CDN server

(request forwarding) Processing time within CDN servers

Need and Feasibility -- Observations from the Internet Establishing the Need: (Latency Analysis)

Average latency for web objects are a significant fraction of a second

A large portion of latencies exceed a second Bounded-delay CDN is needed

Establishing the Feasibility: (Cost Analysis) Internet latencies (for a fixed pair) are not time invariant

but only oscillate with a small range Spikes are not very common Can be attributed to underutilization of Internet backbone Replica locations are relatively static – maintenance cost

is low

Contributions Mapping the latency bound guarantee

problem to a well-studied graph theoretic problem

Designed and implemented a real-time CDN system on a WAN platform

Extensive evaluation results drawn from an Internet deployment of the service prototype

Mapping The problem of achieving latency bound is mapped to a graph

domination problem Formulation

Given a set of CDN servers S = {S1,…, Sn}, a content object C, and its latency bound L

Construct a graph G whose vertices are S Edge SiSj is added to G iff server Si can download C from server Sj

within a time less than L To find minimal dominating set D: a subset of S with minimal

cardinality that for all u in S - D, there is a v in D for which uv is in G

Nodes represent servers, edges connect neighbors reachable within latency bound dominating set is reachable within latency bound from any server

Mapping

B

F

C

A

G

D

E

Mapping

Graph Domination Problem

B

F

C

A

G

D

E

Existing Graph Domination Algorithms Centralized greedy heuristic

Repeatedly selects the vertex with highest remaining degree

Best approximation known Distributed algorithms

DDCH (INFOCOM’00) LRG (PODC’01) Kuhn and Wattenhofer (PODC’03)

Limitations Performance in asynchronous environment Need multiple rounds to finish: long termination time

We developed a new distributed, asynchronous algorithm

Architectural Challenges The CDN system runs in a highly dynamic

and asynchronous environment How to handle content objects with different

sizes Absence of global knowledge

Challenge: Asynchronous environment Our distributed algorithm

Goal: Decentralized, asynchronous, fast termination Idea

Inspired by the centralized counterpart Nodes independently nominate the neighbor with the highest degree Receiving nomination makes a node join the dominating set and

send out dominator announcement Receiving dominator announcement makes a node refrain from

sending nomination

Insights: High degree nodes quickly join the dominating set Joining of high degree nodes quickly inhibits further nominations

Reachable

Algorithm -- example

B

F

A

G

D

C

A, B, D, E send NOMINATION to C

F, G send NOMINATION to E (random tie-breaking)

A: 4B: 3C: 5D: 2

A: 4B: 3C: 5

A: 4B: 3C: 5D: 2E: 3

A: 4C: 5D: 3

E: 3F: 3G: 3

E: 3F: 3G: 3

C: 5E: 3F: 3G: 3

A is reachable

A is reachable

A is reachable

Degree=3

Degree=5

Degree=4

Degree=2

Degree=3Degree=3

Degree=3

E

Mapping

B

F

C

A

G

D

E

Degree=3

Degree=5

Degree=4

Degree=2

Degree=3Degree=3

Degree=3

Challenge: probing objects of different sizes Probing is needed to estimated latency Latency depends on file size which can be any size,

making probing challenging Solution

Probe a series objects of certain sizes Assuming latency has a simple linear relation with object

size Use a recursive least square (RLS) estimator to estimate

the parameters and More sophisticated probing techniques can be plugged in

Challenge: objects of different sizes Validation of our latency-size model

Challenge: absence of global knowledge The system should perform well without

global knowledge Introduce a parameter: visibility

Percentage of servers in the system each server knows when it starts

Low visibility incurs more replicas Two heuristics to reduce number of replicas

Reciprocal mode Highest degree node exchange

Implementation Instrument Squid Proxy Cache Deployed on PlanetLab

PlanetLab is a WAN platform with more than 100 sites across 20+ countries

Deployed on 30~80 nodes

Experiment on PlanetLab

Evaluation outline Efficiency Latency bound guarantee Absence of global knowledge

Evaluation Number of replicas

2

3

4

5

6

set1 set2 set3 set4 set5 set6

No

rmal

ized

Ter

min

atio

n T

ime

DDCH

LRG

0

5

10

15

20

25

30nodes200ms

30nodes300ms

30nodes400ms

80nodes200ms

80nodes300ms

80nodes400ms

Nu

mb

er

of

Re

pli

ca

s

Centralized

DDCH

LRG

DG

Latency Bound:

Evaluation Termination Time

2

3

4

5

6

set1 set2 set3 set4 set5 set6

No

rmal

ized

Ter

min

atio

n T

ime

DDCH

LRG

30 Nodes 80 Nodes

Evaluation Latency bound guarantee

Baselines Single Server Random (with the same number of replicas) Average Latency Greedy (Qiu INFOCOM’00)

Evaluation Latency bound guarantee

Latency Bound Guarantee Number of Replicas

Evaluation Absence of global knowledge

Conclusion Designed and implemented a CDN system

that provides latency bound Based on a distributed algorithm that

performs well in asynchronous environment Experiment results show that latency bound

can be achieved with a very high confidence