Symbiotic Routing in Future Data Centers

Symbiotic Routing in Future Data Centers

Hussam Abu-LibdehPaolo CostaAntony RowstronGreg O’SheaAustin DonnellyMICROSOFT RESEARCH

Presented By Deng Xiang

Hussam Abu-LibdehPaolo CostaAntony RowstronGreg O’SheaAustin DonnellyMICROSOFT RESEARCH

Presented By Deng Xiang

OUTLINE

• Part I: introduction and background

• Part II: four services that use customized routing protocols and their evaluations

• Part III: the extended routing service

• Part IV: conclusion

• Part V: some questions

Part I introduction and background

• Definition of symbiotic routing• Multiple routing protocols running

concurrently, all working with, and benefiting from the base routing protocol

• Services use their own routing protocols to handle the common cases and rely on the base routing protocol the handle the corner cases

• Why we need symbiotic routing?• The networks in large-scale data centers

adopt principles evolved from Internet networking

• The end-systems are isolated from the network and have little control over packets routing

• Building efficient services is hard

Part I introduction and background

Part I

• CamCube: a new data center architecture

• Targets at shipping container sized data centers• Use a 3D torus• Each server connects to other servers directly

without using any switches or routers• Each server is assigned an (x,y,z) coordinate and it

is fixed for the life time of the server

the simplest CamCube

Part I introduction and background

For each server in CamCube…

CamCube API

Core services

other services

Part I introduction and background

• CamCube API• Exposes the 3D coordinate space to services

running on CamCube• Allow sending and receiving of packets to/from

one-hop routing• Allow a service to queue packets on the queue

and query the queue length

Part I introduction and background

• Core services• Expose the server coordinates of local server and

its one-hop neighbors• Monitor liveness of one-hop neighbors and inform

other services server or link failure• The mutiple-hop routing service(the base routing

service)• The ability to perform per-hop inspection and

modification of packets• Key-based routing

Part I introduction and background

• Simulator

• Use 27(3*3*3) servers to simulate 8,000 servers(20*20*20)

• 1 Gbps links

Part I introduction and background

Part II four services with their own routing protocols

• TCP/IP service• Intercept all the packets from the OS’s TCP/IP

stack, tunnel them across CamCube and inject them into the TCP/IP stack at the destination

• Aim at achieving the maximum throughput between source and destination

Part II four services with their own routing protocols

• If use the base routing service…• Out-of-order packet delivery• Congestion• Packet loss• Decrease end-to-end throughput

Part II four services with their own routing protocols

• A custom routing protocol• Step 1: the source determines the set of outbound links

that lies on a shortest path• Step 2: all the links are used to route packets out of the

source• Step 3: at each hop a packet is greedily routed towards the

same axis and if progress cannot be made, another axis is selected

Part II four services with their own routing protocols

• Evaluation• The custom routing protocol versus the base routing service• A single server acts as the source and 2,000 randomly

selected servers act as the destination• 1,500-byte packet and each destination receives 10,000

packets• Iterate over 2,000 servers sequentially

Part II four services with their own routing protocols

Part II four services with their own routing protocols

• Aggregation service• Similar to the reduce step of MapReduce• To aggregate results of data parallel computation

distributed on servers

Part II four services with their own routing protocols

• MapReduce

master

node

N servers

R servers

split a data set into chunks and makes

them equally distributed to N servers

Aggregates the results to R servers

Sends back the total result

Part II four services with their own routing protocols

• N servers(worker nodes)• Each server has a set of key-value pairs and the keys in the set may be

different• Multiple servers can have key-value pairs that have the same key• After computation, send the key-value pairs to the servers to which

keys map

• R servers(reducers)• After receiving all the key-value pairs, combine all the

values for the same key • Send the total result back to the master node

Part II four services with their own routing protocols

• How to aggregate on path• We know the coordinate of the source and the destination• Route the packets from the source server to a destination

server on a deterministic path• Tha path is made deterministic by fixing each of the axes in

turn• At each hop, the service locally buffers the packet in case of

congestion

Part II four services with their own routing protocols

• Advantages• Reduce the amount of data transferred by performing on-path

aggregation• Ensure a good distribution of computational load over the

servers• Use less links and reduce the link load

Part II four services with their own routing protocols

• Evaluation• The aggregation service versus the base routing service• Each server generates a single packet with a key-value pair• The Full situation• The None situation

Part II four services with their own routing protocols

Part III the extended routing service

The extended routing service makes it simple and efficient for services to specify their own routing protocols.

Services themselves need to provide:1) A per-service function F• Operate in the coordinate• Define a set of key coordinates through which packets are

routed2) A policy controls queuing of packets for the service

The base routing service need to provide:1) A component that provide failure information2) A forwarding engine that forward packets towards a specified

key coordinate

Part III the extended routing service

Part III the extended routing service

1. A service S creates a packet and hands it to the extended routing service (ERS).

2. ERS call the function F associated with S, return a set of possible next key coordinates.

3. The packet is routed towards a server responsible for one of the coordinates in the set.

4. The base routing service is in charge of failure.

Part III the extended routing service

Examples:• The function F for the aggregation service

defines a deterministic path through the key coordinate space.

• The function F used by the TCP/IP service at the source returns the key coordinates of all one-hop neighbors.

Part IV: conclusion

• Individual services can obtain better performance by utilizing their own protocols.

• The network load is reduced for all services uses their own optimized protocols.

• An extended routing service is feasible to make other services implement their own routing protocols easily.

Thank you!