Upload
nguyenduong
View
222
Download
0
Embed Size (px)
Citation preview
Ýmir VigfússonGregory ChocklerYoav Tock
Rx for Data Center Communication Scalability
Hussam Abu-LibdehRobert BurgessKen BirmanHaoyuan Li
Mahesh Balakrishnan
IBM Research, Haifa Labs
Cornell University Microsoft Research,Silicon Valley
Useful
– IPMC is fast, and widely supported
– Multicast and pub/sub often used implicitly
– Lots of redundant traffic in data centers
[Anand et al. SIGMETRICS ’09]
Rarely used
– IP Multicast has scalability problems!
IP Multicast in Data Centers
IP Multicast in Data Centers
• Switches have limited state space
Switch model (10Gbps) Group capacity
Alcatel-Lucent OmniSwitch OS6850-4 260
Cisco Catalyst 3750E-48PD-EF 1,000
D-Link DGS-3650 864
Dell PowerConnect 6248P 69
Extreme Summit X450a-48t 792
Foundry FastIron Edge X 448+2XG 511
HP ProCurve 3500yl 1,499
IP Multicast in Data Centers
• NICs also have limited state space
E.g. 16 exact addresses 512-bit Bloom filter
• Packet loss triggers further problems
–Reliability layer may aggravate loss
–Major companies have sufferedmulticast storms
IPMC has dangerous
scalability issues
IP Multicast in Data Centers
Key ideas
• Treat IPMC groups as a scarce resource– Limit the number of physical IPMC groups– Translate logical IPMC groups into either physical
IPMC groups or multicast by iterated unicast.
• Merge similar groups together
Dr. Multicast
• Transparent: Standard IPMC interface to user, standard IGMP interface to network.
• Robust: Distributed, fault-tolerant service.
• Optimizes resource use: Merges similar multicast groups together.
• Scalable in number of groups: Limits number of physical IPMC groups.
Dr. Multicast
Dr. Multicast
• Library maps logical IPMC to physical IPMC or iterated unicast
• Transparent to the application
– IPMC calls intercepted and modified
• Transparent to the network
– Ordinary IPMC/IGMP traffic
• Transparent: Standard IPMC interface to user, standard IGMP interface to network.
• Robust: Distributed, fault-tolerant service.
• Optimizes resource use: Merges similar multicast groups together.
• Scalable in number of groups: Limits number of physical IPMC groups.
Dr. Multicast
Dr. Multicast
• Per-node agent maintains global group membership and mapping
– Library consults local agent
• Leader agent periodically computes new mapping (see later).
• State reconciled via gossip
Library Layer Overhead
• Experiment measuring sends/sec at one sender• Sending to r addresses realizes roughly 1/r ops/sec• Insignificant overhead when mapping logical IPMC group to
physical IPMC group.
Network Overhead and Robustness
• Experiment on 90 Emulab nodes
Nodes introduced 10 at a time.Total network traffic grows linearly.
Average traffic received per-node.Robust to major correlated failure
Half of thenodes die
• Transparent: Standard IPMC interface to user, standard IGMP interface to network.
• Robust: Distributed, fault-tolerant service.
• Optimizes resource use: Merges similar multicast groups together.
• Scalable in number of groups: Limits number of physical IPMC groups.
Dr. Multicast
Optimization Questions
Assign IPMC and unicast addresses s.t. Min. receiver filtering Min. network traffic Min. # IPMC addresses … yet deliver all messages to interested parties
Optimization Questions
Assign IPMC and unicast addresses s.t. receiver filtering network traffic # IPMC addresses (hard)M
)1(
• Knob to control relative costs of CPU filtering and of duplicate traffic.
• Both and are part of administrative policy.
M
MCMD Heuristic
Groups in `user-interest’ space
GRAD STUDENTS FREE FOOD
(1,1,1,1,1,0,1,0,1,0,1,1)(0,1,1,1,1,1,1,0,0,1,1,1)
MCMD Heuristic
Groups in `user-interest’ space
Grow M meta-groups around the groups greedily while cost decreases
MCMD Heuristic
Groups in `user-interest’ space
Grow M meta-groups around the groups greedily while cost decreases
• Social:
– Yahoo! Groups
– Amazon Recommendations
– Wikipedia Edits
– LiveJournal Communities
– Mutual Interest Model
Data sets/models
Users Groups
MCMD Heuristic
• Total cost on samples of 1000 logical groups.
– Costs drop exponentially with more IPMC addresses
• Social:
– Yahoo! Groups
– Amazon Recommendations
– Wikipedia Edits
– LiveJournal Communities
– Mutual Interest Model
• Systems:
– IBM Websphere
Data sets/models
Users Groups
MCMD Heuristic
• Total cost on IBM Websphere data set (simulation)
– Negligible costs when using only 4 IPMC addresses
• Transparent: Standard IPMC interface to user, standard IGMP interface to network.
• Robust: Distributed, fault-tolerant service.
• Optimizes resource use: Merges similar multicast groups together.
• Scalable in number of groups: Limits number of physical IPMC groups.
Dr. Multicast
Group Scalability
• Experiment on Emulab with 1 receiver, 9 senders• MCMD prevents ill-effects when the # of groups scales up
IPMC is useful, but has scalability problems
Dr. Multicast treats IPMC groups as scarce and sensitive resources
– Transparent, backward-compatible
– Scalable in the number of groups
– Robust against failures
– Optimizes resource use by merging similar groups
• Enables safe and scalable use of multicast
Dr. Multicast