Peer-Assisted Content Distribution Networks: Techniques and Challenges

  • View

  • Download

Embed Size (px)


Peer-Assisted Content Distribution Networks: Techniques and Challenges. Pei Cao Stanford University. Traditional Intra-Provider Content Distribution Networks. National Center. Regional Center. Branch. Users. Peer-to-Peer Content Distribution. National Center. Regional Center. Branch. - PowerPoint PPT Presentation

Text of Peer-Assisted Content Distribution Networks: Techniques and Challenges

  • Peer-Assisted Content Distribution Networks: Techniques and ChallengesPei CaoStanford University

  • Traditional Intra-Provider Content Distribution NetworksNational CenterRegional CenterBranch. . .. . . . . .Users. . . . . . . . . . . .

  • Peer-to-Peer Content DistributionNational CenterRegional CenterBranch. . .. . . . . .Users. . . . . . . . . . . .

  • P2P vs CDNP2P: No infrastructure costSupply grows linearly with demandSimple distributed, randomized algorithmsNo QoS CDN:Initial infrastructure costCentralized scheduling algorithmsNetwork efficiencyCapable of supporting QoS

  • Combine P2P with CDN?Use P2P to complement CDNP2P reduces load on the CDN, covers areas where CDN is not installedMust be able to control, or shape, P2P trafficUse CDN to complement P2PCDN steps in when peer-based distribution is falling short, enabling QoSMust be able to detect when peers wont meet the delivery time guarantee

  • OutlineReview of BitTorrent Traffic-shaping BitTorrent: biased neighbor selectionQoS in BitTorrent: delivery time prediction

  • BitTorrent File Sharing NetworkGoal: replicate K chunks of data among N nodes

    Form neighbor connection graph

    Neighbors exchange data

  • BitTorrent: Neighbor SelectionTrackerfile.torrent1SeedWhole fileA5234

  • BitTorrent: Piece ReplicationTrackerfile.torrent1SeedWhole fileA53

  • BitTorrent: Piece Replication AlgorithmsTit-for-tat (choking/unchoking):Each peer only uploads to 7 other peers at a time6 of these are chosen based on amount of data received from the neighbor in the last 20 secondsThe last one is chosen randomly, with a 75% bias toward newcomers(Local) Rarest-first replication:When peer 3 unchokes peer A, A selects which piece to download

  • Analysis of BitTorrentConclusion from modeling studies: BitTorrent is nearly optimal in idealized, homogeneous networksDemonstrated by simulation studiesConfirmed by theoretical modeling studiesIntuition: in a random graph, Prob(Peer As content is a subset of Peer Bs) 50%

  • Traffic-Shaping BitTorrent

  • Random Neighbor GraphExisting studies all assume random neighbor selectionBitTorrent no longer optimal if nodes in the same ISP only connect to each otherRandom neighbor selection high cross-ISP traffic

  • Difficulty in Traffic-Shaping P2P ApplicationsISPs: Different links have different monetary costsPrefer clustering of trafficP2P Applications: No knowledge of underlying ISP topologyUse randomized algorithms that dont do well under clusteringCurrent solution: throttling users suffer

  • A Network-Friendly BitTorrent?ISPs inform BitTorrent of its link preferencesAlgorithm of BitTorrent is adjusted such that both users and ISPs benefit

    Example: Biased Neighbor SelectionWorks when cost function is transitive

  • Biased Neighbor SelectionIdea: of N neighbors, choose N-k from peers in the same ISP, and choose k randomly from peers outside the ISPISP

  • Implementing Biased Neighbor SelectionBy TrackerNeed ISP affiliations of peersPeer to AS mapsPublic IP address ranges from ISPsSpecial X- HTTP headerBy traffic shaping devicesIntercept peer tracker messages and manipulate responsesNo need to change tracker or client

  • Evaluation MethodologyEvent-driven simulatorUse actual client and tracker codes as much as possibleCalculate bandwidth contention, assume perfect fair-share from TCPNetwork settings14 ISPs, each with 50 peers, 100Kb/s upload, 1Mb/s downloadSeed node, 400Kb/s uploadOptional university nodes (1Mb/s upload)Optional ISP bottleneck to other ISPs

  • Limitation of Throttling

  • Throttling: Cross-ISP TrafficRedundancy: Average # of times a data chunk enters the ISP

  • Biased Neighbor Selection: Download Times

  • Biased Neighbor Selection: Cross-ISP Traffic

  • Importance of Rarest-First ReplicationRandom piece replication performs badlyIncreases download time by 84% - 150%Increase traffic redundancy from 3 to 14Biased neighbors + Rarest-First More uniform progress of peers

  • Presence of External High-Bandwidth PeersBiased neighbor selection alone: Average download time same as regular BitTorrentCross-ISP traffic increases as # of university peers increaseResult of tit-for-tatBiased neighbor selection + Throttling: Download time only increases by 12%Most neighbors do not cross the bottleneckTraffic redundancy (i.e. cross-ISP traffic) same as the scenario without university peers

  • Comparison with Simple ClusteringGateway peer: only one peer connects to the peers outside the ISP, all other peers only connect to peers inside the ISPGateway peer must have high bandwidthIt is the seed for this ISPEnds up benefiting peers in other ISPs

  • Combining Biased Neighbor Selection with CachesUnder random neighbor selectionbandwidth requirement of cache is highUnder biased neighbor selectionbandwidth needed from the cache is reduced by an order of magnitude

  • ConclusionsBy choosing neighbors well, BitTorrent can achieve high peer performance without increasing ISP costBiased neighbor selection: choose initial set of neighbors wellCan be combined with throttling and caching

    BitTorrents algorithm can be shaped!

  • Delivery Time Prediction

  • MotivationProvide delivery time guarantee under P2P+CDN What contributes to delivery time of a download via BitTorrent?From simulations: seed bandwidth and even replication of blocks Missing: node join/leave dynamics, TCP effects, etc.

  • Side-by-Side Live ExperimentsTwo clients, running on the same machine, starting at the same time, downloading the same13 experiments from Apr-May 2006File sizes: 700MB ~ 1.4GBNetwork size: 1100 ~ 2100 peersDuration: 10 hrs ~ 2 days

  • Results from ExperimentsEffective download rate: 10 ~ 30KB/sSpeed difference between the two peers: 3% ~ 82%

    What made the slower peer slow?

  • Suspicion #1: Slower Neighbors?Calculate unweighted average of observed throughput at application levelR1: average from all neighborsR2: average from neighbors uploading >250KB of dataR3: average from neighbors uploading >2.5MB of dataLow correlation between download-time ratio and neighbor-speed ratio0.57 for R1, 0.43 for R2, 0.47 for R3Faster neighbors corresponds to slower downloads in 3 experiments

  • Suspicion #2: Fewer Neighbors Uploading to the Peer?Slot analysis: calculate download concurrencyMaximum number of neighbors: 35Neighbors come and go align neighbors into 35 slotsCalculate time-average of number of concurrent slots with neighbors uploadingUpload concurrency varies from 7 to 11Explains one of the download-time/neighbor-speed reversal caseBut doesnt explain the two others

  • Close Neighbors90% of data downloaded from 1-4% of neighborsLet F(p) and G(p) be the number of neighbors that provides p of data to peers F and G, then F(p) > G(p) peer F is slower than GThis holds for p = 90%, 75%, and 50%

  • What makes a neighbor close?Not related to speed, or order of connection to peer, or order of unchoking by peer

  • Cost of Departure of a Close NeighborDeparture cost: if one close neighbor leaves, calculate the time until the earliest next close neighborThe average departure cost: 30 min The convergence time of the tit-for-tat algorithm is slow

  • Why Do Close Neighbors LeaveFive possible reasonsA: Random disconnectB: Finished downloadingC: Peer broke off the relationshipD: Neighbor broke off the relationshipResults: B is most common, followed by C/D, then A

  • ConclusionsContent delivery time in BitTorrent is determined by:Neighbor upload speedStability of neighbor relationshipDisruption of the pairing leads to long delivery timeNeighbors may leave due to random disconnection, completion of download, or finding faster neighbors

  • Using CDN to Complement P2PUse nodes CDN as high-speed specially managed seedsSeeds are called to help whenever a node loses a close neighbor

  • SummaryA way to shape BitTorrent trafficPredicting BitTorrent performance by monitoring close peer relationship

  • Related WorkMany modeling studies of BitTorrentSimulation studiesMeasurements of real torrents

  • Ongoing WorkLive experiments with biased neighbor selectionsA k-regular graph algorithm with faster convergencePrototype implementation of P2P+CDN

    Guarantee that contents travel link just once if a cache is installed at the other end of the link;The application is completely oblivious of the underlying network topology;