Architectures for Peer-to-Peer Content Distribution Christos Gkantsidis Microsoft Research, Cambridge, UK Joint work with: Pablo Rodriguez, John Miller,

  • View
    216

  • Download
    2

Embed Size (px)

Text of Architectures for Peer-to-Peer Content Distribution Christos Gkantsidis Microsoft Research,...

  • Slide 1
  • Architectures for Peer-to-Peer Content Distribution Christos Gkantsidis Microsoft Research, Cambridge, UK Joint work with: Pablo Rodriguez, John Miller, Dinan Gunawardena, and others
  • Slide 2
  • Peer-to-Peer Applications Peer-to-Peer Content Distribution File Sharing Media Streaming Collaboration Distributed Computing TelephonyPlatforms BBC iMP SETI@Home.NET (PeerChannel, WCF) JXTA P2P content distribution is hugely popular: >40-60% of Internet traffic Easy access to an amazing variety of content Challenges of P2P: P2P traffic difficult to manage ISP complaints Dependence of free bandwidth e.g. from universities Massive copyright infringement 2
  • Slide 3
  • Microsoft scenarios where P2P can help Large scale software distribution: Windows / Microsoft updates Beta versions of software (e.g. Visual Studio) Free software MSN Video File sharing (e.g. Foldershare, Groove, Live Messenger) Windows XP P2P Name Resolution Protocol (PNRP) P2P Graphing and Grouping Peer Identity Manager Windows Vista PNRP, Graphing and Grouping, Peer Identity Manager, P2P Contacts, People Near Me Windows Communication Foundation Peer Channel Incidentally, MS has been interested in P2P for some time 3
  • Slide 4
  • Typical Objections P2P does not bring enough value (especially relative the risk)! I know how to deploy servers, why learn something new? How do I control it? How will I monitor it? How can I provision my network? 4
  • Slide 5
  • Typical Objections P2P does not bring enough value (especially relative the risk)! I know how to deploy servers, why learn something new? How do I control it? How will I monitor it? How can I provision my network? External objections Respect network operators (e.g. ISPs) Protect content owners Impact of firewalls, NATs, etc 5
  • Slide 6
  • Peer-to-Peer Distribution Avalanche: P2P file distribution Reduce download times, use network resources optimally Secure distribution: Publisher and content protection Only authorized users can join the distribution networks Cache-friendly distribution Enable network locality RedCarpet: P2P Video-on-Demand Watch movies while downloading without loosing delivery efficiency. New P2P capability! RewindTV: Enhanced Live TV Watch live TV and be able to re-play missed scenes (from other peers) 6
  • Slide 7
  • Peer-to-Peer Distribution Avalanche: P2P file distribution Reduce download times, use network resources optimally Secure distribution: Publisher and content protection Only authorized users can join the distribution networks Cache-friendly distribution Enable network locality 7
  • Slide 8
  • File Swarm = Block Scheduling 8
  • Slide 9
  • Systems progress in current File Swarming systems 9 (From Tian et al., Infocom06) A lot of time spent at the beginning and finish of download: Beginning of download: finding good blocks to exchange End of download: discovering the last missing blocks
  • Slide 10
  • Network Coding Simplified 10 File to Transfer Block 1Block 2Block 3 Encoding
  • Slide 11
  • With Network Coding 11
  • Slide 12
  • Systems progress 12 Smooth download progress: No start-up delay No last-block problem
  • Slide 13
  • 13 Bandwidth Contribution Easily withstands flash crowds Server contribution is fixed, Client contribution scales >10 fold savings in content providers bandwidth using peer-to-peer.
  • Slide 14
  • Results from distributing Visual Studio 14 Data from distribution of beta versions of Visual Studio 2008 Beta (Nov07) Median speeds: ~1.5Mbps for VS Pro ~2.7Mbps for the others
  • Slide 15
  • Peer-to-Peer Distribution Avalanche: P2P file distribution Reduce download times, use network resources optimally Secure distribution: Publisher and content protection Only authorized users can join the distribution networks Cache-friendly distribution Enable network locality 15
  • Slide 16
  • Strong security model All connections TLS- authenticated Data is self-certifying Block data validated by signed homomorphic hash equivalent Strong identity enables solid DOS defenses Misbehaving parties cannot misrepresent themselves 16 Certificate Engine Registrars Publishers Clients Caches
  • Slide 17
  • Peer-to-Peer Distribution Avalanche: P2P file distribution Reduce download times, use network resources optimally Secure distribution: Publisher and content protection Only authorized users can join the distribution networks Cache-friendly distribution Enable network locality 17
  • Slide 18
  • ISP Cache Integration Protocol designed for cacheability Network operators can redirect clients to Avalanche-compatible caches Reduce ISP pain Improve experience for clients, publishers Experience can be accelerated Cache can proxy or relay for poorly connected nodes 18
  • Slide 19
  • Effect of P2P on inter-ISP traffic 19 Transit ISP Broadband ISP $$ server Internet Increase in egress traffic!! $$ => $$$$$
  • Slide 20
  • Does locality work? 20 Transit ISP Broadband ISP $$ server Internet
  • Slide 21
  • Benefits of locality for current P2P scenarios Each ISP is required to upload just over a copy of the file (1.9 GB) May require internal re-engineering 21
  • Slide 22
  • Architecture for P2P distribution 22
  • Slide 23
  • Architecture for P2P distribution 23 Peer matching based on locality Provides extensibility points for ISPs and content owners Content caches Preference to local connections
  • Slide 24
  • Peer-to-Peer Distribution Avalanche: P2P file distribution Reduce download times, use network resources optimally Secure distribution: Publisher and content protection Only authorized users can join the distribution networks Cache-friendly distribution Enable network locality RedCarpet: P2P Video-on-Demand Watch movies while downloading without loosing delivery efficiency. New P2P capability! RewindTV: Enhanced Live TV Watch live TV and be able to re-play missed scenes (from other peers) 24
  • Slide 25
  • Large Scale Content Delivery In current Peer-to-Peer systems: >60% of traffic is for video files Users need to wait to download the entire file before watching Emerging: P2P Live Streaming: Very popular: >200K concurrent users in one swarm Huge selection of content (mostly Asian) 25
  • Slide 26
  • Large Scale Content Delivery Unstructured Peer-to-Peer Scalability Easy to construct/maintain Typical applications: File distribution E.g. eMule, BitTorrent Live streaming E.g. CoolStreaming et al. Q) Can we use unstructured P2P to deliver a video-on-demand experience? Our approach: Pseudo-sequential downloads Pseudo-sequential downloads Pre-fetching of data Pre-fetching of data Network coding for efficient swarming Network coding for efficient swarming Q) Can we use unstructured P2P to deliver a video-on-demand experience? Our approach: Pseudo-sequential downloads Pseudo-sequential downloads Pre-fetching of data Pre-fetching of data Network coding for efficient swarming Network coding for efficient swarming 26
  • Slide 27
  • Challenges of Video on Demand Small start-up times Fast channel switching Good sustainable playback rate Users join at arbitrary points in time Video Seekingfuturefuture 27
  • Slide 28
  • Network Coding vs. Non-NetCoding Flash crowd of 20 clients Best non-Network Coding scheme Global-rarest Network Coding 14% improvement in playback rates Low jitter in download times Better user experience Global rarest vs. Network Coding (Based on a prototype implementation) 28
  • Slide 29
  • Other topic: Topology Management 29 -Issue: Peers at different stages of the download. -Issue: Heterogeneous capacities: -Fast nodes should not be penalized. Data (%)
  • Slide 30
  • 30 Mass Scale Content Distribution Multi-purpose P2P platform File Sharing: Avalanche Software Updates: DeltaCast Video-on-Demand: RedCarpet Live Streaming: RewindTV Network Coding Secure Distribution Topology Construction ISP Friendliness
  • Slide 31
  • Lessons learned P2P technologies are growing in acceptance Being deployed by Movie Studios, Television broadcasters, Music companies, Software distributors P2P systems need to respect the current ecosystem Network operators: locality, caching Content producers: strong security Network coding is feasible and practical today, and leads to performance improvements The unstructured P2P paradigm helps even for video distribution 31