Architectures for Peer-to-Peer Content Distribution Christos Gkantsidis Microsoft Research,...

Architectures for Peer-to-Peer Content Distribution

Christos GkantsidisMicrosoft Research, Cambridge, UK

Joint work with: Pablo Rodriguez, John Miller, Dinan Gunawardena, and others

Peer-to-Peer Applications

BBC iMP

SETI@Home .NET (PeerChannel, WCF)JXTA

P2P content distribution is hugely popular: >40-60% of Internet trafficEasy access to an amazing variety of content

Challenges of P2P: P2P traffic difficult to manage

ISP complaints Dependence of free bandwidth

e.g. from universities Massive copyright infringement

2

Microsoft scenarios where P2P can helpLarge scale software distribution:

Windows / Microsoft updates Beta versions of software (e.g. Visual Studio) Free software

MSN Video File sharing (e.g. Foldershare, Groove, Live Messenger)

Windows XP P2P Name Resolution Protocol (PNRP) P2P Graphing and Grouping Peer Identity Manager

Windows VistaPNRP, Graphing and Grouping, Peer Identity Manager, P2P Contacts, People Near MeWindows Communication Foundation Peer Channel

Incidentally, MS has been interested in P2P for some time …

3

Peer-to-Peer Distribution• Avalanche: P2P file distribution

– Reduce download times, use network resources optimally

– Secure distribution:• Publisher and content protection• Only authorized users can join the distribution networks

– Cache-friendly distribution– Enable network locality

• RedCarpet: P2P Video-on-Demand– Watch movies while downloading …– … without loosing delivery efficiency.– New P2P capability!

• RewindTV: Enhanced Live TV– Watch live TV …– … and be able to re-play missed scenes (from other

peers) 6

Peer-to-Peer Distribution• Avalanche: P2P file distribution

– Reduce download times, use network resources optimally

– Secure distribution:• Publisher and content protection• Only authorized users can join the distribution networks

– Cache-friendly distribution– Enable network locality

7

File Swarm = Block Scheduling

8

System’s progress in current File Swarming systems

9

(From Tian et al., Infocom’06)

A lot of time spent at the beginning and finish of download:• Beginning of download: finding good blocks to exchange• End of download: discovering the last missing blocks

Network Coding Simplified

10

File to Transfer

Block 1 Block 2 Block 3

Encoding

With Network Coding

11

System’s progress

12

• Smooth download progress:– No start-up delay– No last-block problem

13

Bandwidth Contribution

• Easily withstands flash crowds• Server contribution is fixed, Client contribution scales• >10 fold savings in content provider’s bandwidth using peer-to-peer.

Results from distributing Visual Studio

14

Data from distribution of beta versions of Visual Studio 2008 Beta (Nov’07)

Median speeds:~1.5Mbps for VS Pro~2.7Mbps for the others

Peer-to-Peer Distribution• Avalanche: P2P file distribution

– Reduce download times, use network resources optimally

– Secure distribution:• Publisher and content protection• Only authorized users can join the distribution

networks– Cache-friendly distribution– Enable network locality

15

Strong security model

• All connections TLS-authenticated

• Data is self-certifying– Block data validated by

signed homomorphic hash equivalent

• Strong identity enables solid DOS defenses– Misbehaving parties

cannot misrepresent themselves

16

Certificate Engine

Registrars

Publishers

Clients

Caches

Peer-to-Peer Distribution• Avalanche: P2P file distribution

– Reduce download times, use network resources optimally

– Secure distribution:• Publisher and content protection• Only authorized users can join the distribution

networks– Cache-friendly distribution– Enable network locality

17

ISP Cache Integration

• Protocol designed for cacheability– Network operators can redirect clients to

Avalanche-compatible caches– Reduce ISP pain– Improve experience for clients, publishers

• Experience can be accelerated• Cache can proxy or relay for poorly connected

nodes

18

Effect of P2P on inter-ISP traffic

19

Transit ISPTransit ISP

Broadband ISP

Broadband ISP

Broadband ISP

$$$$

$$$$

serverInternetIncrease in egress

traffic!!$$ => $$$$$

Does locality work?

20

Transit ISPTransit ISP

Broadband ISP

Broadband ISP

Broadband ISP

$$$$

$$$$

serverInternet

Benefits of locality for current P2P scenarios

Each ISP is required to upload just over a copy of the file (1.9

GB)

May require internal re-engineering 21

Architecture for P2P distribution

`

Consumer

`

Consumer

`

Consumer

CertificateAuthority

Seed Servers

Registrars

`

Publisher

22

Architecture for P2P distribution

`

Consumer

`

Consumer

`

Consumer

CertificateAuthority

Seed Servers

Registrars

`

Publisher

Cache

`

23

Peer matching based on locality

Provides extensibility points for ISPs and content owners

Content caches

Preference to local connections

Peer-to-Peer Distribution• Avalanche: P2P file distribution

– Reduce download times, use network resources optimally

– Secure distribution:• Publisher and content protection• Only authorized users can join the distribution networks

– Cache-friendly distribution– Enable network locality

• RedCarpet: P2P Video-on-Demand– Watch movies while downloading …– … without loosing delivery efficiency.– New P2P capability!

• RewindTV: Enhanced Live TV– Watch live TV …– … and be able to re-play missed scenes (from other

peers) 24

Large Scale Content Delivery

In current Peer-to-Peer systems:

• >60% of traffic is for video files• Users need to wait to download

the entire file before watching

Emerging: P2P Live Streaming:• Very popular: >200K concurrent

users in one swarm• Huge selection of content

(mostly Asian)

25

Large Scale Content Delivery

• Unstructured Peer-to-Peer– Scalability– Easy to construct/maintain

• Typical applications:– File distribution

E.g. eMule, BitTorrent

– Live streamingE.g. CoolStreaming et al.

Q) Can we use unstructured P2P to deliver Q) Can we use unstructured P2P to deliver a video-on-demand experience?a video-on-demand experience?Our approach:Our approach:• Pseudo-sequential downloads Pseudo-sequential downloads • Pre-fetching of dataPre-fetching of data• Network coding Network coding for efficient swarmingfor efficient swarming

26

Challenges of Video on Demand• Small start-up times

– Fast channel switching

• Good sustainable playback rate

• Users join at arbitrary points in time

• Video Seeking

futurefuture

futurefuture27

Network Coding vs. Non-NetCoding

• Flash crowd of 20 clients

• Best non-Network Coding scheme– Global-rarest

• Network Coding– 14% improvement in

playback rates– Low jitter in download

times Better user experienceGlobal rarest vs. Network Coding

(Based on a prototype implementation)

28

Other topic: Topology Management

29

- Issue: Peers at different stages of the download.- Issue: Heterogeneous capacities:

- Fast nodes should not be penalized.

Dat

a (%

)

Dat

a (%

)

30

Mass Scale Content DistributionMulti-purpose P2P platform

Lessons learned• P2P technologies are growing in acceptance

– Being deployed by Movie Studios, Television broadcasters, Music companies, Software distributors

• P2P systems need to respect the current ecosystem– Network operators: locality, caching– Content producers: strong security

• Network coding is feasible and practical today, and leads to performance improvements

• The unstructured P2P paradigm helps even for video distribution 31