Stream SpiderDistributed Music Streaming Service with Sliced

Music Files from Different Originating Hosts

Justin Steffy and Michael D. Elder

Outline

• Service Operation• Design

– Peer to Peer Client and Service– Availability and Fault Tolerance– Consistency and Correctness

• Implementation– Foundations: Derby, Jetty, Eclipse RCP– Protocols (HTTP/XML)

• Demonstration

System Operation

• Register with service to find a set of peers with music to share

• Search for particular files on other peers

• Play music while streaming contents from multiple peers

Design

• Peer to Peer protocol– Peers can search and request files directly– Peers also service search and file requests– Two levels of peer

• Highly transient peers– Weak consistency– Sporadic availability

• Maintain state of peers on the network

Peers

• Types of Peers– Service Peer

• Executes and responds to file requests• Requests and services music streams• Maintains small “neighbor set” to service requests

– Bootstrap Peer: • Receives initial registration for new Peers• Replicates available registrations• Maintain consistency with other Bootstrap peers• Can also act as a Service Peer

Diagram: Registration

Register A List of Peers

Register C

Register B

Peer B

Peer A

Peer C Bootstrap Peer

List of Peers including A

List of Peers including A & B

Diagram: File Search

Bootstrap Peer

Have files matching A?

List of Files

Peer A

Peer C

Peer B searches for songs that match A

Diagram: Music Streaming

Bootstrap Peer

Request Segment

Octet Stream of data

Peer A

Peer C

Request SegmentOctet Stream of data

Peer B

Availability and Fault Tolerance

• Availability does not depend on any one bootstrap peer– Replicate state among many peers– Loss of a bootstrap causes minimal effect

• Peers tolerate losses in neighbor set– When list gets low, request more peers– Losing a peer while streaming is handled

Important State

• Set of available peers (Shared) – Bootstrap Peers synchronize Peer locations– Service Peers request more peers as their

neighbor set leaves or fails– Peer availability is volatile state, maintained

relative to each Peer

• Spliced Music Streams – Recovery when an identified peer dies while

servicing a stream

Consistency and Correctness

• Bootstrap Servers enforce Weak Consistency when sharing available Peers

• Correctness Criteria– Peers should be able to register with one of

the active Bootstrap Peers– Peers should receive search results from

active peers in its neighbor set– Peers should receive music splices upon

request in order to stream music

Implementation

• Java and Eclipse– Eclipse UI– Database– Servlets– JUnit

• HTTP and XML basis for protocol– Object Models

Foundations

• Derby 10.1– Embedded Java Database– Maintains local state at each Peer

• Jetty 6.1– Embedded Java 2 EE Servlet Container– Act as protocol endpoints

• Eclipse Rich Client Platform (RCP)– Provides for rapid development of application architecture and

UI• Eclipse Modeling Framework (EMF)

– Provides an object model abstraction and XML serialization• JLayer 1.0

– MP3 playback• Milestone development with unit tests (~20)

Protocols

• Based on HTTP and XML• Registration

– Accepts a requested number of peers– Answers set of available peers

• Search– Accepts a filename – Answers set of files that match a portion of the input

• Fragment– Accepts a file id and fragment range– Answers the portion of the file

Registration

• Request – HTTP Query String<Hostname, Port, Number of peers>

• Response - XML<PeerList>

<Peer1 …/><Peer2 …/>…

</PeerList>

Search

• Request – HTTP Query String<Filename>

• Response - XML<FileList>

<File1 …/><File2 …/>…

</FileList>

Fragment Request

• Request – HTTP Query String<FileID, Start, Length>

• Response – Octet Stream– Beginning at start for length bytes

Consistency Protocols

• Status – Determine peer availability– Send status request to a peer– Peer returns a current load value

• Synchronization (Bootstrap only)– Keep list of other bootstrap peers– In round robin style, send all newest peers

seen at this peer (HTTP Post)– Return the new peers seen

Bootstrap Synch

POST with XML Peer List

Return an XML Peer List

•Get last used time of Peer B

•Find all newer peers

•Construct XML and POST to Peer B

•Receive the response and add peers to our list

•Update the last used time of B

Peer A Peer B

•Receive the Peer List

•Add peers to our list

•Find last used time of Peer A

•Gather all peers newer than time

•Send a Peer List

•Update the last used time of A

Key API

• Net API – Contains all server and state– Database Manager – Handles peer state– Bootstrap Manager – Synchs peer state– Peer Manager – Keep alive with other peers– Protocol Requests – Encapsulate requests

• Servlets – Started and forgot by Net API– One per request type– Run independently of rest of system

Architecture

Core

UINet

Diagram: Metamodel

Diagram: Database

Diagram: Playback

Diagram: Net API

Diagram: Servlets

Diagram: Requests

Diagram: UI

Demonstration

• Normal Operation

• Failure of bootstrap

• Failure of service peer