Breadboxp2p file storage and distribution

Team: Brian Smith, Daniel Suskin, Dylan Nunley, Forrest VinesMentor: Brendan Burns

OverviewObjectiveBackgroundDesign

PastryData ManagerClient

Future ImprovementsDemoQuestions

Project ObjectiveStorage that is:

PersistentAvailableReliableInexpensiveSecure

P2P Systems(Past/Present)NapsterGnutellaFreenetOceanstoreBitTorrent

Hierarchical Design

Client

Java RMIData Manager

Pastry API

Pastry

Architectural Design

ClientData

ManagerPastry

ClientData

ManagerPastry

Pastry Overlay Network

Node A Node X

PastryStoring stateRoutingMaintaining state

Storing stateLeaf set

Closest numericallyFinal routing

Neighborhood setLocality, upkeep

Storing StateRouting table

Structure based on IDs

RoutingOption 1: Leaf set

FastestOption 2: Routing table

Prefix-basedOption 3: Other

RandomizedOption 4: Deliver

State maintenance: new nodesTwo phases

Request Build state for new node

Announce Tell others to add to state

State maintenance: old nodesLazy

Routing failsAsk other nodes for new state. Either:

Leaf set Neighborhood set A single routing table entry

State maintenance: heartbeatNeighbors

ScalabilityUntested

But works in theoryState scales well

ID maximum digits x ID representation base + M + L

ReliabilityLargely untested, but:

Lazy repairLow chance of entire leaf set going down

Server/Data ManagerService that is run on each node of the networkConnection between client and pastry substrate

Client

Data Manager Service

Client

Data Manager Service

Pastry

Data StorageService manages local file chunks

Data Manager Service

Data Chunk

Data Chunk

Data Chunk

Scalability and Reliability:ReplicationEach node is responsible for replicating chunks whose

Ids are closest to its nodeIDEach chunk is replicated to the n closest nodesChunks that are no longer within the closest n will

timeout

Client: Put

FILE DATA

CHUNK

MANAGER

compresschunk

send

log

LOG

Client: Get

FILE DATA

CHUNK

MANAGER

decompress

unchunk receive

LOG

GET

check

request

Future ImprovementsPastry

Security (Malicious Nodes)Manager

CachingMessage aggregation

ClientHash CheckGUIEncryption

Demo