CHORD – peer to peer lookup protocol

Shankar Karthik Vaithianathan & Aravind Sivaraman

University of Central Florida

Outline

•History

•General definition of a DHT system

•Chord

•Applications

•Implementation

•Demo/Open questions

server index table data

client a client b client c client d

query

data transferring

History: client-server model

server

index table

data client a

client b

client c

client ddata

data

data

query

data transferring

History: peer-to-peer model (Napster)

query

data transferring

index table

data client a

client b

client c

client d

index table

data

index table

data

index table

data

History: peer-to-peer model (Gnutella)

query

data transferring

index table

data client a

client b

client c

client d

index table

data

index table

data

index table

data

History: peer-to-peer model (DHT systems)

What is normal hashing ? Putting items into say n buckets, often solved by a static solution(e.g H(key)mod N)

Problems

N changes ? We must move every thing (simple solution) or could we move just (#items) / (#buckets) on an average (optimal solution)

what if we do not know the number of buckets ?

Distributed Hash Table

Distributed Hash Table

Consistent Hashing

Let A & B be Hashing. They send an object k to two different buckets, say y and z, respectively. Then either A cannot see bucket z or B cannot see bucket y.

Each bucket is a assigned a number in (0,1), and each item gets a value (0,1). You could then be in according to least distance or always round down(or up) and wrap at the edges.

Distributed Hash Table

Consistent Hashing c’td

If a new bucket is added, we therefore need to move the Contents of bucket immediately before it.

If we let each bucket have m elements and (M = log n) then each bucket gets a fair share of load and on an average we move only the average contents of the bucket around.

If we let each bucket have m elements and (M = log n) then each bucket gets a fair share of load and on an average we move only the average contents of the bucket around.

Distributed Hash Table

•a new class of peer-to-peer routing infrastructures.

•support a hash table-like functionality on Internet-like scale

•Consider the buckets to be computers on the internet and that given a key, they should find the value(IP address)

•The challenges with such a system are:

•Load balancing

• Scalability

• dynamic nature

• no critical point

• deterministic.

•an overlay space and its partition approach

•a routing protocol

local routing table

next-hop decision

•a base hash function

variants: proximity-aware, locality-preserving, etc.

Basic DHT components

Consistent hashingdata

server

Overlay space

Hashing

• Overlay space- one-dimensional unidirectional key space 0 – 2m-1.

Given two m-bit identifiers x and y, d(x, y)=(y-x+2m) % 2m

- Each node is assigned a random m-bit identifier.

- Each node is responsible for all keys equal to or before its identifier until its predecessor node’s identifier in the key space.

Chord [Stoica et al. Sigcomm2001]

• Routing table (finger table)- (at most) m entries. The ith entry of node n contains the pointer to the first node that succeeds n by at least 2(i-1) on the key space, 1 i m.

• Next-hop decision:- For the given target key k, find the closest finger before (to) k and forward the request to it.- Ending condition: The request terminates when k lies between the ID range of current node and its successor node.

- The routing path length is O(log n) for a n-nodes network with high probability (w.h.p.).

Chord – routing protocol

A Chord network with 8 nodes and 8-bit key space

0

32

64

96

128

160

192

224

256

Network node0

256Data

120

Chord – an example (m=8)

Chord – routing table setup

A Chord network with 8 nodes and 8-bit key space

Network node

2550 64 12832 19296 160 224

[1,2)Range 1

[2,4)Range 2

[4,8)Range 3

[8,16)Range 4

[16,32)Range 5

[32,64)Range 6

[64,128)Range 7

[128,256)Range 8

Data

Pointer

Chord – a lookup for key 120

How to look up a key quickly ?

• We need finger table

How to look up a key quickly ?(cont.) finger table for node 1finger table for node 1

How to look up a key quickly ?(cont.)

Node 3: Am I predecessor(1) ?

Predecessor(1) successor(1)

Node 3: Try entry 3, and find node 0

Node 3: Send lookup to node 0

Node 0: Am I predecessor(1) ?

Node 0: successor(1) is node 1

return to node 3 (RPC)

Value of key 1

?

Node joins

Two challenges Each node’s finger table is correctly filled Each key k is stored at node successor(k)

Three operations Initialize the predecessor and fingers of the new node n Update the fingers and predecessors of existing nodes Copy to n all keys for which node n has became their

successor

Initialize the predecessor and fingers of node n

• Idea: Ask an existing node for information needed

Join in

Update the fingers and predecessors of existing nodes

• Observation: when node n joins the network, n will become the ith finger of a node p when the following two conditions meet: P proceeds n by at least 2i-1

The ith finger of node p succeeds n

Solution: Try to find predecessor(n- 2i-1) for all 1<=i<=m; and check whether n is their ith finger, and whether n is their predecessor’s ith finger.

Update the fingers and predecessors of existing nodes (cont.)

Predecessor(6-21-1) =3, update

6

Predecessor(3) =1, no update

Predecessor(6-22-1) =3, update

6

Predecessor(6-23-1) =1, update6

Predecessor(1) =0, update

Predecessor(3) =1, no update

6

Predecessor(0) =3, no update

Join in

Copy to n all keys for which node n has became their successor

• Idea: Node n can become the successor only for keys stored by the node immediately following n

Join in

Extended Chord protocol

• Concurrent joins

• Failures and replication

• Beyond are project scope. [Ref. Stoica et, all sigcomm 2001]

Example applications: Co-operative Mirroring

Time-shared storage

Distributed indexes

Large-Scale combinatorial search

Implementation Our current implementation is simulation of the

Chord protocol in O(log n) steps in java.

The framework can be shown as below

Methods Implemented

Conclusion

Questions ?

Reference :

Chord: A Scalable Peer to peer Lookup Service for InternetApplications

Ion Stoica , Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan

MIT Laboratory for Computer Sciencechord@lcs.mit.edu

http://pdos.lcs.mit.edu/chord/

THANK YOU