Upload
martha-shelton
View
223
Download
2
Tags:
Embed Size (px)
Citation preview
Distributed Search withRendezvous Search SystemsChristof Leng
Dipl.-Inform. Christof Leng
Databases & Distributed SystemsDepartment of Computer ScienceTechnische Universität Darmstadt
http://[email protected]
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"2
Distributed Search: A Traditional View
| 27.01.12 |
Implement distributed key-value lookup e.g., a distributed hash table (DHT) scalable & simple
Build everything else on top of it keyword search, XPath, range queries map every operation to a number of lookups
Downside: implement every query language yourself query processing not portable between DHTs unforeseen side-effects & interactions
Experts needed for every application application domain P2P networking information retrieval
Application
Developer
P2P Networking Expert
Information Retrieval
Expert
… need sophisticated queries …
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"3
Rendezvous Search
“Every query will meet each data item somewhere in the network” don’t worry how it gets there black box approach
Implement the match functionality as if it was local re-use existing libraries
Advantages: separation of concerns flexible scalable (O(√n) search cost) robust (O(√n) replicas)
| 27.01.12 |
Application
Developer
Rendezvous Search System
Information Retrieval
Library
… need sophisticated queries …
glue code
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"4
Example Application
| 27.01.12 |
P2P WikiLucene
Keyword Search
Rendezvous Middleware
Overlay Network
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"5
Rendezvous Search Genealogy
| 27.01.12 |
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"6
Data Center Solutions
| 27.01.12 |
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"7
Grid
Pro: simple efficient flexible
Con: not fault-tolerant not size-adaptive not load-adaptive two-dimensional
structure
Reference:L. A. Barroso, J. Dean, and U. Hölzle. Web Search for a Planet: The Google Cluster Architecture. IEEE Micro, 23(2):22–28, 2003.
| 27.01.12 |
Q Q Q Q Q
Data
Query
D
D
D
D
D
D
QR
Rendezvous Node
Q Q Q Q Q
Data
Query
D
D
D
D
QR
Rendezvous Node
Q Q Q
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"8
ROAR – Rendezvous on a Ring
Pro: simple mapping to 1
dim. size-adaptive load-adaptive fault-tolerant
Con: centralizedor inefficient
Reference:C. Raiciu, F. Huici, M. Handley, and D. S. Rosenblum. ROAR: Increasing the Flexibility and Performance of Distributed Search. In Proceedings of SIGCOMM’09, pages 291–302, 2009. ACM.
| 27.01.12 |
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"9
Structured Peer-to-Peer
| 27.01.12 |
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"10
Bit Zipper
Pro: z-order mapping DHT-based =
decentralized fast tree-based
replication
Con: relies on DHT’s
reliability no replica maintenance
Reference:W. W. Terpstra, S. Behnel, L. Fiege, J. Kangasharju, and A. Buchmann.Bit Zipper Rendezvous—Optimal Data Placement for General P2P Queries.In EDBT’04 Workshop on Peer-to-Peer Computing and Databases, 2004.
| 27.01.12 |
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"11
Deetoo
Pro: 2 rings for projection includes replica
maintenance
Con: maintenance overhead
for 2 rings no responsibility ranges
(probabilistic approach) additional overhead
Reference:T. W. Choi and P. O. Boykin. Deetoo: Scalable unstructured Search built on a Structured Overlay. In HOTP2P - International Workshop on Hot Topics in Peer-to-Peer Systems, Los Alamitos, CA, USA, 2010. IEEE Computer Society.
| 27.01.12 |
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"12
Unstructured Peer-to-Peer
| 27.01.12 |
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"13
Ferreira et al. – Random Walks
| 27.01.12 |
Pro: very resilient topology proven correctness
Con: probabilistic additional overhead slow & fragile random
walks no topology protocol
Reference:R. A. Ferreira, M. K. Ramanathan, A. Awan, A. Grama, and S. Jagannathan. Search with Probabilistic Guarantees in Unstructured Peer-to-Peer Networks. In Proceedings of P2P’05, pages 165–172, Washington, DC, USA, 2005. IEEE Computer Society.
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"14
BubbleStorm
| 27.01.12 |
Pro: resilient & fast size-adaptive load-adaptive topology & replica
maintenance included
Con: probabilistic additional overhead
Reference:W. W. Terpstra, J. Kangasharju, C. Leng, and A. P. Buchmann. BubbleStorm: Resilient, Probabilistic, and Exhaustive Peer-to-Peer Search. In Proceedings of SIGCOMM’07, pages 49–60, New York, NY, USA, 2007. ACM.
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"15
Hautakorpi & Schultz
| 27.01.12 |
Pro: unstructured solution on
top of DHT incremental deployment
possible
Con: probabilistic additional overhead fragile
Reference:J. Hautakorpi and G. Schultz. A Feasibility Study of an Arbitrary Search in Structured Peer-to-Peer Networks. In Proceedings of 19th International Conference on Computer Communications and Networks (ICCCN), pages 1–8. IEEE, Aug. 2010.
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"16
Semi-Structured Peer-to-Peer
| 27.01.12 |
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"17
SplitQuest
| 27.01.12 |
Pro: combines unstructured
resilience with structured efficiency
Con: probabilistic lacks analysis and
prototype evaluation
Reference:P. Lopes and R. A. Ferreira. SplitQuest: Controlled and Exhaustive Search in Peer-to-Peer Networks. In Proceedings of the 9th International Conference on Peer-to-Peer Systems, IPTPS’10, Berkeley, CA, USA, 2010. USENIX Association.
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"18
Comparison
Data Center Structured
Name Grid ROAR Bit Zipper Deetoo
Pro simple adaptive DHT add-on maintenance
Con not adaptive centralizedmaintenance unclear
probabilistic
| 27.01.12 |
Unstructured Semi-Structured
Name Ferreira BubbleStorm Hautakorpi SplitQuest
Pro simpleresilient & adaptive
increm. DHT add-on
efficient & resilient
Con fragile & slow probabilistic probabilistic immature
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"19
Q Q Q Q Q
Data
Query
D
D
D
D
QR
Rendezvous Node
Q Q Q
Rendezvous Search & Cloud Computing
| 27.01.12 |
similar problems similar requirements less research history
could benefit from size adaptivity traffic adaptivity fault tolerance algorithm diversity
Sharding
Key-Value Store Scalability
Replication
D
D
D
Q Q Q M Q QQ
Christof Leng | Rendezvous Search Systems | KuVS/ITG Fachgespräch "Future Internet"20
Conclusion
| 27.01.12 |
Rendezvous Search is a powerful abstraction scalable flexible robust
A wide range of solutions is out there data center unstructured P2P structured P2P semi-structured P2P
Ever needed decentralized but powerful search in a project? don’t try to reinvent the wheel!