A Distributed Search Service for P2P File Sharing in Mobile Applications

A Distributed Search Service for Peer-to-Peer File Sharing in Mobile ApplicationA Distributed Search Service for Peer-to-Peer File Sharing in Mobile Application Presented by Tony Sung On Loy, MC Lab, CUHK IEPresented by Tony Sung On Loy, MC Lab, CUHK IE 1

A Distributed Search Service for P2P A Distributed Search Service for P2P File Sharing in Mobile ApplicationsFile Sharing in Mobile Applications

4 September, 20034 September, 2003

Authors - Christoph Lindemann and Oliver P. Waldhorst, University of Dormund, Dept. of Computer ScienceAuthors - Christoph Lindemann and Oliver P. Waldhorst, University of Dormund, Dept. of Computer Science


ItineraryItinerary

Background StudiesBackground Studies

Introducing Passive Distributed Indexing (PDI)Introducing Passive Distributed Indexing (PDI)

Algorithm DetailsAlgorithm Details

Performance ResultsPerformance Results

Conclusion and Future WorkConclusion and Future Work


Background StudiesBackground StudiesA Mobile Ac-Hoc Network

Short-range Wireless, e.g. Bluetooth

Medium-range Wireless, e.g. IEEE 802.11

Such Ac-Hoc Network can be used for data sharing between mobiles,e.g. Documents, MP3s and Video Clips

How to enable searching of P2P data on top of the architecture?


Background StudiesBackground StudiesRelated Works

SolutionsSolutions CommentsComments

NapsterNapsterHybid P2P using a CenHybid P2P using a Centralized Index Servertralized Index Server

No mobile device in general has the capability to act No mobile device in general has the capability to act as the central serveras the central server

Even there is a central server, it cannot be Even there is a central server, it cannot be reachable from all client due to Hidden-Node reachable from all client due to Hidden-Node ProblemProblem

GnutellaGnutellaFully distributed Fully distributed searching using multi-searching using multi-hop flood algorithmhop flood algorithm

Flooding entire network with query messages limits Flooding entire network with query messages limits its scalabilityits scalability

7DS7DSFirst in mobile First in mobile environment, utilizing environment, utilizing flood algorithmflood algorithm

Similar drawback as GnutellaSimilar drawback as Gnutella


Proposed SolutionProposed SolutionObjectivesObjectives

“to provide a general-purpose file search service which can “to provide a general-purpose file search service which can be used by several kinds of mobile applications running on top”be used by several kinds of mobile applications running on top”

Passive Distributed Indexing (PDI) Passive Distributed Indexing (PDI)

- Each device stores its local documents as a - Each device stores its local documents as a RepositoryRepository

- Uniquely identify documents with its local path- Uniquely identify documents with its local pathand unique device ID, a.k.a. and unique device ID, a.k.a. Document IdentifierDocument Identifier

- A local - A local Index CacheIndex Cache is maintained on each device, is maintained on each device,which forms the core component of this architecturewhich forms the core component of this architecture

- Searching is performed by keyword searches- Searching is performed by keyword searches


Passive Distributed IndexingPassive Distributed IndexingOperation Scenario

Node 1

Node 2

Node 3

…

q: d1, d2

q: d2, d3



Node 1

Node 2

Node 3

[QUE]

q ?…

q: d1, d2

q: d2, d3



Node 1

Node 2

Node 3

…

q: d1, d2

q: d2, d3

[QUE]

q ?



Node 1

Node 2

Node 3

q: d1, d2

q: d1, d2, d3

q: d1, d2, d3

[REP]

q : d1, d2

[REP]

q : d2, d3[REP]

q : d1, d2



Node 1

Node 2

Node 3

q: d1, d2, d3

q: d1, d2, d3

q: d1, d2, d3

[REP]

q : d3


Performance AnalysisPerformance AnalysisIndependent Parameters

No. of Devices,

Transmission Range,

Mobility Model

No. of Documents,

No. of Keywordsof Interest,

Distribution of Keywords

Inter-request Timeof Queries

Index Cache Size,

Max. TTL,

(Document Timeout)

System Param.System Param. Application Param.Application Param. Protocol Param.Protocol Param.


Performance AnalysisPerformance AnalysisValues for Simulation


Performance AnalysisPerformance AnalysisPerformance Measure

?



?

Nall = 5



?

Nall = 5 Nrep = 3



?

Nall = 5 Nrep = 3

Query Hit Rate = Nrep / Nall

(other performance measures, e.g. system response time, is left for future work.)


Analysis of ResultsAnalysis of ResultsSensitivity to System Parameters : No. of Devices & Index Cache Size

Local Index Local Index Cache has Cache has very little very little ImpactImpact

Limited impact Limited impact of of ↑in No. of ↑in No. of devices on devices on PDI PDI performance performance (1)(1)

↑↑in No. of in No. of devices leads devices leads to to ↑in↑in PDI PDI performance performance (2)(2)

(1, 2) Small index cache cannot accommodate entries for all matching documents

Conclusion : Index Cache size can be small when No. of devices is small, whereas sufficient index cache size can boost performance in case of large No. of devices


Analysis of ResultsAnalysis of ResultsSensitivity to System Parameters : No. of Devices & Forwarding TTL

Message Message forwarding has forwarding has very little very little ImpactImpact

Forwarding Forwarding improves improves performance performance by 20% by 20% (1)(1)

(1) A higher probability of reaching more devices for forwarding in medium No. of devices

(2) High No. of devices fills local index cache with nearby entries, which replaces message-forwarding adequately

Conclusion : Forwarding is useful in medium density systems, but should be disabled for high density systems to avoid unnecessary network traffic

Advantage Advantage vanished vanished when No. of when No. of devices grows devices grows further further (2)(2)


Analysis of ResultsAnalysis of ResultsSensitivity to System Parameters : Transmission Range & Index Cache Size

Local Index Local Index Cache has Cache has very little very little ImpactImpact (1) (1)

Index Cache Index Cache Size Size significantly significantly improves improves performanceperformance

(1) Small No. of devices is reached with very low transmission range, thus increase in cache size makes no impact

Conclusion : Index Cache size can be small for short-range devices such as Bluetooth, whereas No. of devices should be high to compensate for the low Hit Rate


Analysis of ResultsAnalysis of ResultsSensitivity to System Parameters : Transmission Range & Forwarding TTL

PDI with PDI with message message forwarding forwarding disabled gains disabled gains best best performance performance for high-range for high-range devicesdevices

(1) Responses for uncommon entries are still forwarded over great distances, that fills index caches with junk entries

Conclusion : When transmission range is high, message forwarding should be disabled


Analysis of ResultsAnalysis of ResultsSensitivity to App Parameters : Zipf

Zipf-like distribution is used to model PDF of searching keywords

For keyword kj,

Pr(k = kj) ≈ j- α ,

for 0 <= α <= 1

Therefore, the higher the α, more localized is the query stream


Analysis of ResultsAnalysis of ResultsSensitivity to App Parameters : Zipf & Index Cache Size

PDI is PDI is extremely extremely sensitivity to sensitivity to locality in locality in request request stream for stream for small Index small Index CacheCache

Conclusion : For applications offering no significant locality in the request stream, sizes of Index Cache must be chosen adequate

PDI can PDI can achieve a hit achieve a hit rate of > 70% rate of > 70% despite of despite of locality in large locality in large Index CacheIndex Cache


Analysis of ResultsAnalysis of ResultsSensitivity to App Parameters : Zipf & Forwarding TTL

PDI is gains PDI is gains performance performance improvements improvements from packet from packet forwarding for forwarding for higher locality, higher locality, 2-hop 2-hop forwarding forwarding performs performs similarly with similarly with higher Hopshigher Hops

Conclusion : 2-hop message forwarding should be enabled in applications offering a high degree of locality in request stream

For even For even higher locality, higher locality, 2-hop 2-hop forwarding forwarding out-performs out-performs the othersthe others


Analysis of ResultsAnalysis of ResultsSensitivity to App Parameters : No. of Document & Index Cache Size

Performance Performance decreases decreases linearly with linearly with No. of No. of documents per documents per devicedevice

(1) Has been shown elsewhere what this behaviors is explained if a Zipf-like request distribution is assumed

Conclusion : Maybe more sophisticated Forwarding Strategies rather than increasing Index Cache Size should be employed to improve the performance

Performance Performance increases with increases with Index Cache Index Cache size in only a size in only a log-like log-like fashion fashion (1)(1)


Analysis of ResultsAnalysis of ResultsSensitivity to App Parameters : No. of Document & Index Cache Size

Performance Performance is improved by is improved by 10% if a small 10% if a small No. of No. of documents documents exists in each exists in each device, with device, with near-maximal near-maximal performance performance with 2-hop with 2-hop forwardingforwarding

Conclusion : 2-hop forwarding can improves performance in small No. of documents per device, but all forwarding gains no performance when No. of documents per device is large

For large No. For large No. of documents of documents per device, no per device, no significant significant difference in difference in forwarding forwarding strategystrategy


Analysis of ResultsAnalysis of ResultsTransient Behaviors

Real Hit Rate Real Hit Rate is constant is constant over timeover time

Real Hit Rate : Rate of hits reported from devices actually hold a matching document

Conclusion : System will attain its maximal performance automatically and no initial warm-up mechanism is required

PDI Hit Rate PDI Hit Rate increases increases steadily after steadily after simulation simulation startstart



General-purpose Distributed Document Search Service

Utilizes Local Caching of Query Results to Avoid Flooding the Network

Tunable(Cache Size, TTL, Document Timeout)

to Support Different Environments & Applications

PDI is …

Provides an Initial Filling of Index Caches in a Very Short Time, No Warm-up Mechanism is Needed


Requires Sufficiently Large Index Cache Size

2-hop Packet Forwarding should be DisabledIf Either

The No. of Devices or Transmission Range is High

Requires Sufficient Large Index Cache Size


Contributions of Simulation Results

High Density,Low Query Locality

Medium Density, Medium-range

Large No. ofDocuments



Future Works include …

1. Investigation on the Impact of Document Modifications on the Performance of PDI, and the Design of the Appropriate Workaround Mechanism

2. Evaluation of the Performance of PDI considering Sophisticated Workload Models that Contains Location Depended Queries

3. Development of a Prototype Implementation of PDI and Field Tests



Comments …

1. PDI is a very simple solution for porting P2P File Sharing to Ac-Hoc Mobile Network

2. The Paper contains comprehensive simulation results and analysis of the PDI mechanism

3. However, the author did not suggest further modification on the PDI mechanism based on the analyzed results

4. There is also no analytical comparisons to any other similar implementations

5. PDI is yet to be challenged for improvement