Percolation search in power law networks: making unstructured peer-to-peer networks scalable

Abstract

We introduce a scalable searching protocol for locating and retrieving content in random networks with Power-Law (PL) and heavy-tailed degree distributions. The proposed algorithm is capable of finding any content in the network with probability one in time O(log N), with a total traffic that provably scales sub-linearly with the network size, N. Moreover, the protocol finds all contents reliably, even if every node in the network starts with a unique content. The scaling behavior of the size of the giant connected component of a random graph with heavy tailed degree distributions under bond percolation is at the heart of our results. The percolation search algorithm can be directly applied to make unstructured Peer-to-Peer (P2P) networks, such as Gnutella, Limewire and other file-sharing systems (which naturally display heavy-tailed degree distributions and scale-free network structures), scalable. For example, simulations of the protocol on the limewire crawl number 5 network[12], consisting of over 65,000 links and 10,000 nodes, show that even for such snapshot networks, the traffic can be reduced by a factor of at least 100, while achieving hit-rates greater than 90%.

Optimizing capacity-heterogeneous unstructured P2P networks for random-walk traffic

“…Replication is used to improve the query success rate and reduce the query latency by making pointers to shared files available in the paths of query walks. One-hop [4] and random-walk replication [8], [16] are the two most common schemes used in unstructured P2P networks. To ensure a capacity-proportional distribution of replication load in the network, we propose to use the random-walk replication scheme with CPMH walks and call this strategy CPMH replication.…”

Multiple power-law structures in heterogeneous complex networks

“…The primary motivations for designing local dynamics so that a PL topology emerges include, (i) PL networks are resistant to random deletions and have vanishingly small percolation thresholds [11]; (ii) PL networks have a natural hierarchy allowing more capable processors to act as hubs; moreover, computing resources are heterogeneous to begin with and PL networks provide a natural set-up for the resource hierarchy to be embedded into a networking hierarchy, and (iii) the structure of PL networks can be exploited to provide scalable key-words based search capabilities [19,20,21]. While these properties of PL networks, have been proven to be true for random PL networks, our recent results show that the grown random networks generated using the local dynamics formulated in this paper (particularly, the ad hoc dynamics, where nodes randomly leave the network) lead to networks that are much closer to random PL graphs than those generated by previously-proposed algorithms [14] [27].…”

“…As was shown in Section IV, the same compensatory mechanism developed in [12] can ensure the emergence of scale-free structures with heavy tails among more stable groups, the groups anticipated to be composed of nodes with high capacity, while the majority of the nodes will have a light tailed degree distribution and are therefore exempted from the search paths. The results of this paper will serve as an essential part of ad-hoc network formation protocols that can support efficient search [19,20,21], robustness, and allow highly dynamic operations.…”

Hierarchical multidimensional search in peer-to-peer networks

“…Most of them, however, are based on query flooding (e.g., [4]), which is communicationally inefficient, or based on random walking (e.g., [15]), which is ineffective for similarity queries. Therefore, a structuralization of the network is more favorable, which should include two constituent components: a communication architecture and an indexing architecture.…”