iCluster: A Self-organizing Overlay Network for P2P Information Retrieval

Abstract

Abstract. We present iCluster, a self-organizing peer-to-peer overlay network for supporting full-fledged information retrieval in a dynamic environment. iCluster works by organizing peers sharing common interests into clusters and by exploiting clustering information at query time for achieving low network traffic and high recall. We define the criteria for peer similarity and peer selection, and we present the protocols for organizing the peers into clusters and for searching within the clustered organization of peers. iCluster is evaluated on a realistic peer-to-peer environment using real-world data and queries. The results demonstrate significant performance improvements (in terms of clustering efficiency, communication load and retrieval accuracy) over a state-of-the-art peerto-peer clustering method. Compared to exhaustive search by flooding, iCluster exchanged a small loss in retrieval accuracy for much less message flow.

Anonymity and Censorship Resistance in Unstructured Overlay Networks

“…This makes it possible to apply our framework to any kind of unstructured overlay network, by choosing an arbitrary routing strategy for the SUBMITQ message. For instance, our approach could be applied to Gnutella-style networks by adopting flooding as the routing for the SUBMITQ message, or small-world networks [24,20] by adopting interestbased query routing, while leaving the rest of the underlying protocols (e.g., rewiring in the case of small world networks) intact.…”

“…Documents were clustered using the incremental k-means algorithm and each cluster was assigned to a single peer to simulate different peer interests. This peer specialisation assumption is common in SONs [6,18,2], and does not restrict our setting since a peer with multiple interests might cluster its documents locally and create one identity for each interest in the spirit of [20]. These interests are then used to build a semantic overlay network according to standard techniques [6,18,2].…”

“…To maintain short-distance links up-to-date and ensure the clustering property of the SON, a periodic rewiring procedure [6,18,2,20] is executed by all peers, aiming at discovering new more similar peers, or refreshing links that have become outdated due to network dynamics. Long-distance links, usually updated using random walks, are necessary to avoid creating tightly clustered groups of peers that are disconnected from the rest of the network.…”

“…Semantic Overlay Networks (SONs) are an instance of unstructured networks that has lately been given considerable attention [6,18,2,20]. In a SON, peers that are semantically, thematically, or socially close (i.e., peers sharing similar interests or resources)…”

Rewiring strategies for semantic overlay networks

“…To show proof of concept, we adopt a generic P2P architecture that utilises widely adopted techniques from SONs, such as characterisation of peers by content, peer clustering and link rewiring. In our SON architecture, peers are characterised by documents in the spirit of [8,10,13,14,20,23,25]. Each peer in the network maintains links to other peers with similar content to form clusters.…”