Sets

Abstract

We present SETS, an architecture for efficient search in peer-to-peer networks, building upon ideas drawn from machine learning and social network theory. The key idea is to arrange participating sites in a topic-segmented overlay topology in which most connections are short-distance, connecting pairs of sites with similar content. Topically focused sets of sites are then joined together into a single network by long-distance links. Queries are matched and routed to only the topically closest regions. We discuss a variety of design issues and tradeoffs that an implementor of SETS would face. We show that SETS is efficient in network traffic and query processing load.

Shard ranking and cutoff estimation for topically partitioned collections

“…In order to compare the search efficiency enabled by each of the approaches we define a cost metric. The total amount of work required to process a query against a collection is typically proportional to the number of documents that match the query 6 . The volume of index data transferred and processed for a query (the dominant cost) as well as the computational cost 4 http://trec.nist.gov/data/docs eng.html 5 http://www.lemurproject.org/indri/ 6 Every document that contains at least one of the query terms is considered as a matched document for the query.…”

“…Query routing is complicated further by the fact that peers are not naturally topically clustered [31]. Hence, clustering peers by content [14,20] is a possible solution as it has been shown to make query processing more efficient [6].…”

“…A topical cluster of cooperative peers is similar to a topical shard. However, no mechanism exists to dynamically determine the number of peer clusters to contact for a given query in this set-up, for example: Bawa et al [6] proposed a heuristic cutoff equal to a quarter of the number of topical clusters. Furthermore, clustering mechanism may fail to discover the structure of the document distribution and may not work well for unpopular topics.…”

Leveraging Heterogeneity Make Resource Location Efficient in Peer-to-Peer Networks

“…In comparison, some other researchers take advantage of the different types of nodes' shared resources to improve the search efficiency in P2P networks by resource topic segment technology [16], the key idea is to arrange participating nodes in a topic-segmented overlay topology in which most connections are short-distance, connecting pairs of nodes with similar resources. Topically focused sets of nodes are then joined together into a single network by long-distance links.…”

A Dynamic Cluster Construction Method Based on Query Characteristics in Peer-to-Peer Networks

“…In order to improve the efficiency of information retrieval in P2P networks, there have been many researches on categorizing data items based on their similarities and clustering peers that have similar data items into the same category [1] - [8], [10] - [13]. These methods propagate query messages to all peers in a cluster.…”