Arpeggio: Metadata Searching and Content Sharing with Chord

Abstract

Abstract. Arpeggio is a peer-to-peer file-sharing network based on the Chord lookup primitive. Queries for data whose metadata matches a certain criterion are performed efficiently by using a distributed keywordset index, augmented with index-side filtering. We introduce index gateways, a technique for minimizing index maintenance overhead. Because file data is large, Arpeggio employs subrings to track live source peers without the cost of inserting the data itself into the network. Finally, we introduce postfetching, a technique that uses information in the index to improve the availability of rare files. The result is a system that provides efficient query operations with the scalability and reliability advantages of full decentralization, and a content distribution system tuned to the requirements and capabilities of a peer-to-peer network. Overview and Related WorkPeer-to-peer file sharing systems, which let users locate and obtain files shared by other users, have many advantages: they operate more efficiently than the traditional client-server model by utilizing peers' upload bandwidth, and can be implemented without a central server. However, many current file sharing systems trade-off scalability for correctness, resulting in systems that scale well but sacrifice completeness of search results or vice-versa.Distributed hash tables have become a standard for constructing peer-to-peer systems because they overcome the difficulties of quickly and correctly locating peers. However, the lookup by name DHT operation is not immediately sufficient to perform complex search by content queries of the data stored in the network. It is not clear how to perform searches without sacrificing scalability or query completeness. Indeed, the obvious approaches to distributed full-text document search scale poorly [9].In this paper, however, we consider systems, such as file sharing, that search only over a relatively small amount of metadata associated with each file, but that have to support highly dynamic and unstable network topology, content,

MetaFlow: A Scalable Metadata Lookup Service for Distributed File Systems in Data Centers

“…On average, each lookup request needs to interact with O(log N ) nodes to locate a metadata object. For example, Arpeggio [20], a peer-to-peer file-sharing network, uses Chord to support distributed metadata lookup. (ii) One-Hop [21].…”

HyperDex

“…Although Mercury's implementation is unavailable, Cassandra uses a similar design and its performance illustrates how a multi-ring-based system would perform. Arpeggio [13] provides search over multiple attributes by enumerating and creating an index of all k x fixed-size subsets of attributes using a Chord ring. Both of these approaches insert redundant pointers into rings without concern for consistency.…”

Scalable ultrapeer-based DHT protocol for file sharing

“…Otherwise, dynamic querying method increases the value of TTL of its queries and floods the query messages until the corresponding user gets enough query results from the network. Table (DHT) is typically used for building distributed lookup services of structured P2P systems [3,4,6,10,16,18,9,15]. DHT is implemented by organizing N peers in a structured overlay P2P system.…”