Storage and search in dynamic peer-to-peer networks

Abstract

We study robust and efficient distributed algorithms for searching, storing, and maintaining data in dynamic Peer-to-Peer (P2P) networks. P2P networks are highly dynamic networks that experience heavy node churn (i.e., nodes join and leave the network continuously over time). Our goal is to guarantee, despite high node churn rate, that a large number of nodes in the network can store, retrieve, and maintain a large number of data items. Our main contributions are fast randomized distributed algorithms that guarantee the above with high probability even under high adversarial churn. In particular, we present the following main results:1. A randomized distributed search algorithm that with high probability guarantees that searches from as many as n − o(n) nodes (n is the stable network size) succeed in O(log n)-rounds despite O(n/ log 1+δ n) churn, for any small constant δ > 0, per round. We assume that the churn is controlled by an oblivious adversary (that has complete knowledge and control of what nodes join and leave and at what time and has unlimited computational power, but is oblivious to the random choices made by the algorithm).2. A storage and maintenance algorithm that guarantees, with high probability, data items can be efficiently stored (with only Θ(log n) copies of each data item) and maintained in a dynamic P2P network with churn rate up to O(n/ log 1+δ n) per round. Our search algorithm together with our storage and maintenance algorithm guarantees that as many as n − o(n) nodes can efficiently store, maintain, and search even under O(n/ log 1+δ n) churn per round. Our algorithms require only polylogarithmic in n bits to be processed and sent (per round) by each node.To the best of our knowledge, our algorithms are the first-known, fully-distributed storage and search algorithms that provably work under highly dynamic settings (i.e., high churn rates per step). Furthermore, they are localized (i.e., do not require any global topological knowledge) and scalable. A technical contribution of this paper, which may be of independent interest, is

Enabling Robust and Efficient Distributed Computation in Dynamic Peer-to-Peer Networks

“…The protocol serves as a building block for enabling algorithms for fundamental distributed computing problems such as agreement, search and storage, and leader election in dynamic P2P networks. In particular, the expander maintenance protocol in conjunction with the protocols for agreement [19], and search and storage [22] enables a fully-distributed and autonomous protocol for these problems.…”

“…The work of [59] presented an efficient distributed algorithm for Byzantine leader election that could tolerate up to O(n 1/2−ε ) Byzantine nodes (for a small constant ε > 0) and up to O( √ n/ polylog(n) churn per round, took a polylogarithmic number of rounds. The work of [22] focussed on the problem of storing, maintaining, and searching data in P2P networks. It presented a storage and maintenance algorithm that guaranteed with high probability, that data items can be efficiently stored (with only O(1) copies of each data item) and maintained in a dynamic P2P network with churn rate up to O(n/ polylog(n)) per round.…”

“…However, none of these works guarantee the maintenance of an expander under the more challenging situation of high continuous adversarial churn. This is a fundamental ingredient that is needed to enable the applicability of previous results ( [19], [21], [22], [59]) under a more realistic setting.…”

Fast Byzantine Leader Election in Dynamic Networks

“…There has been a lot of recent work on distributed agreement, byzantine agreement, and fault-tolerant protocols in dynamic networks. We refer to [17,6,4,3] and the references therein for details on these works. Here we restrict ourselves to those works that are most closely related.…”

Desenvolvimento de mecanismos para auxiliar no estudo e seleção de técnicas de armazenamento distribuído de arquivos em ambientes de computação multinuvem

“…Em Augustine et al (2013), é proposto um algoritmo para armazenamento e recuperação de dados em um ambiente P2P altamente dinâmico com muita rotatividade entre os nós. nesse trabalho, o IDA foi utilizado para providenciar um overhead menor no tamanho dos arquivos em relação às técnicas de replicação tradicionais.…”