Simple efficient load balancing algorithms for peer-to-peer systems

Abstract

Load balancing is a critical issue for the efficient operation of peerto-peer networks. We give two new load-balancing protocols whose provable performance guarantees are within a constant factor of optimal. Our protocols refine the consistent hashing data structure that underlies the Chord (and Koorde) P2P network. Both preserve Chord's logarithmic query time and near-optimal data migration cost.Consistent hashing is an instance of the distributed hash table (DHT) paradigm for assigning items to nodes in a peer-to-peer system: items and nodes are mapped to a common address space, and nodes have to store all items residing closeby in the address space.Our first protocol balances the distribution of the key address space to nodes, which yields a load-balanced system when the DHT maps items "randomly" into the address space. To our knowledge, this yields the first P2P scheme simultaneously achieving O(log n) degree, O(log n) look-up cost, and constant-factor load balance (previous schemes settled for any two of the three).Our second protocol aims to directly balance the distribution of items among the nodes. This is useful when the distribution of items in the address space cannot be randomized. We give a simple protocol that balances load by moving nodes to arbitrary locations "where they are needed." As an application, we use the last protocol to give an optimal implementation of a distributed data structure for range searches on ordered data.

Wikipedia workload analysis for decentralized hosting

“…Third, the unbalanced page popularity distribution will most likely result in the nodes responsible for the most popular pages being saturated with requests while the rest of the system observes a low load. Note that load balancing within a DHT is considered a difficult problem for which a number of specific solutions exist [19]. Finally, similarly to DistriWiki, Piki requires a special user interface application to read pages from the system, which discards its application for a popular web site like Wikipedia.…”

Online Balancing of Range-Partitioned Data with Applications to Peer-to-Peer Systems

“…In a concurrent work, Karger and Ruhl [17] provide an alternative solution to the storage balance problem. The scheme is a randomized algorithm that offers a highprobability bound on the imbalance ratio, and is analyzed under a dynamic, but non-adversarial, setting.…”

“…However, the best achievable bound on imbalance ratio using this algorithm appears to be more than 128, which is much higher than the load imbalance bounds we guarantee. Bharambe et al [7] also use a load-balancing scheme similar to [17] as a heuristic to balance range partitions.…”

Efficient range query processing over DHTs based on the balanced Kautz tree

“…Many algorithms have been proposed, such as ID selection [21], virtual servers [22], and item balancing [23], to support load-balanced data mapping in DHTs. But these algorithms can neither generate the Kautz strings required by DK nor guarantee to return the results in a bounded delay.…”