Efficient schemes for nearest neighbor load balancing

Diekmann, Ralf; Frommer, Andreas; Monien, Burkhard

doi:10.1016/s0167-8191(99)00018-6

Cited by 127 publications

(120 citation statements)

References 11 publications

Supporting

Mentioning

116

Contrasting

Unclassified

Order By: Relevance

“…, N } and a measure x i for every node i ∈ V . The average consensus problem consists in computing the average x A = N −1 i x i in an iterative distributed way, exchanging information among nodes exclusively along the available edges in G. This problem appears in a number of different contexts since the early 80's (decentralized computation [1], load balancing [2], [3], [4]) and, recently, has attracted much attention for possible applications to sensor networks (data fusion problems [5], [6], [7], [8], [9], clock syncronization [10]) and to coordinated control for mobile autonomous agents [11], [12], [13], [14], [15], [16], [17]. Other places where consensus algorithms have been studied in general are [18], [19], [20], [21], [22], [23] Several algorithms for average consensus can be found in the literature: they differentiate on the basis of the amount of communication and computation they use, on their scalability with respect to the number of nodes, on their adaptability to time-varying graphs, and, finally, they can be deterministic or random.…”

Section: Introductionmentioning

confidence: 99%

“…Most of papers study the same algorithm: every node runs a first order linear dynamical system to update its estimation and the systems are coupled through the available communication edges. Different schemes (higher order, with memory) however have shown up in the literature, see [4], [3], [22]. The type of problem typically faces in the literature are: necessary and sufficient conditions for convergence, speed of convergence, optimization issues.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Randomized consensus algorithms over large scale networks

Fagnani

Zampieri

2007

2007 Information Theory and Applications Workshop

134

236

View full text Add to dashboard Cite

Abstract-Suppose we have a directed graph G with set of nodes V = {1, . . . , N } and a measure x i for every node i ∈ V . The average consensus problem consists in computing the average x A = N −1 P i x i in an iterative way, exchanging information among nodes exclusively along the available edges in G. This problem appears in a number of different contexts since the 80's (decentralized computation, load balancing, clock syncronization) and, recently, has attracted much attention for possible applications to sensor networks (data fusion problems) and to coordinated control for mobile autonomous agents. Several algorithms for average consensus can be found in the literature: they differentiate on the basis of the amount of communication and computation they use, on their scalability with respect to the number of nodes, on their adaptability to time-varying graphs, and, finally, they can be deterministic or random. In this presentation we will focus on random algorithms: we will review some algorithms present in the literature and we will propose some new ones. We will present some performance results which will allow to make some comparison. Finally, we will establish some probabilistic concentration results which will give a stronger significance to previous results.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Randomized consensus algorithms over large scale networks

Fagnani

Zampieri

2007

2007 Information Theory and Applications Workshop

134

236

View full text Add to dashboard Cite

show abstract

“…Here the load transferred over an edge (i, j) in step t does not only depend on the load difference of i and j, but also on the amount of load transferred over the edge in step t − 1. Diekmann, Frommer, and Monien [4] extend the idea of [13] and propose a general framework to analyze the convergence behavior of a wide range of diffusion type methods.…”

Section: Continuous Diffusionmentioning

confidence: 99%

Randomized diffusion for indivisible loads

Berenbrink

Cooper

Friedetzky

et al. 2015

Journal of Computer and System Sciences

View full text Add to dashboard Cite

The full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. AbstractWe present a new randomized diffusion-based algorithm for balancing indivisible tasks (tokens) on a network. Our aim is to minimize the discrepancy between the maximum and minimum load. The algorithm works as follows. Every vertex distributes its tokens as evenly as possible among its neighbors and itself. If this is not possible without splitting some tokens, the vertex redistributes its excess tokens among all its neighbors randomly (without replacement).In this paper we prove several upper bounds on the load discrepancy for general networks. These bounds depend on some expansion properties of the network, that is, the second largest eigenvalue, and a novel measure which we refer to as refined local divergence. We then apply these general bounds to obtain results for some specific networks. For constant-degree expanders and torus graphs, these yield exponential improvements on the discrepancy bounds compared to the algorithm of Rabani, Sinclair, and Wanka [14]. For hypercubes we obtain a polynomial improvement. In contrast to previous papers, our algorithm is vertex-based and not edge-based. This means excess tokens are assigned to vertices instead to edges, and the vertex reallocates all of its excess tokens by itself. This approach avoids nodes having "negative loads" (like in [8,10]), but causes additional dependencies for the analysis.

show abstract

“…The diffusion scheme is a simple and well-known scheme sending data to neighboring processes [19,20,21,22]. Another scheme, proposed in [11], is also based on sending tasks to neighbors.…”

Section: Related Workmentioning

confidence: 99%

Dynamic load balancing for petascale quantum Monte Carlo applications: The Alias method

Sudheer

Krishnan

Srinivasan

et al. 2013

Computer Physics Communications

View full text Add to dashboard Cite

Diffusion Monte Carlo is a highly accurate Quantum Monte Carlo method for the electronic structure of materials, but it requires frequent load balancing or population redistribution steps to maintain efficiency on parallel machines. This step can be a significant factor affecting performance, and will become more important as the number of processing elements increases. We propose a new dynamic load balancing algorithm, the Alias Method, and evaluate it theoretically and empirically. An important feature of the new algorithm is that the load can be perfectly balanced with each process receiving at most one message. It is also optimal in the maximum size of messages received by any process. We also optimize its implementation to reduce network contention, a process facilitated by the low messaging requirement of the algorithm: a simple renumbering of the MPI ranks based on proximity and a space filling curve significantly improves the MPI Allgather performance. Empirical results on the petaflop Cray XT Jaguar supercomputer at ORNL showing up to 30% improvement in performance on 120,000 cores. The load balancing algorithm may be straightforwardly implemented in existing codes. The algorithm may also be employed by any method with many near identical computational tasks that requires load balancing.

show abstract

Efficient schemes for nearest neighbor load balancing

Cited by 127 publications

References 11 publications

Randomized consensus algorithms over large scale networks

Randomized consensus algorithms over large scale networks

Randomized diffusion for indivisible loads

Dynamic load balancing for petascale quantum Monte Carlo applications: The Alias method

Contact Info

Product

Resources

About