Distributing Unit Size Workload Packages in Heterogeneous Networks

Elsässer, Robert; Monien, Burkhard; Schamberger, Stefan

doi:10.7155/jgaa.00118

Cited by 22 publications

(35 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, one should be able to send 0.5 tasks, 0.1 tasks, etc. Modified versions of diffusive type schemes have also been proposed which remove restrictions on arbitrary divisibility [23]. Multi-level graph partitioning based dynamic schemes are also popular [24].…”

Section: Related Workmentioning

confidence: 99%

Dynamic load balancing for petascale quantum Monte Carlo applications: The Alias method

Sudheer

Krishnan

Srinivasan

et al. 2013

Computer Physics Communications

View full text Add to dashboard Cite

Diffusion Monte Carlo is a highly accurate Quantum Monte Carlo method for the electronic structure of materials, but it requires frequent load balancing or population redistribution steps to maintain efficiency on parallel machines. This step can be a significant factor affecting performance, and will become more important as the number of processing elements increases. We propose a new dynamic load balancing algorithm, the Alias Method, and evaluate it theoretically and empirically. An important feature of the new algorithm is that the load can be perfectly balanced with each process receiving at most one message. It is also optimal in the maximum size of messages received by any process. We also optimize its implementation to reduce network contention, a process facilitated by the low messaging requirement of the algorithm: a simple renumbering of the MPI ranks based on proximity and a space filling curve significantly improves the MPI Allgather performance. Empirical results on the petaflop Cray XT Jaguar supercomputer at ORNL showing up to 30% improvement in performance on 120,000 cores. The load balancing algorithm may be straightforwardly implemented in existing codes. The algorithm may also be employed by any method with many near identical computational tasks that requires load balancing.

show abstract

Section: Related Workmentioning

confidence: 99%

Dynamic load balancing for petascale quantum Monte Carlo applications: The Alias method

Sudheer

Krishnan

Srinivasan

et al. 2013

Computer Physics Communications

View full text Add to dashboard Cite

show abstract

“…Here we consider the (arguably more realistic [15]) case of discrete diffusion where tokens are indivisible. Quantifying by how much the integrality assumption decreases the efficiency of load balancing is an interesting question and has been posed by many authors (e.g., [8,[11][12][13][14][15]). …”

Section: Introductionmentioning

confidence: 99%

“…This means excess tokens are assigned to vertices instead to edges, and the vertex reallocates all of its excess tokens by itself. This approach avoids nodes having "negative loads" (like in [8,10]), but causes additional dependencies for the analysis. …”

mentioning

confidence: 99%

Randomized diffusion for indivisible loads

Berenbrink

Cooper

Friedetzky

et al. 2015

Journal of Computer and System Sciences

View full text Add to dashboard Cite

The full-text may be used and/or reproduced, and given to third parties in any format or medium, without prior permission or charge, for personal research or study, educational, or not-for-prot purposes provided that:• a full bibliographic reference is made to the original source • a link is made to the metadata record in DRO • the full-text is not changed in any way The full-text must not be sold in any format or medium without the formal permission of the copyright holders.Please consult the full DRO policy for further details. AbstractWe present a new randomized diffusion-based algorithm for balancing indivisible tasks (tokens) on a network. Our aim is to minimize the discrepancy between the maximum and minimum load. The algorithm works as follows. Every vertex distributes its tokens as evenly as possible among its neighbors and itself. If this is not possible without splitting some tokens, the vertex redistributes its excess tokens among all its neighbors randomly (without replacement).In this paper we prove several upper bounds on the load discrepancy for general networks. These bounds depend on some expansion properties of the network, that is, the second largest eigenvalue, and a novel measure which we refer to as refined local divergence. We then apply these general bounds to obtain results for some specific networks. For constant-degree expanders and torus graphs, these yield exponential improvements on the discrepancy bounds compared to the algorithm of Rabani, Sinclair, and Wanka [14]. For hypercubes we obtain a polynomial improvement. In contrast to previous papers, our algorithm is vertex-based and not edge-based. This means excess tokens are assigned to vertices instead to edges, and the vertex reallocates all of its excess tokens by itself. This approach avoids nodes having "negative loads" (like in [8,10]), but causes additional dependencies for the analysis.

show abstract

“…A very natural question is how much this integrality assumption decreases the efficiency of load balancing. In fact, finding a precise quantitative relationship between the discrete and the idealized case is an open problem posed by many authors, e.g., [9,11,15,16,28,31,33,36].…”

mentioning

confidence: 99%

“…We measure the smoothness of the load by the so-called discrepancy (see e.g. [9,11,16,33]) which is the difference between the maximum and minimum load among all nodes.…”

mentioning

confidence: 99%

Quasirandom Load Balancing

Friedrich¹,

Gairing²,

Sauerwald

2012

SIAM J. Comput.

View full text Add to dashboard Cite

Abstract. We propose a simple distributed algorithm for balancing indivisible tokens on graphs. The algorithm is completely deterministic, though it tries to imitate (and enhance) a randomized algorithm by keeping the accumulated rounding errors as small as possible.Our new algorithm surprisingly closely approximates the idealized process (where the tokens are divisible) on important network topologies. On d-dimensional torus graphs with n nodes it deviates from the idealized process only by an additive constant. In contrast, the randomized rounding approach of Friedrich and Sauerwald [11] can deviate up to Ω(polylog(n)) and the deterministic algorithm of Rabani, Sinclair and Wanka [33] has a deviation of Ω(n 1/d ). This makes our quasirandom algorithm the first known algorithm for this setting which is optimal both in time and achieved smoothness. We further show that on the hypercube as well, our algorithm has a smaller deviation from the idealized process than the previous algorithms.To prove these results, we derive several combinatorial and probabilistic results that we believe to be of independent interest. In particular, we show that first-passage probabilities of a random walk on a path with arbitrary weights can be expressed as a convolution of independent geometric probability distributions.1. Introduction. Load balancing is a requisite for the efficient utilization of computational resources in parallel and distributed systems. The aim is to reallocate the load such that afterward, each node has approximately the same load. Load balancing problems have various applications, e.g., for scheduling [37], routing [5], and numerical computation [38,39].Typically, load balancing algorithms iteratively exchange load along edges of an undirected connected graph. In the natural diffusion paradigm, an arbitrary amount of load can be sent along each edge at each step [31,33]. For the idealized case of divisible load, a popular diffusion algorithm is the first-order-scheme by Subramanian and Scherson [36] whose convergence rate is fairly well captured in terms of the spectral gap [27].However, for many applications the assumption of divisible load may be invalid. Therefore, we consider the discrete case where the load can only be decomposed into indivisible unit-size tokens. A very natural question is how much this integrality assumption decreases the efficiency of load balancing. In fact, finding a precise quantitative relationship between the discrete and the idealized case is an open problem posed by many authors, e.g., [9,11,15,16,28,31,33,36].A simple method for approximating the idealized process was analyzed by Rabani, Sinclair, and Wanka [33]. Their approach (which we will call "RSW algorithm") is to round down the fractional flow of the idealized process. They introduce a very useful parameter of the graph called local divergence and prove that it gives tight upper bounds on the deviation between the idealized process and their discrete process. However, one drawback of the RSW algorithm is that it can ...

show abstract

Distributing Unit Size Workload Packages in Heterogeneous Networks

Cited by 22 publications

References 26 publications

Dynamic load balancing for petascale quantum Monte Carlo applications: The Alias method

Dynamic load balancing for petascale quantum Monte Carlo applications: The Alias method

Randomized diffusion for indivisible loads

Quasirandom Load Balancing

Contact Info

Product

Resources

About