Tight bounds for parallel randomized load balancing

Abstract

Given a distributed system of n balls and n bins, how evenly can we distribute the balls to the bins, minimizing communication? The fastest non-adaptive and symmetric algorithm achieving a constant maximum bin load requires Θ(log log n) rounds, and any such algorithm running for r ∈ O(1) rounds incurs a bin load of Ω((log n/ log log n) 1/r ). In this work, we explore the fundamental limits of the general problem. We present a simple adaptive symmetric algorithm that achieves a bin load of 2 in log * n + O(1) communication rounds using O(n) messages in total. Our main result, however, is a matching lower bound of (1 − o(1)) log * n on the time complexity of symmetric algorithms that guarantee small bin loads. The essential preconditions of the proof are (i) a limit of O(n) on the total number of messages sent by the algorithm and (ii) anonymity of bins, i.e., the port numberings of balls need not be globally consistent. In order to show that our technique yields indeed tight bounds, we provide for each assumption an algorithm violating it, in turn achieving a constant maximum bin load in constant time.An extended abstract of preliminary work appeared at STOC 2011 [25] and the corresponding article has been published on arxiv [24].

Multiple-Choice Balanced Allocation in (Almost) Parallel

“…These assumptions are unrealistic in various load balancing applications, e.g., where the balls model the jobs in some parallel or distributed setting and the choices of the balls must be performed independently and in parallel, or in scenarios where the balls cannot easily access the current load of the bins, for example, because of the delay in receiving this information. To cope with this, various multiple-choice strategies have been developed for parallel environments to deal with concurrent requests [1,2,11,13] and communication delays [6,7,12]. They base their decisions on the number of parallel requests, allow extra rounds of communication, and in some cases let balls re-choose.…”

Balls-into-leaves

“…[1,17], which show that significant complexity gains can be achieved over the naive random balls-into-bins strategy. For a complete survey of existing results on parallel load-balancing, we refer the reader to [16]. To the best of our knowledge, none of the known parallel load-balancing techniques can be used to obtain sub-logarithmic wait-free tight renaming.…”

“…To the best of our knowledge, none of the known parallel load-balancing techniques can be used to obtain sub-logarithmic wait-free tight renaming. Existing work either relaxes the exact one-ball-per-bin requirement [17], or requires balls to always have consistent views when making their choice (which cannot be guaranteed under crash faults).…”

“…Moreover, seen as an assignment problem, renaming is related to the extensive line of work on randomized load balancing, e.g. [18,1,5,17]. Surprisingly, a careful analysis of existing load balancing techniques reveals that none of them can be used to achieve sub-logarithmic tight renaming, since they either are designed for a fault-free setting, or relax the one-to-one allocation requirement.…”

Algebraic Methods in the Congested Clique

“…As such, it has been recently gaining increasing attention [24,25,36,37,46,49,51,57,59,63], in an attempt to understand the relative computational power of distributed computing models.…”