Load balancing in large-scale systems with multiple dispatchers

Boor, Mark van der; Borst, Sem; Leeuwaarden, Johan van

doi:10.1109/infocom.2017.8057012

Cited by 19 publications

(22 citation statements)

References 24 publications

(31 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In these cases the fluid limit, for suitable initial states, is the same as that for a single dispatcher, and in particular the fixed point is the same, hence, the JIQ scheme continues to achieve asymptotically optimal delay performance with minimal communication overhead. As one of the few exceptions, [7] allows the loads at the various dispatchers to be different.…”

Section: Multiple Dispatchersmentioning

confidence: 99%

See 1 more Smart Citation

Scalable Load Balancing in Networked Systems: Universality Properties and Stochastic Coupling Methods

Boor

Borst

Leeuwaarden

et al. 2019

Proceedings of the International Congress of Mathematicians (ICM 2018)

Self Cite

View full text Add to dashboard Cite

We present an overview of scalable load balancing algorithms which provide favorable delay performance in large-scale systems, and yet only require minimal implementation overhead. Aimed at a broad audience, the paper starts with an introduction to the basic load balancing scenario -referred to as the supermarket model -consisting of a single dispatcher where tasks arrive that must immediately be forwarded to one of N single-server queues. The supermarket model is a dynamic counterpart of the classical balls-and-bins setup where balls must be sequentially distributed across bins.A popular class of load balancing algorithms are so-called power-of-d or JSQ(d) policies, where an incoming task is assigned to a server with the shortest queue among d servers selected uniformly at random. As the name reflects, this class includes the celebrated Join-the-Shortest-Queue (JSQ) policy as a special case (d = N ), which has strong stochastic optimality properties and yields a mean waiting time that vanishes as N grows large for any fixed subcritical load. However, a nominal implementation of the JSQ policy involves a prohibitive communication burden in large-scale deployments. In contrast, a simple random assignment policy (d = 1) does not entail any communication overhead, but the mean waiting time remains constant as N grows large for any fixed positive load.In order to examine the fundamental trade-off between delay performance and implementation overhead, we consider an asymptotic regime where the diversity parameter d(N ) depends on N . We investigate what growth rate of d(N ) is required to match the optimal performance of the JSQ policy on fluid and diffusion scale, and achieve a vanishing waiting time in the limit. The results demonstrate that the asymptotics for the JSQ(d(N )) policy are insensitive to the exact growth rate of d(N ), as long as the latter is sufficiently fast, implying that the optimality of the JSQ policy can asymptotically be preserved while dramatically reducing the communication overhead.Stochastic coupling techniques play an instrumental role in establishing the asymptotic optimality and universality properties, and augmentations of the coupling constructions allow these properties to be extended to infinite-server settings and network scenarios. We additionally show how the communication overhead can be reduced yet further by the so-called Join-the-Idle-Queue (JIQ) scheme, leveraging memory at the dispatcher to keep track of idle servers.In the present paper we review scalable load balancing algorithms (LBAs) which achieve excellent delay performance in large-scale systems and yet only involve low implementation overhead. LBAs play a critical role in distributing service requests or tasks (e.g. compute jobs, data base look-ups, file transfers) among servers or distributed resources in parallel-processing systems. The analysis and design of LBAs has attracted strong attention in recent years, mainly spurred by crucial scalability challenges arising in cloud networks and data centers with massive...

show abstract

Section: Multiple Dispatchersmentioning

confidence: 99%

“…In order to counter the above-described performance degradation for asymmetric dispatcher loads, [7] proposes two enhancements. Enhancement A uses a non-uniform token allotment: When a server becomes idle, it sends a token to dispatcher r with probability β r .…”

Section: Multiple Dispatchersmentioning

confidence: 99%

Scalable Load Balancing in Networked Systems: Universality Properties and Stochastic Coupling Methods

Boor

Borst

Leeuwaarden

et al. 2019

Proceedings of the International Congress of Mathematicians (ICM 2018)

Self Cite

View full text Add to dashboard Cite

show abstract

“…More precisely, we could easily generalize our algorithm to server pools with several load balancers, each with their own bucket. The corresponding queueing model, still tractable using known results on networks of quasireversible queues [11], extends that of [16].…”

Section: Serversmentioning

confidence: 98%

Dynamic Load Balancing with Tokens

Comte¹

2018

2018 IFIP Networking Conference (IFIP Networking) and Workshops

View full text Add to dashboard Cite

Efficiently exploiting the resources of data centers is a complex task that requires efficient and reliable load balancing and resource allocation algorithms. The former are in charge of assigning jobs to servers upon their arrival in the system, while the latter are responsible for sharing server resources between their assigned jobs. These algorithms should take account of various constraints, such as data locality, that restrict the feasible job assignments. In this paper, we propose a token-based mechanism that efficiently balances load between servers without requiring any knowledge on job arrival rates and server capacities. Assuming a balanced fair sharing of the server resources, we show that the resulting dynamic load balancing is insensitive to the job size distribution. Its performance is compared to that obtained under the best static load balancing and in an ideal system that would constantly optimize the resource utilization. * c IFIP, 2018. This is the author's version of the work. It is posted here by permission of IFIP for your personal use. Not for redistribution. The definitive version will be published in IFIP NETWORKING 2018 proceedings.

show abstract

“…Load balancing on cloud computing has attracted many researchers around the world and has also gained important achievements [1][2][3][4][5][6], [8][9][10][11]. Load balancing in communication is a very important factor in improving the performance of cloud data centers.…”

Section: Related Workmentioning

confidence: 99%

“…Mark van der Boor et al [8] introduce two enhancements of the ordinary JIQ scheme where tokens are either distributed non-uniformly or occasionally exchanged among the various dispatchers. Join the-Idle-Queue (JIQ) algorithms, which rely on tokens issued by idle servers in dispatching tasks.…”

Section: Related Workmentioning

confidence: 99%