Dynamic Load Balancing with Tokens

Abstract

Efficiently exploiting the resources of data centers is a complex task that requires efficient and reliable load balancing and resource allocation algorithms. The former are in charge of assigning jobs to servers upon their arrival in the system, while the latter are responsible for sharing server resources between their assigned jobs. These algorithms should take account of various constraints, such as data locality, that restrict the feasible job assignments. In this paper, we propose a token-based mechanism that efficiently balances load between servers without requiring any knowledge on job arrival rates and server capacities. Assuming a balanced fair sharing of the server resources, we show that the resulting dynamic load balancing is insensitive to the job size distribution. Its performance is compared to that obtained under the best static load balancing and in an ideal system that would constantly optimize the resource utilization. * c IFIP, 2018. This is the author's version of the work. It is posted here by permission of IFIP for your personal use. Not for redistribution. The definitive version will be published in IFIP NETWORKING 2018 proceedings.

Pass-and-swap queues

“…We shall also see that, in addition to their theoretical appeal, P&S queues can be used to describe the dynamics of load-distribution and scheduling protocols in a cluster of machines in which jobs have assignment constraints. This cluster model, which can represent various computer clusters or manufacturing systems in which not every machine is able to fulfill the service requirement of any job, has played a central role in several studies of product-form queueing models over the past decade; see, for example, [2][3][4][5]9,[12][13][14][15]. Roughly speaking, this cluster model can be interpreted as a P&S queue where customers represent jobs and servers represent machines.…”

“…We also study irreducibility properties of closed networks of P&S queues and demonstrate that, under particular assumptions, the stationary distribution of such closed networks also has a product form. These closed networks form a class of independent interest, since we show later that they can be used to model finite-capacity queues with token-based structures, akin to those of [4] and [12,13].…”

“…Since (12) represents the same set of equations as (7), it is already known that (3) satisfies (12). Showing that (3) satisfies ( 13) is equivalent to showing that the balance function in (4) satisfies the following equation in each state c = (c 1 , .…”

“…This cluster model can represent various queueing systems, like the computer clusters or manufacturing systems mentioned in the introduction, in which not every machine is able to fulfill the service requirement of any job. It has played a central role in several studies of product-form queueing models over the past decade; see, for example, [2][3][4][5]9,[12][13][14][15]. We now explain how P&S queues can be applied to analyze the performance of existing and new load-distribution and scheduling protocols in such clusters of machines.…”

“…1. These models arise naturally in many timely applications, such as redundancy scheduling [9,15] and load balancing [12,13] in computer systems, resource management in manufacturing systems and call centers [2,3], and multiple instances of stochastic matching models [1,22]. Although the dynamics of these queues are rather intricate, their stationary distributions all have a product form, which facilitates exact derivation of performance measures.…”