Randomized load balancing with general service time distributions

Abstract

Randomized load balancing greatly improves the sharing of resources in a number of applications while being simple to implement. One model that has been extensively used to study randomized load balancing schemes is the supermarket model. In this model, jobs arrive according to a rate-nλ Poisson process at a bank of n rate-1 exponential server queues. A notable result, due to Vvedenskaya et.al. (1996), showed that when each arriving job is assigned to the shortest of d ≥ 2 randomly chosen queues, the equilibrium queue sizes decay doubly exponentially in the limit as n → ∞. This is a substantial improvement over the case d = 1, where queue sizes decay exponentially.The method of analysis used in the above paper and in the subsequent literature applies to jobs with exponential service time distributions and does not easily generalize. It is desirable to study load balancing models with more general, especially heavy-tailed, service time distributions since such service times occur widely in practice.This paper describes a modularized program for treating randomized load balancing problems with general service time distributions and service disciplines. The program relies on an ansatz which asserts that any finite set of queues in a randomized load balancing scheme becomes independent as n → ∞. This allows one to derive queue size distributions and other performance measures of interest. We establish the ansatz when the service discipline is FIFO and the service time distribution has a decreasing hazard rate (this includes heavy-tailed service times). Assuming the ansatz, we also obtain the following results: (i) as n → ∞, the process of job arrivals at any fixed queue tends to a Poisson process whose rate depends on the size of the queue, (ii) when the service discipline at each server is processor sharing or LIFO with preemptive resume, the distribution of the number of jobs is insensitive to the service distribution, and (iii) the tail behavior of the queue-size distribution in terms of the service distribution for the FIFO service discipline.

Choosing among heterogeneous server clouds

“…Such independence holds at any finite time and also at the equilibrium, provided that the initial server occupancies satisfy certain assumptions. This is formally known as the propagation of chaos [11,36] or asymptotic independence property [5,6] in the literature.…”

“…This is because (10)- (11) show that du ( j) k /dt is nondecreasing in u (i) n for n = k and i = j [8]. Since this implies that 5 Recall that the generator A of the semigroup {T(t)} t≥0 acting on functions f : U → R having bounded partial derivatives is given by A f (g) = lim t↓0…”

Asymptotic independence of queues under randomized load balancing

“…This allows computation of queue size distributions and other quantities of interest. Employing the ansatz, it is shown in Bramson et al [6] that the limiting equilibrium distribution will sometimes have a doubly exponential tail, but that other behavior is also possible, depending on the service discipline and the tail of the service distribution F (·).…”

“…We next consider SQ(N) policies, which we are only able to analyze when the service discipline is FIFO and the service time distribution has a decreasing hazard rate (DHR). This includes heavy-tailed service distributions and is shown in [6] to lead to interesting phenomena. Last, we show the ansatz holds for a sufficiently small arrival rate, with no assumptions on the policy for selecting a queue, as long as the service distribution has 2 moments.…”

On a doubly dynamically controlled supermarket model with impatient customers

“…Luczak and McDiarmid [33] showed that the length of the longest queue scales as ðlog log NÞ=log d þ Oð1Þ. Certain generalization of the supermarket model has been explored in studying various variations, for example, modeling more crucial factors by Vvedenskaya and Suhov [51], Mitzenmacher [38], Mitzenmacher et al [39], Bramson et al [10][11][12], Li and Lui [30,31], Li et al [32,29] and Li [27,28]; fast Jackson networks by Martin and Suhov [35], Martin [34] and Suhov and Vvedenskaya [46].…”