Redundancy Scheduling in Systems with Bi-Modal Job Service Time Distributions

Abstract

Queuing systems with redundant requests have drawn great attention because of their promise to reduce the job completion time and variability. Despite a large body of work on the topic, we are still far from fully understanding the benefits of redundancy in practice. We here take one step towards practical systems by studying queuing systems with bi-modal job service time distribution. Such distributions have been observed in practice, as can be seen in, e.g., Google cluster traces. We develop an analogy to a classical urns and balls problem, and use it to study the queuing time performance of two non-adaptive classical scheduling policies: random and round-robin. We introduce new performance indicators in the analogous model, and argue that they are good predictors of the queuing time in non-adaptive scheduling policies. We then propose a non-adaptive scheduling policy that is based on combinatorial designs, and show that it has better performance indicators. Simulations confirm that the proposed scheduling policy, as the performance indicators suggest, reduces the queuing times compared to random and round-robin scheduling.

Scheduling in the Presence of Data Intensive Compute Jobs

“…To prove our arguments, in the following, we develop an analogy to the classical Urns&Balls problem and drive our performance indicators in the analogue problem. Interested readers may refer to [3] for detailed analysis.…”

“…In distributed computing systems, redundancy can reduce fluctuations in the service time of the jobs [2], and has gained attention both in theoretical works [3]- [5] and in practical systems [6], [7]. Redundancy is particularly beneficial in systems with high variability in job service times [8].…”

“…In systems with redundancy, if all the redundant copies of a job get queued behind the copies of a data intensive job then redundancy brings no benefit to the performance. The round-robin policy increases the chance of this phenomenon [3].…”

Balanced Nonadaptive Redundancy Scheduling

“…Block designs are combinatorial structures with many applications, including design of experiments, software testing, cryptography, and finite and algebraic geometry. They have recently been considered for straggler mitigation in distributed machine learning [1], [26], [27]. The importance of such structures come from their ability to regulate overlaps between the copies of different jobs, as shown in [1].…”

“…They have recently been considered for straggler mitigation in distributed machine learning [1], [26], [27]. The importance of such structures come from their ability to regulate overlaps between the copies of different jobs, as shown in [1]. This paper is organized as follows.…”