Restrained utilization of idleness for transparent scheduling of background tasks

Abstract

A common practice in system design is to treat features intended to enhance performance and reliability as low priority tasks by scheduling them during idle periods, with the goal to keep these features transparent to the user. In this paper, we present an algorithmic framework that determines the schedulability of non-preemptable low priority tasks in storage systems. The framework estimates when and for how long idle times can be utilized by low priority background tasks, without violating pre-defined performance targets of user foreground tasks. The estimation is based on monitored system information that includes the histogram of idle times. This histogram captures accurately important statistical characteristics of the complex demands of the foreground activity. The robustness and the effectiveness of the proposed framework is corroborated via extensive trace driven simulations under a wide range of system conditions and background activities, and via experimentation on a Linux kernel 2.6.22 prototype.

Workload interleaving with performance guarantees in data centers

“…In [71,72], the authors propose a framework to estimate when and for how long to utilize idle periods in a system for processing low priority background tasks without violating pre-defined foreground performance targets. This is achieved by extending the non-work-conserving nature of background scheduling as first suggested in [40,30].…”

“…The consequence is that the system may remain idle while background tasks are still outstanding after the estimated time to utilize an idle interval for background scheduling elapses. Key to the methodology developed in [71,72] are the statistical characteristics of idle times which are used for effective background task scheduling.…”

“…The work that is most related to the work presented here is [71], where the authors propose a framework to estimate when and for how long idle periods can be used for processing low priority background tasks without violating pre-defined foreground performance target. We generalize this algorithm by introducing a new analytic formulation.…”

“…We generalize this algorithm by introducing a new analytic formulation. The algorithm part is based on [71], but the developed extensions allow the system to serve background work as fast as possible. Experiments are driven by a set of traces collected in a distributed scenario.…”

“…In some cases the non-linear interaction of different co-located jobs is taken into account [63]. Instead, we look to use as much of the spare capacity as possible for time-insensitive background tasks, as in the case of a server handling the continuous and bursty workload of foreground user traffic while also intending to perform replication, integrity checking, data analysis, or other work [71,130].…”