Distribution-based query scheduling

Abstract

Query scheduling, a fundamental problem in database management systems, has recently received a renewed attention, perhaps in part due to the rise of the "database as a service" (DaaS) model for database deployment. While there has been a great deal of work investigating different scheduling algorithms, there has been comparatively little work investigating what the scheduling algorithms can or should know about the queries to be scheduled. In this work, we investigate the efficacy of using histograms describing the distribution of likely query execution times as input to the query scheduler. We propose a novel distribution-based scheduling algorithm, Shepherd, and show that Shepherd substantially outperforms state-of-the-art point-based methods through extensive experimentation with both synthetic and TPC workloads.

Distribution Based Workload Modelling of Continuous Queries in Clouds

“…using system load dynamics monitoring) [8], or predictive techniques (e.g. estimating the workload performance) [6], [9], [16], [30], [31] for decision making. In all these predictive approaches, they estimate the workload performance as a single point value [6], [16], assume that the PDF for the workload execution time is already available [31], or estimate (and not predict) the PDF using sampling based techniques [9], [30].…”

“…As the single point estimation gives no clue of the confidence on the estimation, they use a committee based ML model in the next work [30]. Along similar lines, [9] approximates the probability distribution using a histogram-based approach. However, this approach is only a simple approximation of distribution based on a number of already collected query execution A number of inquiries [18], [21], [24] have been made into the elasticity management of either data-intensive systems or single/multi-tier web applications using control theory [21], [24], Queueing theory [25], fuzzy logic [18], [19] and so on.…”

Uncertainty aware query execution time prediction

“…The problem of predicting query running time is revitalized by the recent move towards providing database as a service (DaaS). Many important decision-making procedures, including admission control [43,49], query scheduling [14], and system sizing [45], rely on estimation of query running time. Distributional information enables more robust decision procedures in contrast to point estimates.…”

“…Distributional information enables more robust decision procedures in contrast to point estimates. Recent work [14] has shown the benefits in query scheduling by leveraging distributional information. Similar ideas have also been raised in [49] for admission control.…”

“…The problem of predicting query execution time has received a great deal of recent research attention (e.g., [6,7,19,20,47,48]). Knowledge about query execution time is essential to many important database management issues, including query optimization, admission control [43,49], query scheduling [14], and system sizing [45]. Existing predictors rely on information such as cardinality estimates and system performance constants that are difficult to know exactly.…”