PQR: Predicting Query Execution Times for Autonomous Workload Management

Gupta, Chetan; Mehta, Abhay; Dayal, Umeshwar

doi:10.1109/icac.2008.12

Cited by 75 publications

(50 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By using interaction-aware techniques, we are able to achieve prediction accuracy similar to [13] for batch BI workloads with complex interacting queries. Machine learning techniques are used in [16] to predict the range of execution time for a query in a database system. Similarly, machine learning is employed for database provisioning in [9,14].…”

Section: Related Workmentioning

confidence: 99%

Predicting completion times of batch query workloads using interaction-aware models and simulation

Ahmad

Duan

Aboulnaga

et al. 2011

Proceedings of the 14th International Conference on Extending Database Technology

View full text Add to dashboard Cite

A question that database administrators (DBAs) routinely need to answer is how long a batch query workload will take to complete. This question arises, for example, while planning the execution of different report-generation workloads to fit within available time windows. To answer this question accurately, we need to take into account that the typical workload in a database system consists of mixes of concurrent queries. Interactions among different queries in these mixes need to be modeled, rather than the conventional approach of considering each query separately. This paper presents a new approach for estimating workload completion times that takes the significant impact of query interactions into account. This approach builds performance models using an experimentdriven technique, by sampling the space of possible query mixes and fitting statistical models to the observed performance at these samples. No prior assumptions are made about the internal workings of the database system or the cause of query interactions, making the models robust and portable. We show that a careful choice of sampling and statistical modeling strategies can result in accurate models, and we present a novel interaction-aware workload simulator that uses these models to estimate workload completion times. An experimental evaluation with complex TPC-H queries on IBM DB2 shows that this approach consistently predicts workload completion times with less than 20% error.

show abstract

Section: Related Workmentioning

confidence: 99%

Predicting completion times of batch query workloads using interaction-aware models and simulation

Ahmad

Duan

Aboulnaga

et al. 2011

Proceedings of the 14th International Conference on Extending Database Technology

View full text Add to dashboard Cite

show abstract

“…Several factors affect the execution time of a database command, including the physical database design, the system load, data that is main memory and hardware characteristics. Indeed, although there are several works on query execution-time estimation [8,25] and on query execution progress estimators [2,13,14], estimating the time required to execute a database command is still an open issue, and existing techniques are subject to a certain level of error.…”

Section: Execution Time-estimation In Database Systemsmentioning

confidence: 99%

Elections and Reputation for High Dependability and Performance in Distributed Workload Execution

Costa

Furtado

2015

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Reputation systems use historical information about a service as the basis to predict its future behavior. In this work, we present a reputation-based task scheduling strategy for distributed database systems, which efficiently schedules workloads composed (or not) by best-effort oriented tasks and by tasks that have execution-related constraints. Available data services may participate in an election to choose the service that would execute each task. During task scheduling, each candidate service commits itself to satisfy some requirements or to finish a task execution by a certain time, depending on the task's execution objective.In order to achieve high performance while still satisfying specified requirements, the election mechanism combines two reputation models: one classifies candidate services over their dependability (qualitative perspective), while the other is used to build expectations on a tasks execution time (quantitative perspective). Candidate classification mechanisms, recall elections and reelections rules are used to prevent performance or dependability degradation by the presence of malicious or erroneous services.We evaluated proposed strategies over widely used real-world DBMS (Oracle 11g and Microsoft SQL Server) using benchmark data and queries. Obtained results prove that the proposed scheduling mechanism leads to a highly dependable and efficient system even in the presence of malicious or erroneous data services.

show abstract

“…In [9] the authors also use machine learning to predict query performance. They predict a range for their query execution latency.…”

Section: Related Workmentioning

confidence: 99%

Performance prediction for concurrent database workloads

Duggan

Çetintemel

Papaemmanouil

et al. 2011

Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data

114

View full text Add to dashboard Cite

Current trends in data management systems, such as cloud and multi-tenant databases, are leading to data processing environments that concurrently execute heterogeneous query workloads. At the same time, these systems need to satisfy diverse performance expectations. In these newly-emerging settings, avoiding potential Quality-of-Service (QoS) violations heavily relies on performance predictability, i.e., the ability to estimate the impact of concurrent query execution on the performance of individual queries in a continuously evolving workload.This paper presents a modeling approach to estimate the impact of concurrency on query performance for analytical workloads. Our solution relies on the analysis of query behavior in isolation, pairwise query interactions and sampling techniques to predict resource contention under various query mixes and concurrency levels. We introduce a simple yet powerful metric that accurately captures the joint effects of disk and memory contention on query performance in a single value. We also discuss predicting the execution behavior of a time-varying query workload through queryinteraction timelines, i.e., a fine-grained estimation of the time segments during which discrete mixes will be executed concurrently. Our experimental evaluation on top of PostgreSQL/TPC-H demonstrates that our models can provide query latency predictions within approximately 20% of the actual values in the average case.

show abstract

PQR: Predicting Query Execution Times for Autonomous Workload Management

Cited by 75 publications

References 8 publications

Predicting completion times of batch query workloads using interaction-aware models and simulation

Predicting completion times of batch query workloads using interaction-aware models and simulation

Elections and Reputation for High Dependability and Performance in Distributed Workload Execution

Performance prediction for concurrent database workloads

Contact Info

Product

Resources

About