Regularized Cost-Model Oblivious Database Tuning with Reinforcement Learning

Basu, Debabrota; Lin, Qian; Chen, Weidong; Vo, Hoang Tam; Yuan, Zihong; Senellart, Pierre; Bressan, Stéphane

doi:10.1007/978-3-662-53455-7_5

Cited by 18 publications

(10 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Scholars also apply reinforcement learning to solve the problems on other purposes of the data center task scheduling. Basu et al applied reinforcement learning to build cost-models on standard online transaction processing datasets [22]. ey modeled the execution of queries and updates as a Markov decision process whose states are database configurations, actions are configuration changes, and rewards are functions of the cost of configuration change and query and update evaluation.…”

Section: Reinforcement Learning-based Studiesmentioning

confidence: 99%

A Deep Reinforcement Learning Approach to the Optimization of Data Center Task Scheduling

et al. 2020

View full text Add to dashboard Cite

With more businesses are running online, the scale of data centers is increasing dramatically. The task-scheduling operation with traditional heuristic algorithms is facing the challenges of uncertainty and complexity of the data center environment. It is urgent to use new technology to optimize the task scheduling to ensure the efficient task execution. This study aimed at building a new scheduling model with deep reinforcement learning algorithm, which integrated the task scheduling with resource-utilization optimization. The proposed scheduling model was trained, tested, and compared with classical scheduling algorithms on real data center datasets in experiments to show the effectiveness and efficiency. The experiment report showed that the proposed algorithm worked better than the compared classical algorithms in the key performance metrics: average delay time of tasks, task distribution in different delay time levels, and task congestion degree.

show abstract

Section: Reinforcement Learning-based Studiesmentioning

confidence: 99%

A Deep Reinforcement Learning Approach to the Optimization of Data Center Task Scheduling

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Past efforts have considered more general reinforcement learning (RL) for physical design tuning [25], [35]. Compared to most MAB approaches, deep RL invites over parameterisation, which can slow convergence (see Figure 8), whereas MAB typically provides better convergence, simpler implementation, and safety guarantees via strategic exploration and knowledge transfer (see Section III).…”

Section: Why Not (General) Reinforcement Learning?mentioning

confidence: 99%

“…Furthermore, this classifier incurs up to 10% recommendation time, impacting recommendation cost in all cases, especially where recommendation cost already dominates the cost for PDTool (TPC-DS, IMDb). When it comes to tuning, the closest approaches employ variants of RL index selection or partitioning [25], [35], [46] or configuration tuning [5]. [35] describes RL-based index selection, which depends solely on the recommendation tool for query-level recommendations and is affected by decision combinatorial explosion, both issues addressed in our work.…”

Section: Related Workmentioning

confidence: 99%

“…When it comes to tuning, the closest approaches employ variants of RL index selection or partitioning [25], [35], [46] or configuration tuning [5]. [35] describes RL-based index selection, which depends solely on the recommendation tool for query-level recommendations and is affected by decision combinatorial explosion, both issues addressed in our work. Unlike its more general counterpart (RL), MABs have advantages of faster convergences as demonstrated in Section V-C, simple implementation, and theoretical guarantees.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

DBA bandits: Self-driving index tuning under ad-hoc, analytical workloads with safety guarantees

Perera¹,

Oetomo²,

Rubinstein³

et al. 2020

Preprint

View full text Add to dashboard Cite

Automating physical database design has remained a long-term interest in database research due to substantial performance gains afforded by optimised structures. Despite significant progress, a majority of today's commercial solutions are highly manual, requiring offline invocation by database administrators (DBAs) who are expected to identify and supply representative training workloads. Even the latest advancements like query stores provide only limited support for dynamic environments. This status quo is untenable: identifying representative static workloads is no longer realistic; and physical design tools remain susceptible to the query optimiser's cost misestimates (stemming from unrealistic assumptions such as attribute value independence and uniformity of data distribution).We propose a self-driving approach to online index selection that eschews the DBA and query optimiser, and instead learns the benefits of viable structures through strategic exploration and direct performance observation. We view the problem as one of sequential decision making under uncertainty, specifically within the bandit learning setting. Multi-armed bandits balance exploration and exploitation to provably guarantee average performance that converges to a fixed policy that is optimal with perfect hindsight. Our comprehensive empirical results demonstrate up to 75% speed-up on shifting and ad-hoc workloads and 28% speed-up on static workloads compared against a state-ofthe-art commercial tuning tool.

show abstract

“…Specifically, unsupervised ML techniques can model the data distribution for cardinality estimation (CardEst) [14,39,41,42,46] and indexing [6,7,18,27]; supervised ML models can replace the cost estimator (CostEst) [25,34,35] and execution scheduler [23,31]; and reinforcement learning methods solve decision making # The second and third authors contribute equally to this paper. problems such as configuration tuning [1,20,44] and join order selection (JoinSel) [12,22,24,29,43]. Motivation: Despite these ML methods' promising results on each individual task, the existing ML techniques in DBMS do not explore the following transferabilities and will inevitably lead to impractical solutions and/or ineffective models.…”

Section: Introductionmentioning

confidence: 99%

A Unified Transferable Model for ML-Enhanced DBMS

Wu¹,

Yan²,

Yu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, the database management system (DBMS) community has witnessed the power of machine learning (ML) solutions for DBMS tasks. Despite their promising performance, these existing solutions can hardly be considered satisfactory. First, these MLbased methods in DBMS are not effective enough because they are optimized on each specific task, and cannot explore or understand the intrinsic connections between tasks. Second, the training process has serious limitations that hinder their practicality, because they need to retrain the entire model from scratch for a new DB. Moreover, for each retraining, they require an excessive amount of training data, which is very expensive to acquire and unavailable for a new DB. We propose to explore the transferabilities of the ML methods both across tasks and across DBs to tackle these fundamental drawbacks.In this paper, we propose a unified model MTMLF that uses a multi-task training procedure to capture the transferable knowledge across tasks and a pre-train fine-tune procedure to distill the transferable meta knowledge across DBs. We believe this paradigm is more suitable for cloud DB service, and has the potential to revolutionize the way how ML is used in DBMS. Furthermore, to demonstrate the predicting power and viability of MTMLF, we provide a concrete and very promising case study on query optimization tasks. Last but not least, we discuss several concrete research opportunities along this line of work.

show abstract

Regularized Cost-Model Oblivious Database Tuning with Reinforcement Learning

Cited by 18 publications

References 30 publications

A Deep Reinforcement Learning Approach to the Optimization of Data Center Task Scheduling

A Deep Reinforcement Learning Approach to the Optimization of Data Center Task Scheduling

DBA bandits: Self-driving index tuning under ad-hoc, analytical workloads with safety guarantees

A Unified Transferable Model for ML-Enhanced DBMS

Contact Info

Product

Resources

About