High-Dimensional Bayesian Optimization with Multi-Task Learning for RocksDB

Abstract

RocksDB is a general-purpose embedded key-value store used in multiple different settings. Its versatility comes at the cost of complex tuning configurations. This paper investigates maximizing the throughput of RocksDB IO operations by auto-tuning ten parameters of varying ranges. Off-theshelf optimizers struggle with high-dimensional problem spaces and require a large number of training samples.We propose two techniques to tackle this problem: multitask modeling and dimensionality reduction through clustering. By incorporating adjacent optimization in the model, the model converged faster and found complicated settings that other tuners could not find. This approach had an additional computational complexity overhead, which we mitigated by manually assigning parameters to each sub-goal through our knowledge of RocksDB. The model is then incorporated in a standard Bayesian Optimization loop to find parameters that maximize RocksDB's IO throughput.Our method achieved x1.3 improvement when benchmarked against a simulation of Facebook's social graph traffic, and converged in ten optimization steps compared to other state-of-the-art methods that required fifty steps.CCS Concepts: • Information systems → Key-value stores; • Computing methodologies → Gaussian processes.

JointConf: Jointly autotuning configuration parameters for modularized graph databases

“…For instance, JanusGraph itself mainly focuses on graph serialization and query execution, while providing adapters to integrate third‐party softwares as its functional module for data storage and indices. Unfortunately, although there are already a few significant works toward automatically tuning parameters for different databases such as HBase, 4 Elasticsearch, 5 RocksDB, 6 and MySQL, 7 these solutions cannot be directly applied in the scenarios of modularized GDBs because they solely consider one specific software. What is worse, due to the complicated interactions across different modules, sequentially tuning each software with previous solutions may also fail to efficiently find the optimal configuration for modularized GDBs.…”