Kill Two Birds with One Stone: Auto-tuning RocksDB for High Bandwidth and Low Latency

Jia, Yichen; Chen, Feng

doi:10.1109/icdcs47774.2020.00113

Cited by 2 publications

(1 citation statement)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SHAMan [33] is another blackbox auto-tuning framework for HPC I/O that allows selection between various heuristics, including regression-based surrogate modeling, simulated annealing, and genetic algorithms. Similarly in [34] the authors use a number of heuristics to perform autotuning of LSM-tree-based key/value storage systems, with a direct application to RocksDB. CAPES [35] adopts deep-Q learning, a deep reinforcement approach and an alternative to genetic algorithms and Bayesian optimization, to optimize the performance of Lustre file systems, which is an alternative to genetic algorithms and Bayesian optimization.…”

Section: A Autotuning Hpc Storage Servicesmentioning

confidence: 99%

HPC Storage Service Autotuning Using Variational- Autoencoder -Guided Asynchronous Bayesian Optimization

Dorier

Égelé

Balaprakash

et al. 2022

2022 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

Distributed data storage services tailored to specific applications have grown popular in the high-performance computing (HPC) community as a way to address I/O and storage challenges. These services offer a variety of specific interfaces, semantics, and data representations. They also expose many tuning parameters, making it difficult for their users to find the best configuration for a given workload and platform.To address this issue, we develop a novel variationalautoencoder-guided asynchronous Bayesian optimization method to tune HPC storage service parameters. Our approach uses transfer learning to leverage prior tuning results and use a dynamically updated surrogate model to explore the large parameter search space in a systematic way.We implement our approach within the DeepHyper opensource framework, and apply it to the autotuning of a highenergy physics workflow on Argonne's Theta supercomputer. We show that our transfer-learning approach enables a more than 40× search speedup over random search, compared with a 2.5× to 10× speedup when not using transfer learning. Additionally, we show that our approach is on par with state-ofthe-art autotuning frameworks in speed and outperforms them in resource utilization and parallelization capabilities.

show abstract

Section: A Autotuning Hpc Storage Servicesmentioning

confidence: 99%