LOCAT: Low-Overhead Online Configuration Auto-Tuning of Spark SQL Applications

Abstract

Spark SQL has been widely deployed in industry but it is challenging to tune its performance. Recent studies try to employ machine learning (ML) to solve this problem. They however suffer from two drawbacks. First, it takes a long time (high overhead) to collect training samples. Second, the optimal configuration for one input data size of the same application might not be optimal for others.To address these issues, we propose a novel Bayesian Optimization (BO) based approach named LOCAT to automatically tune the configurations of Spark SQL applications online. LOCAT innovates three techniques. The first technique, named QCSA, eliminates the configuration-insensitive queries by Query Configuration Sensitivity Analysis (QCSA) when collecting training samples. The second technique, dubbed DAGP, is a Datasize-Aware Gaussian Process (DAGP) which models the performance of an application as a distribution of functions of configuration parameters as well as input data size. The third technique, called IICP, Identifies Important Configuration Parameters (IICP) with respect to performance and only tunes the important parameters. As such, LOCAT can tune the configurations of a Spark SQL application with low overhead and adapt to different input data sizes.We employ Spark SQL applications from benchmark suites𝑇 𝑃𝐶− 𝐷𝑆, 𝑇 𝑃𝐶 − 𝐻 , and 𝐻𝑖𝐵𝑒𝑛𝑐ℎ running on two significantly different clusters, a four-node ARM cluster and an eight-node x86 cluster, to evaluate LOCAT. The experimental results on the ARM cluster show that LOCAT accelerates the optimization procedures of Tuneful [22], DAC [66], GBO-RL [36], and QTune [37] by factors of 6.4×, 7.0×, 4.1×, and 9.7× on average, respectively. On the x86 cluster, LOCAT reduces the optimization time of Tuneful, DAC, GBO-RL, and QTune by factors of 6.4×, 6.3×, 4.0×, and 9.2× on average, respectively. Moreover, LOCAT improves the performance of the applications on

Rover: An online Spark SQL tuning service via generalized transfer learning

“…CherryPick [1] directly performs BO on a discretized search space. LOCAT [38] further combines dynamic sensitivity analysis and datasize-aware Gaussian process (GP) to perform optimization on important parameters. Despite the competitive converged results, the aforementioned methods suffer from the re-optimization issue [23], which is, the performance model needs retraining and still requires a number of online configuration evaluations for each coming task.…”

“…Benchmarks. For end-to-end comparison, we follow LOCAT [38] and use three SQL-related tasks from the widely used Spark benchmark HiBench [16]: (1) 'Join' is a query that executes in two phases: if M ≠ ∅ then 3:…”

“…optimization. We first run Bayesian optimization over three different search spaces: 1) Rover space (10 params); 2) the Tuneful [8] space (30 params); 3) the Tuneful space with dynamic shrinking used in LOCAT [38]. The results are shown in Figure 8(a).…”

“…Recent studies [1,8,38] apply the Bayesian optimization (BO) framework to reduce the required number of evaluations to find a near-optimal configuration. In brief, BO trains a surrogate on evaluated configurations and their performance, and then selects the next configuration by balancing exploration and exploitation.…”

“…Baselines. For end-to-end comparison, we compare Rover with the following SOTA BO-based tuning algorithms: (1) CherryPick [1]: A method with a discretized search space; (2) Tuneful [8]: A method that explores significant parameters and applies a multi-task GP to use the most similar previous task; (3) LOCAT [38] that identifies important parameters and dynamically reduces the search space; (4) ResTune [44]: A transfer learning method that uses all the history knowledge to accelerate the tuning process. Experiment Settings.…”