A scalable auto-tuning framework for compiler optimization

Abstract

We describe a scalable and general-purpose framework for auto-tuning compiler-generated code. We combine Active Harmony's parallel search backend with the CHiLL compiler transformation framework to generate in parallel a set of alternative implementations of computation kernels and automatically select the one with the best-performing implementation. The resulting system achieves performance of compiler-generated code comparable to the fully automated version of the ATLAS library for the tested kernels. Performance for various kernels is 1.4 to 3.6 times faster than the native Intel compiler without search. Our search algorithm simultaneously evaluates different combinations of compiler optimizations and converges to solutions in only a few tens of search-steps.

Using meta-heuristics and machine learning for software optimization of parallel computing systems: a systematic literature review

“…Greedy Algorithm and Hill Climbing Cooper et al [21] use meta-heuristics to find the optimal compilation parameters while reducing the number of evaluations during search space exploration from 10000 to a single one using profiling data and estimated virtual execution Genetic Algorithm Sandrieser et al [88] obtain speedup of 23% for hyper-block formation Parallel Rank Ordering Tiwari and Hollingsworth [93] and Tiwari et al [92] use PRO for automatic tuning of compilation process and report 46% performance improvement compared to the original code [57,58,99] Determining the best partitioning strategy Assumes that for any two functions with similar features, the same partitioning strategy can be used [95] Determining loops that benefit from parallelization and their best scheduling policy Targets OpenMP loop constructs only. Uses profiling to detect loop candidates, which may significantly increase the compilation time [1,21] Adaptive tuning of the compilation process Profiling data needs to be collected to perform the virtual executions.…”

Analytical Bounds for Optimal Tile Size Selection

“…This section demonstrates the integration of the analytical bounds with existing search optimization algorithms, the Nelder-Mead Simplex method [22] and the Parallel Rank Ordering (PRO) method [30]. In order to handle boundary constraints due to the DL/ML model, we used the extended version of the PRO algorithm introduced in the Active Harmony framework [32]. The same extension to handle boundaries was employed in our implementation of the Simplex method, and its stopping criteria are based on the work by Luersen [21].…”

“…In contrast, we have demonstrated (see Table 3) that the best performance is often realized only for rectangular tiles. Search-based techniques for finding tile sizes (and unroll factors) have received much attention in performance optimization [4,19,31,32,33]. The ATLAS system employs extensive empirical tuning to find the best tile sizes for different problem sizes in the BLAS library; tuning is done once at installation.…”

“…Tiwari et al [32] note: "a key challenge that faces auto-tuners, especially as we expand the scope of their capabilities, involves scalable search among alternative implementations." The Active Harmony project [31,32] uses several different algorithms to reduce the size of the search space such as the Nelder-Mead simplex algorithm.…”

“…Empirical approaches to tile size optimization treat the loop nest as a black box, and perform empirical auto-tuning for a given architecture [4,31,32,33] by actually executing the tiled code for a range of different tile sizes. The highly successful ATLAS (Automatically Tuned Linear Algebra Software) system [33] uses empirical tuning at library installation time to find the best tile sizes for different problem sizes on the target machine.…”