Autotuning CUDA compiler parameters for heterogeneous applications using the OpenTuner framework

Abstract

A Graphics Processing Unit (GPU) is a parallel computing coprocessor specialized in accelerating vector operations. The enormous heterogeneity of parallel computing platforms justifies and motivates the development of automated optimization tools and techniques. The Algorithm Selection Problem consists in finding a combination of algorithms, or a configuration of an algorithm, that optimizes the solution of a set of problem instances. An autotuner solves the Algorithm Selection Problem using search and optimization techniques.In this paper, we implement an autotuner for the Compute Unified Device Architecture compiler's parameters using the OpenTuner framework. The autotuner searches for a set of compilation parameters that optimizes the time to solve a problem. We analyze the performance speedups, in comparison with high-level compiler optimizations, achieved in three different GPU devices, for 17 heterogeneous GPU applications, 12 of which are from the Rodinia Benchmark Suite. The autotuner often beats the compiler's high-level optimizations, but underperformed for some problems. We achieved over 2x speedup for Gaussian Elimination and almost 2x speedup for Heart Wall, both problems from the Rodinia Benchmark, and over 4x speedup for a matrix multiplication algorithm. problem defines a search space. Various optimization techniques search this space, guided by the performance metrics, for the algorithm or configuration that best solve the problem.There are specialized autotuners for domains such as matrix multiplication [2], dense [3], or sparse [4] matrix linear algebra and parallel programming [5]. Other autotuning frameworks provide more general tools for the representation and search of program configurations, enabling the implementation of autotuners for different problem domains [6,7].The OpenTuner framework [6] provides tools for the implementation of autotuners for various problem domains. It implements different search techniques that explore the same search space for program optimizations. Running and measuring program execution time -that is, the empirical exploration of the search space -is done sequentially. The framework also provides support for parallel compilation.In this paper, we implemented an autotuner for the CUDA compiler using the use the OpenTuner framework [6] and used it to search for the compilation parameters that optimize the performance of 17 heterogeneous GPU applications, 12 of which are from the Rodinia Benchmark Suite [8]. We used three different NVIDIA GPUs in the experiments, the Tesla K40, the GTX 980, and the GTX 750.Our main contribution is to show that it is possible to optimize code written for GPUs by automatically tuning just the parameters of the CUDA compiler. We propose a thorough methodology for analyzing result correctness and checking for invalid flag combinations and compilation errors. The optimization achieved by autotuning often beats the compiler high-level optimization options, such as -O1, -O2, and -O3. The autotuner found compilation options that achieved...

ATF: A generic directive‐based auto‐tuning framework

“…We employ the CLTune program that CLBlast uses for tuning XgemmDirect, and we implement the OpenTuner program for this kernel according to the work of Bruel et al, where we use the unconstrained search space; we report a penalty value in case of a configuration for which XgemmDirect's constraints are not satisfied.…”

“…Due to these dependencies, OpenTuner is not capable of auto‐tuning GEMM. For this restriction, the OpenTuner community has offered workarounds, eg, re‐designing the user program so that its tuning parameters become independent or setting a penalty value for configurations where the constraints are not met . However, the first workaround usually requires a significant effort from the user, while the second may cause a poor tuning result as we demonstrate for GEMM in Section 7.…”

“…In contrast, CLTune is only suitable for auto-tuning programs written in OpenCL and only in terms of runtime performance. ATF provides OpenCL-specific tuning directives for auto-tuning OpenCL programs (Listing 2, line [11][12][13][14][15][16][17][18][19][20]. We argue that the usage of ATF for OpenCL is better than CLTune due to the following reasons.…”

“…configurations where the constraints are not met. 12 However, the first workaround usually requires a significant effort from the user, while the second may cause a poor tuning result as we demonstrate for GEMM in Section 7. Moreover, OpenTuner is optimized for programs whose tuning parameters have large ranges, and thus, it does not provide search techniques for small ranges, eg, exhaustive search that finds the probably best result.…”

“…shows the OpenTuner program for auto-tuning GCC's optimization options for raytracer. The user defines one tuning parameter per option (line[8][9][10][11][12][13][14][15][16][17][18][19][23][24][25][26][27][28][29][30][31][32][33][34][35][36][37][38], -322 in total -by overriding the manipulator function (line 23) of OpenTuner's class MeasurementInterface (line 21) and by using a so-called OpenTuner configuration manipulator (line 28); The OpenTuner provides a straightforward Python script for extracting the GCC's options used in line[8][9][10][11][12][13][14][15][16][17][18][19]. To define the cost function, the user overrides OpenTuner's run function (line 40-61); in this function, he has to explicitly construct a GCC command with the optimization options according to the input configuration (line 45-55).…”