Exploiting Historical Data: Pruning Autotuning Spaces and Estimating the Number of Tuning Steps

Abstract

Autotuning, the practice of automatic tuning of applications to provide performance portability, has received increased attention in the research community, especially in high performance computing. Ensuring high performance on a variety of hardware usually means modifications to the code, often via different values of a selected set of parameters, such as tiling size, loop unrolling factor or data layout. However, the search space of all possible combinations of these parameters can be large, which can result in cases where the benefits of autotuning are outweighed by its cost, especially with dynamic tuning. Therefore, estimating the tuning time in advance or shortening the tuning time is very important in dynamic tuning applications.We have found that certain properties of tuning spaces do not vary much when hardware is changed. In this paper, we demonstrate that it is possible to use historical data to reliably predict the number of tuning steps that is necessary to find a wellperforming configuration, and to reduce the size of the tuning space. We evaluate our hypotheses on a number of HPC benchmarks written in CUDA and OpenCL, using several different generations of GPUs and CPUs.

Exploiting historical data: Pruning autotuning spaces and estimating the number of tuning steps

“…As we can see in the tables, both methods are able to significantly reduce the size of the state space, while sacrificing only a few percent of performance in most cases. On GPU, the two pruning strategies usually result in very different search spaces-in fact, the results of the conservative pruning strategy are very similar to the results of the naive pruning method from our previous work, 16 since in most cases, the parameters were pruned based on low mutual information rather than performance degradation.…”

“…On GPU, the two pruning strategies usually result in very different search spaces—in fact, the results of the conservative pruning strategy are very similar to the results of the naive pruning method from our previous work, 16 since in most cases, the parameters were pruned based on low mutual information rather than performance degradation.…”

“…TA B L E 8 State space reduction (SSR), the number of pruned/all parameters (PP), and performance retention for all tested hardware/benchmark combinations, after conservative pruning of parameters with low mutual information on GPU On GPU, the two pruning strategies usually result in very different search spaces-in fact, the results of the conservative pruning strategy are very similar to the results of the naive pruning method from our previous work, 16 since in most cases, the parameters were pruned based on low mutual information rather than performance degradation.…”

“…For the purposes of this article, we will call TA B L E 7 State space reduction (SSR), the number of pruned/all parameters (PP), and performance retention for all tested hardware/benchmark combinations, after aggressive pruning of parameters with low mutual information on GPU this the naive pruning method. We have already evaluated this approach in our previous work, 16 and while it worked very well for most problems, there was a case where too many parameters were pruned and the well-performing configurations were lost. This was, however, not caused by poor correlation in parameter significance between different hardware generations-even on the hardware that was used to select the parameters, the space was overly pruned.…”

Using hardware performance counters to speed up autotuning convergence on GPUs

“…On the other hand, our model does not need to be trained on multiple inputs and GPUs. In our previous work [26], we used data measured on one hardware to prune dimensions on different hardware. While this approach works well for speeding up exhaustive search, it brings no advantage when coupled with a searcher based on mathematical optimization.…”