Autotuning of OpenCL Kernels with Global Optimizations

Filipovič, Jiří; Petrovič, Filip; Benkner, Siegfried

doi:10.1145/3152821.3152877

Cited by 19 publications

(17 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They may select one of the predefined variants of a tuned function [34], or generate and compile implementations according to the values of the tuning parameters. We distinguish between compiler-based tuning, where the space of code transformation is generated automatically [37,43,39] and user-defined code optimization parameters autotuning [35,13,36,15]. User-defined code optimization parameters tuning requires expert programmers to identify and implement tuning possibilities in the source code manually (e. g., by using preprocessor macros).…”

Section: Related Workmentioning

confidence: 99%

“…The KTT API has been derived from the CLTune project [35], so it is very similar to CLTune when we use it for offline tuning. Additionally, KTT API allows for tuning compositions of multiple kernels, tuning of how kernels are called from host code [15], and novel features for dynamic tuning.…”

Section: Architecture Of the Kernel Tuning Toolkitmentioning

confidence: 99%

“…The reduction benchmark computes the sum of all elements in an input vector. We have used the autotuned implementation from our previous work [15]. There are two special optimizations affected by tuning parameters and not listed in Table 2:…”

Section: Reductionmentioning

confidence: 99%

See 2 more Smart Citations

A benchmark set of highly-efficient CUDA and OpenCL kernels and its dynamic autotuning with Kernel Tuning Toolkit

Petrovič

Střelák

Hozzová

et al. 2020

Future Generation Computer Systems

Self Cite

View full text Add to dashboard Cite

In recent years, the heterogeneity of both commodity and supercomputers hardware has increased sharply. Accelerators, such as GPUs or Intel Xeon Phi co-processors, are often key to improving speed and energy efficiency of highly-parallel codes. However, due to the complexity of heterogeneous architectures, optimization of codes for a certain type of architecture as well as porting codes across different architectures, while maintaining a comparable level of performance, can be extremely challenging. Addressing the challenges associated with performance optimization and performance portability, autotuning has gained a lot of interest. Autotuning of performance-relevant source-code parameters allows to automatically tune applications without hard coding optimizations and thus helps with keeping the performance portable. In this paper, we introduce a benchmark set of ten autotunable kernels for important computational problems implemented in OpenCL or CUDA. Using our Kernel Tuning Toolkit, we show that with autotuning most of the kernels reach near-peak performance on various GPUs and outperform baseline implementations on CPUs and Xeon Phis. Our evaluation also demonstrates that autotuning is key to performance portability. In addition to offline tuning, we also introduce dynamic autotuning of code optimization parameters during application runtime. With dynamic tuning, the Kernel Tuning Toolkit enables applications to re-tune performance-critical kernels at runtime whenever needed, for example, when input data changes. Although it is generally believed that autotuning spaces tend to be too large to be searched during application runtime, we show that it is not necessarily the case when tuning spaces are designed rationally. Many of our kernels reach near peak-performance with moderately sized tuning spaces that can be searched at runtime with acceptable overhead. Finally we demonstrate, how dynamic performance tuning can be integrated into a real-world application from cryo-electron microscopy domain.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Architecture Of the Kernel Tuning Toolkitmentioning

confidence: 99%

See 1 more Smart Citation

A benchmark set of highly-efficient CUDA and OpenCL kernels and its dynamic autotuning with Kernel Tuning Toolkit

Petrovič

Střelák

Hozzová

et al. 2020

Future Generation Computer Systems

Self Cite

View full text Add to dashboard Cite

show abstract

“…The benchmarks are composed of important computational kernels spanning across multiple application domains: 3D Fourier Reconstruction 9 and 2D convolution (adopted from CLTune 3 ) are image processing kernels, BiCG, GEMM (adopted from CLTune 3 ), GEMM batched, Matrix Transpose and Reduction 10 are linear algebra kernels, direct Coulomb summation 10 is a computational chemistry kernel, N‐body (autotuned version of NVIDIA CUDA SDK sample) and Hotspot (based on implementation from Rodinia benchmark 11 ) are differential equation solvers. These benchmarks autotune a variety of tuning parameters, changing implementation properties such as work‐group size, cache blocking, thread coarsening, explicit caching in local memory, loop unrolling, explicit vectorization or data layout optimization (ie, array of structures vs structure of arrays).…”

Section: Benchmark Setmentioning

confidence: 99%

Exploiting historical data: Pruning autotuning spaces and estimating the number of tuning steps

Oľha

Hozzová

Fousek

et al. 2020

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Autotuning, the practice of automatic tuning of applications to provide performance portability, has received increased attention in the research community, especially in high performance computing. Ensuring high performance on a variety of hardware usually means modifications to the code, often via different values of a selected set of parameters, such as tiling size, loop unrolling factor, or data layout. However, the search space of all possible combinations of these parameters can be large, which can result in cases where the benefits of autotuning are outweighed by its cost, especially with dynamic tuning. Therefore, estimating the tuning time in advance or shortening the tuning time is very important in dynamic tuning applications. We have found that certain properties of tuning spaces do not vary much when hardware is changed. In this article, we demonstrate that it is possible to use historical data to reliably predict the number of tuning steps that is necessary to find a well-performing configuration and to reduce the size of the tuning space. We evaluate our hypotheses on a number of HPC benchmarks written in CUDA and OpenCL, using several different generations of GPUs and CPUs.

show abstract

“…We have implemented several optimization strategies, which may easily interfere with each other. Thus, we have used a Kernel Tuning Toolkit (KTT) (Filipovič et al, 2017), to automatically search for the optimal combination of optimizations.…”

Section: Gpu Implementationmentioning

confidence: 99%

A GPU acceleration of 3-D Fourier reconstruction in cryo-EM

Střelák

Sorzano

Carazo

et al. 2019

The International Journal of High Performance Computing Applica

Self Cite

View full text Add to dashboard Cite

Cryo-electron microscopy is a popular method for macromolecules structure determination. Reconstruction of a 3-D volume from raw data obtained from a microscope is highly computationally demanding. Thus, acceleration of the reconstruction has a great practical value. In this article, we introduce a novel graphics processing unit (GPU)-friendly algorithm for direct Fourier reconstruction, one of the main computational bottlenecks in the 3-D volume reconstruction pipeline for some experimental cases (particularly those with a large number of images and a high internal symmetry). Contrary to the state of the art, our algorithm uses a gather memory pattern, improving cache locality and removing race conditions in parallel writing into the 3-D volume. We also introduce a finely tuned CUDA implementation of our algorithm, using auto-tuning to search for a combination of optimization parameters maximizing performance on a given GPU architecture. Our CUDA implementation is integrated in widely used software Xmipp, version 3.19, reaching 11.4× speedup compared to the original parallel CPU implementation using GPU with comparable power consumption. Moreover, we have reached 31.7× speedup using four GPUs and 2.14×–5.96× speedup compared to optimized GPU implementation based on a scatter memory pattern.

show abstract

Autotuning of OpenCL Kernels with Global Optimizations

Cited by 19 publications

References 19 publications

A benchmark set of highly-efficient CUDA and OpenCL kernels and its dynamic autotuning with Kernel Tuning Toolkit

A benchmark set of highly-efficient CUDA and OpenCL kernels and its dynamic autotuning with Kernel Tuning Toolkit

Exploiting historical data: Pruning autotuning spaces and estimating the number of tuning steps

A GPU acceleration of 3-D Fourier reconstruction in cryo-EM

Contact Info

Product

Resources

About