Specializing Compiler Optimizations through Programmable Composition for Dense Matrix Computations

Abstract

General purpose compilers aim to extract the best average performance for all possible user applications. Due to the lack of specializations for different types of computations, compiler attained performance often lags behind those of the manually optimized libraries. In this paper, we demonstrate a new approach, programmable composition, to enable the specialization of compiler optimizations without compromising their generality. Our approach uses a single pass of sourcelevel analysis to recognize a common pattern among dense matrix computations. It then tags the recognized patterns to trigger a sequence of general-purpose compiler optimizations specially composed for them. We show that by allowing different optimizations to adequately communicate with each other through a set of coordination handles and dynamic tags inserted inside the optimized code, we can specialize the composition of general-purpose compiler optimizations to attain a level of performance comparable to those of manually written assembly code by experts, thereby allowing selected computations in applications to benefit from similar levels of optimizations as those manually applied by experts.

High-Performance Generalized Tensor Operations

“…Several approaches can fully automatically optimize MMA[×, +] and obtain high-performance code that outperforms expert-tuned implementations. The POET optimization library (Yi et al 2014) and AUGEM framework (Wang et al 2013) use annotations and templates of sequential code, respectively, written by domain experts to guide general-purpose compilers to produce optimized MMA[×, +] kernels from specifically prepared code. The Portable Compiler Approach (POCA) (Su et al 2017) generates an optimized micro-kernel based on LLVM IR representing MMA [×, +] and subsequent domain-specific but architecture-independent optimizations of its micro-kernel.…”

Hybrid-Grained Dynamic Load Balanced GEMM on NUMA Architectures

“…We implemented our method in the OpenBLAS [3] library and evaluated it on Phytium 2000+, an emerging high-performance many-core processor based on Arm's AArch64 architecture. We restrict our evaluation to DGEMM, as in prior work [10][11][12], for two reasons. First, the basic idea of the hybrid-grained load-balancing method applies to other variants of GEMM such as SGEMM, CGEMM and ZGEMM.…”

“…ATLAS [1] adopts the auto-tuning method to automatically generate kernels with different parameters in C and find the best-performing one by running them on the actual computing system. POET [12,16,17] and AUGEM [11] use a directive-based programming approach. POCA [14] is a compiler-based approach which generates and optimize kernels automatically and portably.…”