KernelFaRer

Abstract

Well-crafted libraries deliver much higher performance than code generated by sophisticated application programmers using advanced optimizing compilers. When a code pattern for which a well-tuned library implementation exists is found in the source code of an application, the highest performing solution is to replace the pattern with a call to the library. Idiom-recognition solutions in the past either required pattern matching machinery that was outside of the compilation framework or provided a very brittle solution that would fail even for minor variants in the pattern source code. This article introduces Kernel Find & Replacer ( KernelFaRer ), an idiom recognizer implemented entirely in the existing LLVM compiler framework. The versatility of KernelFaRer is demonstrated by matching and replacing two linear algebra idioms, general matrix-matrix multiplication (GEMM), and symmetric rank-2k update (SYR2K). Both GEMM and SYR2K are used extensively in scientific computation, and GEMM is also a central building block for deep learning and computer graphics algorithms. The idiom recognition in KernelFaRer is much more robust than alternative solutions, has a much lower compilation overhead, and is fully integrated in the broadly used LLVM compilation tools. KernelFaRer replaces existing GEMM and SYR2K idioms with computations performed by BLAS, Eigen, MKL (Intel’s x86), ESSL (IBM’s PowerPC), and BLIS (AMD). Gains in performance that reach 2000× over hand-crafted source code compiled at the highest optimization level demonstrate that replacing application code with library call is a performant solution.

Fast matrix multiplication via compiler‐only layered data reorganization and intrinsic lowering

“…Thus, the performance results for Polly cannot be included in the comparison at this time. As reported by Carvalho et al 16 , we also failed to reproduce the performance results reported by Gareev et al's recent work 19 . Enabling Polly's GEMM idiom recognition and optimization pass 10 the code runs 4.8× slower than Tiling+Packing.…”

“…Therefore, for f64 the value of nr should be reduced in half to reflect the number of VSRs available. With this reduction, an ATile tile occupies 16 VSRs and a BTile tile also occupies 16 VSRs. The extraction of operands into vector registers in lines 6 and 9 of Algorithm 2 must be changed accordingly.…”

“…A naive implementation of a × × SGEMM in C++ serves as the base code for our algorithm. Carvalho et al's pattern-matching algorithm 16 is used to automatically identify and replace these GEMM loops in the source code with the code generated by the macro-level algorithm. The input matrices are stored and accessed in column-major order and the values mr = 16, nr = 4, and kr = 64 are used for all platforms, except on POWER10 which used mr = 16, nr = 8, and kr = 128.…”

“…The acceleration of other idioms requires changes to our pattern recognition component, KernelFaRer 16 . For a detailed discussion, please refer to the KernelFaRer paper.…”

“…Carvalho et al introduce KernelFaRer, a robust pattern recognition system that can identify matrix-multiplication patterns in the LLVM IR level and can replace these with library calls 16 . While this approach can lead to speedups on the order of 1000s in comparison with non-optimized code, it has the drawback of requiring the integration of libraries into a computer system that may not have it.…”