High-Performance Generalized Tensor Operations

Abstract

The efficiency of tensor contraction is of great importance. Compilers cannot optimize it well enough to come close to the performance of expert-tuned implementations. All existing approaches that provide competitive performance require optimized external code. We introduce a compiler optimization that reaches the performance of optimized BLAS libraries without the need for an external implementation or automatic tuning. Our approach provides competitive performance across hardware architectures and can be generalized to deliver the same benefits for algebraic path problems. By making fast linear algebra kernels available to everyone, we expect productivity increases when optimized libraries are not available. CCS Concepts: • Software and its engineering → Compilers; • Computing methodologies → Linear algebra algorithms;

Declarative Loop Tactics for Domain-specific Optimization

“…Generalized matrix multiplication (GEMM BLAS kernel) is one of the important computation patterns and is the most commonly optimized kernel in history [31]. However, state-of-the-art compilers achieve only a fraction of the theoretical machine performance for a simple textbook-style implementation [15]. A recent improvement within Polly introduced a custom transformation for GEMM-like kernels that is controlled outside of the main affine scheduling mechanism [15].…”

“…However, state-of-the-art compilers achieve only a fraction of the theoretical machine performance for a simple textbook-style implementation [15]. A recent improvement within Polly introduced a custom transformation for GEMM-like kernels that is controlled outside of the main affine scheduling mechanism [15]. This transformation applies to a generalized case of tensor contraction of the form…”

“…To evaluate the quality of our implementation, we hash the binaries generated by our custom Polly, with Loop Tactics embedded, with an original version containing the custom transformation (git: commit 592b2406) and compare them for strict equality. We consider a tensor contraction with E = (×, +) of the form C → α × C + β × A × B, where A, B, and C are square matrices and α and β are constants set to 1 as in Reference [15]. We use the compilation strings Clang -O3 -march=native -mllvm -polly -ffp-contract=fast -ffast-math and Clang -O3 -march=native -polly -mllvm -polly-enable-matchers-optimization -ffp-contract=fast -ffast-math for the original Polly and the modified one, respectively.…”

“…• We demonstrate how to use Loop Tactics to perform domain-specific optimizations [22,31], notably reducing code complexity and footprint compared to manual pattern matching [15] (Sections 5.1 and 5.2).…”

User-Directed Loop-Transformations in Clang

“…Only the copy-in and copy-out to the local memory allocation modify the schedule tree; these are new statements that are inserted before, respectively after the code that uses them. Parts of the code already existed in Polly as part of its matrix-matrix multiplication optimization [30], but had to be generalized to arbitrary loops. To define an index function and size of the packed array, we use the bounding box technique from [31].…”