Design of a High-Performance GEMM-like Tensor–Tensor Multiplication

Abstract

We present "GEMM-like Tensor-Tensor multiplication" (GETT), a novel approach for dense tensor contractions that mirrors the design of a high-performance general matrix-matrix multiplication (GEMM). The critical insight behind GETT is the identification of three index sets, involved in the tensor contraction, which enable us to systematically reduce an arbitrary tensor contraction to loops around a highly tuned "macro-kernel". This macro-kernel operates on suitably prepared ("packed") sub-tensors that reside in a specified level of the cache hierarchy. In contrast to previous approaches to tensor contractions, GETT exhibits desirable features such as unit-stride memory accesses, cache-awareness, as well as full vectorization, without requiring auxiliary memory. We integrate GETT alongside the so called TransposeTranspose-GEMM-Transpose and Loops-over-GEMM approaches into an open source "Tensor Contraction Code Generator" (TCCG). The performance results for a wide range of tensor contractions suggest that GETT has the potential of becoming the method of choice: While GETT exhibits excellent performance across the board, its effectiveness for bandwidth-bound tensor contractions is especially impressive, outperforming existing approaches by up to 12.4×. More precisely, GETT achieves speedups of up to 1.41× over an equivalent-sized GEMM for bandwidth-bound tensor contractions while attaining up to 91.3% of peak floating-point performance for compute-bound tensor contractions.

TSFC: A Structure-Preserving Form Compiler

“…Although rearranging compound expressions may drastically reduce the required computational work in some cases, the efficient implementation of primitive tensor operations is necessary for the effective use of hardware. Traditionally, BLAS (Basic Linear Algebra Subroutines) implementations provide optimized dense linear algebra operations; Springer and Bientinesi [51] present efficient strategies for generalising to tensor-tensor multiplication. However, these routines are optimized for large tensors and matrices.…”

Format abstraction for sparse tensor algebra compilers

“…An early effort for dense higher-order tensor algebra was the Tensor Contraction Engine [Auer et al 2006]. libtensor [Epifanovsky et al 2013], CTF [Solomonik et al 2014], and GETT [Springer and Bientinesi 2016] are examples of systems and techniques that transform tensor contractions into dense matrix multiplications by transposing tensor operands. TBLIS [Matthews 2017] and InTensLi [Li et al 2015] avoid explicit transpositions by computing tensor contractions in-place.…”

Strassen's Algorithm for Tensor Contraction

“…We will leverage this observation in our distributed memory experiment. Figure 6: Performance for representative user cases of benchmark from [6]. TC is identified by the index string, with the tensor index bundle of each tensor in the order C-A-B, e.g.…”

“…Standing on the shoulders of giants. This paper builds upon a number of recent developments: The GotoBLAS algorithm for matrix multiplication (GEMM) [1] that underlies the currently fastest implementations of GEMM for CPUs; The refactoring of the GotoBLAS algorithm as part of the BLAS-like Library Instantiation Software (BLIS) [2,3], which exposes primitives for implementing BLAS-like operations; The systematic parallelization of the loops that BLIS exposes so that high-performance can be flexibly attained on multicore and many-core architectures [4]; The casting of tensor contraction (TC) in terms of the BLIS primitives [5,6] without requiring the transposition (permutation) used by traditional implementations; The practical high-performance implementation of the classical Strassen's algorithm (Strassen) [7] in terms of variants of the BLIS primitives; and the extension of this implementation [8] to a family of Strassen-like algorithms (Fast Matrix Multiplication algorithms) [9]. Together, these results facilitate what we believe to be the first extension of Strassen's algorithm to TC.…”