Towards an efficient use of the BLAS library for multilinear tensor contractions

Napoli, Edoardo Di; Fabregat‐Traver, Diego; Quintana-Ortí, Gregorio; Bientinesi, Paolo

doi:10.1016/j.amc.2014.02.051

Cited by 33 publications

(33 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BLAS procedure) the number of efficient algorithms for tensor contractions is rather limited. In practice, due to the high computational complexity of tensor contractions, especially for tensor networks with loops, this operation is often performed approximately [66,107,138,167]. A variant of Tensor Trace [128] for the case of the partial tensor selfcontraction considers a tensor A P R RˆI 1ˆI2ˆ¨¨¨ˆINˆR and yields a reducedorder tensor r…”

mentioning

confidence: 99%

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

Cichocki

Lee

Oseledets

et al. 2016

FNT in Machine Learning

351

329

View full text Add to dashboard Cite

Modern applications in engineering and data science are increasingly based on multidimensional data of exceedingly high volume, variety, and structural richness. However, standard machine learning algorithms typically scale exponentially with data volume and complexity of cross-modal couplings - the so called curse of dimensionality - which is prohibitive to the analysis of large-scale, multi-modal and multi-relational datasets. Given that such data are often efficiently represented as multiway arrays or tensors, it is therefore timely and valuable for the multidisciplinary machine learning and data analytic communities to review low-rank tensor decompositions and tensor networks as emerging tools for dimensionality reduction and large scale optimization problems. Our particular emphasis is on elucidating that, by virtue of the underlying low-rank approximations, tensor networks have the ability to alleviate the curse of dimensionality in a number of applied areas. In Part 1 of this monograph we provide innovative solutions to low-rank tensor network decompositions and easy to interpret graphical representations of the mathematical operations on tensor networks. Such a conceptual insight allows for seamless migration of ideas from the flat-view matrices to tensor network operations and vice versa, and provides a platform for further developments, practical applications, and non-Euclidean extensions. It also permits the introduction of various tensor network operations without an explicit notion of mathematical expressions, which may be beneficial for many research communities that do not directly rely on multilinear algebra. Our focus is on the Tucker and tensor train (TT) decompositions and their extensions, and on demonstrating the ability of tensor networks to provide linearly or even super-linearly (e.g., logarithmically) scalable solutions, as illustrated in detail in Part 2 of this monograph

show abstract

mentioning

confidence: 99%

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

Cichocki

Lee

Oseledets

et al. 2016

FNT in Machine Learning

351

329

View full text Add to dashboard Cite

show abstract

“…1: C abc = A ai B ibc : 9 exemplary algorithms out of 36. 6 As already mentioned, given a contraction, there is no obvious a-priori choice of kernel and slicings to attain the highest performance. We therefore generate all possible combinations.…”

Section: Algorithm Generationmentioning

confidence: 99%

“…The most prominent project targeting the efficient computation of tensor contractions is probably the Tensor Contraction Engine, a compiler built specifically for multi-tensor multi-index contractions to be executed within memory constraints [4]; in light of the wide diffusion and nearly optimal efficiency of the BLAS library, an extension to TCE was proposed to compute contractions via BLAS operations [5]. In the same spirit, we provided simple rules to build a taxonomy for all contractions between two tensors, identifying which BLAS routines are usable and how to best exploit them [6].…”

Section: Introductionmentioning

confidence: 99%

On the Performance Prediction of BLAS-based Tensor Contractions

Peise

Fabregat‐Traver

Bientinesi

2015

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Abstract. Tensor operations are surging as the computational building blocks for a variety of scientific simulations and the development of highperformance kernels for such operations is known to be a challenging task. While for operations on one-and two-dimensional tensors there exist standardized interfaces and highly-optimized libraries (BLAS), for higher dimensional tensors neither standards nor highly-tuned implementations exist yet. In this paper, we consider contractions between two tensors of arbitrary dimensionality and take on the challenge of generating highperformance implementations by resorting to sequences of BLAS kernels. The approach consists in breaking the contraction down into operations that only involve matrices or vectors. Since in general there are many alternative ways of decomposing a contraction, we are able to methodically derive a large family of algorithms. The main contribution of this paper is a systematic methodology to accurately identify the fastest algorithms in the bunch, without executing them. The goal is instead accomplished with the help of a set of cache-aware micro-benchmarks for the underlying BLAS kernels. The predictions we construct from such benchmarks allow us to reliably single out the best-performing algorithms in a tiny fraction of the time taken by the direct execution of the algorithms.

show abstract

“…This explains extra parameters in gemm (matrix multiplication) and gemv (matrix-vector) multiplication that allow block-strided access (which is equivalent to allowing matrices within larger matrices). Recently it was observed that the gemm interface is not sufficient when multiplying matrices in higher-dimensional tensors [13].…”

Section: Related Workmentioning

confidence: 99%

Automatic locality-friendly interface extension of numerical functions

Hess

Groß

Püschel

2014

Proceedings of the 2014 International Conference on Generative Programming: Concepts and Experiences

View full text Add to dashboard Cite

Raising the level of abstraction is a key concern of software engineering, and libraries (either used directly or as a target of a program generation system) are a successful technique to raise programmer productivity and to improve software quality. Unfortunately successful libraries may contain functions that may not be general enough. For example, many numeric performance libraries contain functions that work on one-or higher-dimensional arrays. A problem arises if a program wants to invoke such a function on a non-contiguous subarray (e.g., in C the column of a matrix or a subarray of an image). If the library developer did not foresee this scenario, the client program must include explicit copy steps before and after the library function call, incurring a possibly high performance penalty. A better solution would be an enhanced library function that allows for the desired access pattern. Exposing the access pattern allows the compiler to optimize for the intended usage scenario(s). As we do not want the library developer to generate all interesting versions manually, we present a tool that takes a library function written in C and generates such a customized function for typical accesses. We describe the approach, discuss limitations, and report on the performance. As example access patterns we consider those most common in numerical applications: striding and block striding, general permutations, as well as scaling. We evaluate the tool on various library functions including filters, scans, reductions, sorting, FFTs, and linear algebra operations. The automatically generated custom version is in most cases significantly faster than using individual steps, offering speed-ups that are typically in the range of 1.2-1.8x.

show abstract

Towards an efficient use of the BLAS library for multilinear tensor contractions

Cited by 33 publications

References 19 publications

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

On the Performance Prediction of BLAS-based Tensor Contractions

Automatic locality-friendly interface extension of numerical functions

Contact Info

Product

Resources

About