Fast Multidimensional Convolution in Low-Rank Tensor Formats via Cross Approximation

Abstract

We propose new cross-conv algorithm for approximate computation of convolution in different low-rank tensor formats (tensor train, Tucker, Hierarchical Tucker). It has better complexity with respect to the tensor rank than previous approaches. The new algorithm has a high potential impact in different applications. The key idea is based on applying cross approximation in the "frequency domain", where convolution becomes a simple elementwise product. We illustrate efficiency of our algorithm by computing the three-dimensional Newton potential and by presenting preliminary results for solution of the Hartree-Fock equation on tensor-product grids. CANDECOMP/PARAFAC model) which dates back to 1927 [28]. A tensor is said to be in canonical format if it can be represented in the formwhere the minimal possible r is called canonical rank. If a good CP approximation is known, many basic operations are fast to compute [5,35,25,26,39,50,6].Nevertheless, the CP decomposition suffers from a serious drawback: there are no robust algorithms to compute it numerically for d > 2 [12]. Note that in two dimensions it can be computed in a stable way by using SVD or, if the matrix is large, by rank-revealing algorithms.The Tucker format [61,10,11,39] is another classic decomposition of tensors. It can be computed via stable algorithms but the number of parameters grows exponentially in d. As a result, it is typically used only for problems with small d, especially for the three-dimensional case. In higher dimensions other stable tensor formats, namely tensor train (TT) [51,47] or hierarchical Tucker (HT) [27,19] formats can be used. In contrast with the Tucker format, they do not suffer from the "curse of dimensionality". For more details regarding low-rank representations of tensors see the book by Hackbusch [24] and reviews [38,21,43].Related work. In this paper we focus on fast computation of multidimensional convolution. Although it is not difficult to implement convolution in complexity linear in d or n, a strong rank dependence may occur. The rank of the result is generally equal to the product of the ranks of f , g, and then one should truncate the representation with necessary accuracy (by truncation we mean approximation in the same format with smaller rank). This approach was considered in [58,36] and may lead to high complexity when the ranks are large. A remarkable work is [29] where an algorithm for the computation of convolution in so-called Quantized TT (QTT) [37,45] was proposed. This algorithm has complexity O(d log α n) and is asymptotically the best one. However, for n of practical interest the algorithm proposed in this paper is faster for the same discretization and approximation accuracy ε. This is due to high constant hidden in O(·) term in the QTT algorithm.The algorithm proposed in this paper is simple. At first, we use a classic idea of representing discrete convolution in the form of several Fourier transforms and one element-wise multiplication in the "frequency domain". The crucial step is to interpolate t...

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions

“…For the general case, it has been proven that when the intersection submatrix W is of maximum volume 7 , the matrix cross-approximation is close to the optimal SVD solution. The problem of finding a submatrix with maximum volume has exponential complexity, however, suboptimal matrices can be found using fast greedy algorithms [4,144,179,222].…”

Calculating vibrational spectra of molecules using tensor train decomposition

“…For the canonical decomposition power method with shifts was generalized in [33,34] and used in the RRBPM method. The preconditioned inverse iteration (PINVIT) for tensor formats was considered in [22,23,35]. The inverse iteration used in this paper differs from the PIN-VIT, which is basically preconditioned steepest descent.…”

Memory Footprint Reduction for the FFT-Based Volume Integral Equation Method via Tensor Decompositions

“…Since the algorithm is based on SVD, it is stable and the low multilinear rank approximation always exists. Tucker decomposition can be constructed by using only some rows, columns and fibers of A with a cross-Tucker approximation algorithm [39], [40] with linear complexity O (nr 1 r 2 r 3 ) over the O n 3 complexity of HOSVD (n denotes tensor's linear size). Such algorithms are based on wellknown 2D cross approximation methods [54]- [57].…”

“…Specifically, Tucker decomposition provides an optimal fit and a stable approximation for three-dimensional (3D) tensors [33]. It can be implemented either with the wellknown higher-order singular value decomposition (HOSVD) [34], which has been successfully applied in multidimensional data analysis in the past years [35]- [38], or with cross approximation-based techniques [39], [40]. Another interesting decomposition is the canonical polyadic (CP) model [41]- [44], which gives the most compact representation of the initial arXiv:1811.00484v3 [cs.NA] 16 Jul 2019 array.…”