Parallel Algorithms for Tensor Train Arithmetic

Daas, Hussam Al; Ballard, Grey; Benner, Peter

doi:10.1137/20m1387158

Cited by 19 publications

(9 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the tensor literature, there are several parallel and distributed systems for processing large-scale tensors. We can list here some efficient tools for: (a) distributed CP decomposition (e.g., DFacTo [132], SPLATT [133]), (b) distributed Tucker decomposition (e.g., DHOSVD [85], SGD-Tucker [134]), and (c) distributed TT decomposition (e.g., ADTT [110], ATTAC [135]), etc. These tools mainly distribute the unfolding matrices or sub-tensors among several clusters and integrate their low-rank tensor approximations to find the overall low-rank approximation of the underlying tensor.…”

Section: Efficient and Scalable Tensor Trackingmentioning

confidence: 99%

A Contemporary and Comprehensive Survey on Streaming Tensor Decomposition

Thanh

Abed-Meraim

Trung

et al. 2023

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

Tensor decomposition has been demonstrated to be successful in a wide range of applications, from neuroscience and wireless communications to social networks. In an online setting, factorizing tensors derived from multidimensional data streams is however non-trivial due to several inherent problems of real-time stream processing. In recent years, many research efforts have been dedicated to developing online techniques for decomposing such tensors, resulting in significant advances in streaming tensor decomposition or tensor tracking. This topic is emerging and enriches the literature on tensor decomposition, particularly from the data stream analystics perspective. Thus, it is imperative to carry out an overview of tensor tracking to help researchers and practitioners understand its development and achievements, summarise the current trends and advances, and identify challenging problems. In this article, we provide a contemporary and comprehensive survey on different types of tensor tracking techniques. We particularly categorize the state-of-the-art methods into three main groups: streaming CP decompositions, streaming Tucker decompositions, and streaming decompositions under other tensor formats (i.e., tensor-train, t-SVD, and BTD). In each group, we further divide the existing algorithms into sub-categories based on their main optimization framework and model architectures. Finally, we present several research challenges, open problems, and potential directions of tensor tracking in the future.

show abstract

Section: Efficient and Scalable Tensor Trackingmentioning

confidence: 99%

A Contemporary and Comprehensive Survey on Streaming Tensor Decomposition

Thanh

Abed-Meraim

Trung

et al. 2023

IEEE Trans. Knowl. Data Eng.

View full text Add to dashboard Cite

show abstract

“…When we consider u TT (Ax) = u TT (y) in coordinates y, equation (12) is the FTT expansion for u TT (y). However, when we consider v(x) = u TT (Ax) in coordinates x we have that each mode ψ i (α i−1 ; a i • x; α i ) in the tensor ridge function (12) is no longer a univariate function of x i as in (8), but rather a d-variate ridge function, which, has the property of being constant in all directions orthogonal to the vector a i (e.g., [7,29]). An important problem is determining the FTT expansion…”

Section: Tensor Ridge Functionsmentioning

confidence: 99%

“…The final estimate for computing the matrix D i is O(d 2 n 3 r 3 ), which dominates the cost of performing one-step of (49). We point out that the computation of D i may be incorporated into high performance computing algorithms for tensor train rounding, e.g., [8,36].…”

Section: Computational Costmentioning

confidence: 99%

Tensor rank reduction via coordinate flows

Dektor¹,

Venturi²

2022

Preprint

View full text Add to dashboard Cite

Recently, there has been a growing interest in efficient numerical algorithms based on tensor networks and low-rank techniques to approximate high-dimensional functions and the numerical solution to highdimensional PDEs. In this paper, we propose a new tensor rank reduction method that leverages coordinate flows and can greatly increase the efficiency of high-dimensional tensor approximation algorithms. The idea is very simple: given a multivariate function, determine a coordinate transformation so that the function in the new coordinate system has smaller tensor rank. We restrict our analysis to linear coordinate transformations, which give rise to a new class of functions that we refer to as tensor ridge functions. By leveraging coordinate flows and tensor ridge functions, we develop an optimization method based on Riemannian gradient descent for determining a quasi-optimal linear coordinate transformation for tensor rank reduction. The theoretical results we present for rank reduction via linear coordinate transformations can be generalized to larger classes of nonlinear transformations. We demonstrate the effectiveness of the proposed new tensor rank reduction method on prototype function approximation problems, and in computing the numerical solution of the Liouville equation in dimensions three and five.

show abstract

“…In the tensor literature, there are several parallel and distributed systems for processing large-scale tensors. We can list here some efficient tools for: (a) distributed CP decomposition (e.g., DFacTo [176], SPLATT [177]), (b) distributed Tucker decomposition (e.g., DHOSVD [88], SGD-Tucker [178]), and (c) distributed TT decomposition (e.g., ADTT [114], ATTAC [179]), etc. These tools mainly distribute the unfolding matrices or sub-tensors among several clusters and integrate their low-rank tensor approximations to find the overall low-rank approximation of the underlying tensor.…”

Section: Efficient and Scalable Tensor Trackingmentioning

confidence: 99%

A Contemporary and Comprehensive Survey on Streaming Tensor Decomposition

Thanh¹,

abed-meraim²,

Linh-Trung³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p>Tensor decomposition has been demonstrated to be successful in a wide range of applications, from neuroscience and wireless communications to social networks. In an online setting, factorizing tensors derived from multidimensional data streams is however non-trivial due to several inherent problems of real-time stream processing. In recent years, many research efforts have been dedicated to developing online techniques for decomposing such tensors, resulting in significant advances in streaming tensor decomposition or tensor tracking. This topic is emerging and enriches the literature on tensor decomposition, particularly from the data stream analystics perspective. Thus, it is imperative to carry out an overview of tensor tracking to help researchers and practitioners understand its development and achievements, summarise the current trends and advances, and identify challenging problems. In this article, we provide a contemporary and comprehensive survey on different types of tensor tracking techniques. We particularly categorize the state-of-the-art methods into three main groups: streaming CP decompositions, streaming Tucker decompositions, and streaming decompositions under other tensor formats (i.e., tensor-train, t-SVD, and BTD). In each group, we further divide the existing algorithms into sub-categories based on their main optimization framework and model architectures. Finally, we present several research challenges, open problems, and potential directions of tensor tracking in the future.</p>

show abstract

Parallel Algorithms for Tensor Train Arithmetic

Cited by 19 publications

References 32 publications

A Contemporary and Comprehensive Survey on Streaming Tensor Decomposition

A Contemporary and Comprehensive Survey on Streaming Tensor Decomposition

Tensor rank reduction via coordinate flows

A Contemporary and Comprehensive Survey on Streaming Tensor Decomposition

Contact Info

Product

Resources

About