Parallel algorithms for computing the tensor-train decomposition

Shi, Tianyi; Ruth, Maximilian; Townsend, Alex

doi:10.48550/arxiv.2111.10448

Cited by 3 publications

(11 citation statements)

References 44 publications

(64 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Theorem 7 below. We again remark that this independent set of equations is similar to the ones presented in a recent work [17]. However the motivation there is to determine the cores in a parallel fashion and not for improving statistical estimation.…”

Section: Main Idea Of the Algorithmmentioning

confidence: 60%

“…Throughout this section, we assume that the input p of TT-RS (Algorithm 1) is a Markov model, that is, p is a probability density function and satisfies (17) p(x 1 , . .…”

Section: Application Of Tt-rs To Markov Modelmentioning

confidence: 99%

“…A recent work [17] determines the TT from values of a high-dimensional function in a computationally distributed fashion. In particular, [17] forms an independent set of equations with sketching techniques to determine the tensor cores in a parallel way where the data format falls under the first discussed modality. Our method has similarities with [17] in that our starting point is also a similar system of equations for determining each core independently.…”

Section: Introductionmentioning

confidence: 99%

“…In particular, [17] forms an independent set of equations with sketching techniques to determine the tensor cores in a parallel way where the data format falls under the first discussed modality. Our method has similarities with [17] in that our starting point is also a similar system of equations for determining each core independently. However, since our goal, which is to determine a TT based on empirical samples of a density, is different from [17], the purpose and means of sketching are fundamentally different.…”

Section: Introductionmentioning

confidence: 99%

“…Our method has similarities with [17] in that our starting point is also a similar system of equations for determining each core independently. However, since our goal, which is to determine a TT based on empirical samples of a density, is different from [17], the purpose and means of sketching are fundamentally different. We apply sketching such that each equation in the independent system of equations has size that is constant with respect to the dimension of the problem, and hence we can estimate the coefficient matrices of the linear system in a statistically-efficient way without error accumulation.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Generative modeling via tensor train sketching

Hur¹,

Hoskins²,

Lindsey³

et al. 2022

Preprint

View full text Add to dashboard Cite

In this paper we introduce a sketching algorithm for constructing a tensor train representation of a probability density from its samples. Our method deviates from the standard recursive SVD-based procedure for constructing a tensor train. Instead we formulate and solve a sequence of small linear systems for the individual tensor train cores. This approach can avoid the curse of dimensionality that threatens both the algorithmic and sample complexities of the recovery problem. Specifically, for Markov models, we prove that the tensor cores can be recovered with a sample complexity that is constant with respect to the dimension. Finally, we illustrate the performance of the method with several numerical experiments.

show abstract

Section: Main Idea Of the Algorithmmentioning

confidence: 60%

“…Throughout this section, we assume that the input p of TT-RS (Algorithm 1) is a Markov model, that is, p is a probability density function and satisfies (17) p(x 1 , . .…”

Section: Application Of Tt-rs To Markov Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Generative modeling via tensor train sketching

Hur¹,

Hoskins²,

Lindsey³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Tensor rank reduction via coordinate flows

Dektor¹,

Venturi²

2022

Preprint

View full text Add to dashboard Cite

Recently, there has been a growing interest in efficient numerical algorithms based on tensor networks and low-rank techniques to approximate high-dimensional functions and the numerical solution to highdimensional PDEs. In this paper, we propose a new tensor rank reduction method that leverages coordinate flows and can greatly increase the efficiency of high-dimensional tensor approximation algorithms. The idea is very simple: given a multivariate function, determine a coordinate transformation so that the function in the new coordinate system has smaller tensor rank. We restrict our analysis to linear coordinate transformations, which give rise to a new class of functions that we refer to as tensor ridge functions. By leveraging coordinate flows and tensor ridge functions, we develop an optimization method based on Riemannian gradient descent for determining a quasi-optimal linear coordinate transformation for tensor rank reduction. The theoretical results we present for rank reduction via linear coordinate transformations can be generalized to larger classes of nonlinear transformations. We demonstrate the effectiveness of the proposed new tensor rank reduction method on prototype function approximation problems, and in computing the numerical solution of the Liouville equation in dimensions three and five.

show abstract

Streaming Tensor Train Approximation

Kreßner¹,

Vandereycken²,

Voorhaar³

2022

Preprint

View full text Add to dashboard Cite

Tensor trains are a versatile tool to compress and work with high-dimensional data and functions. In this work we introduce the Streaming Tensor Train Approximation (STTA), a new class of algorithms for approximating a given tensor T in the tensor train format. STTA accesses T exclusively via two-sided random sketches of the original data, making it streamable and easy to implement in parallel -unlike existing deterministic and randomized tensor train approximations. This property also allows STTA to conveniently leverage structure in T , such as sparsity and various low-rank tensor formats, as well as linear combinations thereof. When Gaussian random matrices are used for sketching, STTA is admissible to an analysis that builds and extends upon existing results on the generalized Nyström approximation for matrices. Our results show that STTA can be expected to attain a nearly optimal approximation error if the sizes of the sketches are suitably chosen. A range of numerical experiments illustrates the performance of STTA compared to existing deterministic and randomized approaches.

show abstract

Parallel algorithms for computing the tensor-train decomposition

Cited by 3 publications

References 44 publications

Generative modeling via tensor train sketching

Generative modeling via tensor train sketching

Tensor rank reduction via coordinate flows

Streaming Tensor Train Approximation

Contact Info

Product

Resources

About