Dynamical Tensor Approximation

Abstract

Abstract. For the approximation of time-dependent data tensors and of solutions to tensor differential equations by tensors of low Tucker rank, we study a computational approach that can be viewed as a continuous-time updating procedure. This approach works with the increments rather than the full tensor and avoids the computation of decompositions of large matrices. In this method, the derivative is projected onto the tangent space of the manifold of tensors of Tucker rank (r 1 , . . . , r N ) at the current approximation. This yields nonlinear differential equations for the factors in a Tucker decomposition, suitable for numerical integration. Approximation properties of this approach are analyzed.Key words. Low-rank approximation, time-varying tensors, continuous updating, Tucker decomposition, tensor differential equations.AMS subject classifications. 15A18,15A69,65F99,65L051. Introduction. Tensor approximation is an active research area with both interesting theoretical questions and numerous applications in the compression and retrieval of large structured data; see, e.g., [3,4,6,11,19] and references therein. While these papers deal with the approximation of a fixed tensor, we here consider the approximation of time-varying (or parameter-dependent) tensors by tensors of low Tucker-rank. There arises the question as how to update a given tensor approximation, a problem that has been addressed in a different, discrete-time setting in [16,17]. The approach considered in the present paper can be viewed as a continuous-time updating technique, which works only with the increments in the tensors rather than the tensors themselves and does not require to compute any decompositions of large matrices. It extends the dynamical low-rank approximation of matrices [8,15] to the higher-order tensor case.Consider a time-varying family of tensors A(t) ∈ R I1×...×I N , for 0 ≤ t ≤ t. Let M r denote the manifold of all order-N tensors of Tucker rank r = (r 1 , . . . , r N ) (see §2 below for the notion of Tucker rank or mode rank), where r n ≤ I n (and typically r n ≪ I n ) for n = 1, . . . , N . The best approximation to A(t) in M r (with respect to the Frobenius norm ∥ · ∥) is

Discretized Dynamical Low-Rank Approximation in the Presence of Small Singular Values

“…Differential equations for the factors of a low-rank factorization similar to the singular value decomposition were derived and their approximation properties were studied. Extensions to time-dependent tensors in various tensor formats were given in [2,12,18,19]; see also [15] for a review of dynamical low-rank approximation.The approach yields differential equations on low-rank matrix and tensor manifolds, which need to be solved numerically. Recently, very efficient integrators based on splitting the projection onto the tangent space of the low-rank manifold have been proposed and studied for matrices and for tensors in the tensor-train format in [16] and [17], respectively.…”

“…Differential equations for the factors of a low-rank factorization similar to the singular value decomposition were derived and their approximation properties were studied. Extensions to time-dependent tensors in various tensor formats were given in [2,12,18,19]; see also [15] for a review of dynamical low-rank approximation.…”

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives

“…There are two straightforward ways of solving (3.116). The original approach of Koch and Lubich [2007] (later generalized to the Tucker and TT models [Koch and Lubich, 2010]) is to write down ordinary differential equations for the parameters U(t), S(t), V(t) of the SVD like decomposition in the form…”

On Reduced Models for the Chemical Master Equation

“…[24,25]. In this approach, the solution of an evolution equation is represented by a sum of rank-1 tensor products which are propagated along with the solution.…”