Dynamical Approximation by Hierarchical Tucker and Tensor-Train Tensors

Abstract

We extend results on the dynamical low-rank approximation for the treatment of time-dependent matrices and tensors (Koch and Lubich; see [SIAM to the recently proposed hierarchical Tucker (HT) tensor format (Hackbusch and Kühn; see [J. Fourier Anal. Appl., 15 (2009), pp. 706-722]) and the tensor train (TT) format (Oseledets; see [SIAM J. Sci. Comput., 33 (2011), pp. 2295-2317), which are closely related to tensor decomposition methods used in quantum physics and chemistry. In this dynamical approximation approach, the time derivative of the tensor to be approximated is projected onto the time-dependent tangent space of the approximation manifold along the solution trajectory. This approach can be used to approximate the solutions to tensor differential equations in the HT or TT format and to compute updates in optimization algorithms within these reduced tensor formats. By deriving and analyzing the tangent space projector for the manifold of HT/TT tensors of fixed rank, we obtain curvature estimates, which allow us to obtain quasi-best approximation properties for the dynamical approximation, showing that the prospects and limitations of the ansatz are similar to those of the dynamical low rank approximation for matrices. Our results are exemplified by numerical experiments. DYNAMICAL APPROXIMATION BY HT AND TT TENSORS 471embedded manifold M ⊆ V typically depending on much fewer parameters than the linear parametrization of V, the dynamical tensor approximation may be utilized: Assuming Y (0) ∈ M, an approximation Y (t) ∈ M is determined such that its derivative at every time t is the element of the tangent space T Y (t) M closest toȦ(t):In terms of the orthogonal projection P X onto the tangent space T X M at X ∈ M, the solution to (1.2) is equivalently characterized by projectingwhich results in a differential equation on the approximation manifold M.After an explicit parametrization of the manifold M under consideration, one obtains from (1.3) a set of nonlinear differential equations for the parameters of this parametrization, suitable for numerical integration. The above ansatz has been studied for manifolds of matrices of fixed rank k and tensors of fixed (multilinear) Tucker rank (k 1 , . . . , k d ) in [15,24,16], respectively. In the case of a tensor differential equation,Ȧ(t) = F (A(t)) is replaced by the approximate value F (Y (t)) on the right-hand side of (1.2). In the context of the time-dependent Schrödinger equation, the approach is known as the Dirac-Frenkel time-dependent variational principle [20].Classical tensor formats stemming from data analysis (that is, the canonical decomposition and the Tucker format [8,17]) exhibit certain structural weaknesses that make them unsuitable for the treatment of problems of the kind exemplified by (1.1) and (1.2) when the dimension d is large; see, e.g., the introduction in [11]. This motivated the development of recent tensor formats such as the HT format, in which a recursive, hierarchical construction of Tucker type is employed for tensor representati...

Optimization on the Hierarchical Tucker manifold – Applications to tensor completion

“…Even though the authors in [4] develop an HT truncation method that does not need access to every entry of the tensor in order to form the approximation, their approach requires algorithm-driven access to the entries, which does not apply for the seismic examples we consider below. An HT approach for solving dynamical systems is outlined in [37], which considers similar manifold structure as in this article applied in a different context. The authors in [45] also consider the smooth manifold properties of HT tensors to construct tensor completion algorithms using a Hierarchical SVD-based approach.…”

Discretized Dynamical Low-Rank Approximation in the Presence of Small Singular Values

“…Differential equations for the factors of a low-rank factorization similar to the singular value decomposition were derived and their approximation properties were studied. Extensions to time-dependent tensors in various tensor formats were given in [2,12,18,19]; see also [15] for a review of dynamical low-rank approximation.The approach yields differential equations on low-rank matrix and tensor manifolds, which need to be solved numerically. Recently, very efficient integrators based on splitting the projection onto the tangent space of the low-rank manifold have been proposed and studied for matrices and for tensors in the tensor-train format in [16] and [17], respectively.…”

“…Differential equations for the factors of a low-rank factorization similar to the singular value decomposition were derived and their approximation properties were studied. Extensions to time-dependent tensors in various tensor formats were given in [2,12,18,19]; see also [15] for a review of dynamical low-rank approximation.…”

Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 2 Applications and Future Perspectives

“…Similar results are also available for the more general hierarchical Tucker models. For example, [Uschmajew and Vandereycken, 2013] developed the manifold structure for the HT tensors, while Lubich et al [2013] developed the concept of dynamical low-rank approximation for both HT and TT formats. Moreover, Riemannian optimization in the Tucker and TT/HT formats has been successfully applied to large-scale tensor completion problems [Da Silva and Herrmann, 2013, Kasai and Mishra, 2015.…”