Optimization on the Hierarchical Tucker manifold - applications to tensor completion

Abstract

In this work, we develop an optimization framework for problems whose solutions are wellapproximated by Hierarchical Tucker (HT) tensors, an efficient structured tensor format based on recursive subspace factorizations. By exploiting the smooth manifold structure of these tensors, we construct standard optimization algorithms such as Steepest Descent and Conjugate Gradient for completing tensors from missing entries. Our algorithmic framework is fast and scalable to large problem sizes as we do not require SVDs on the ambient tensor space, as required by other methods. Moreover, we exploit the structure of the Gramian matrices associated with the HT format to regularize our problem, reducing overfitting for high subsampling ratios. We also find that the organization of the tensor can have a major impact on completion from realistic seismic acquisition geometries. These samplings are far from idealized randomized samplings that are usually considered in the literature but are realizable in practical scenarios. Using these algorithms, we successfully interpolate large-scale seismic data sets and demonstrate the competitive computational scaling of our algorithms as the problem sizes grow.

TTML: tensor trains for general supervised machine learning

“…In order to adapt our methods to such a different tensor format, we need a good black box approximation method, and an efficient implementation of retraction and vector transport to perform RCGD (or an alternative method to efficiently solve the tensor completion problem in the given format). For the HT format, black box approximation is described in [Ballani et al, 2013], whereas (Riemannian) tensor completion for HT is described in [Da Silva andHerrmann, 2014, Rauhut et al, 2015]. Adapting our methods to the HT format should therefore be straightforward.…”

Variants of Alternating Least Squares Tensor Completion in the Tensor Train Format

“…Remark 36 (Comparison with HTOpt) As a comparison of our results with the HTOpt algorithm from [21,22] we perform the first three test of Table 1, maximal number of iterations to 1000. The following table shows the approximation quality on the control set C and given point set P , the accuracy of the near best exponential sum approximation (res exp ) from Remark 35, and the number of iterative steps:…”