Riemannian Optimization for High-Dimensional Tensor Completion

Abstract

Tensor completion aims to reconstruct a high-dimensional data set where the vast majority of entries is missing. The assumption of low-rank structure in the underlying original data allows us to cast the completion problem into an optimization problem restricted to the manifold of fixed-rank tensors. Elements of this smooth embedded submanifold can be efficiently represented in the tensor train (TT) or matrix product states (MPS) format with storage complexity scaling linearly with the number of dimensions. We present a nonlinear conjugate gradient scheme within the framework of Riemannian optimization which exploits this favorable scaling. Numerical experiments and comparison to existing methods show the effectiveness of our approach for the approximation of multivariate functions. Finally, we show that our algorithm can obtain competitive reconstructions from uniform random sampling of few entries compared to adaptive sampling techniques such as cross-approximation.

Stable ALS approximation in the TT-format for rank-adaptive tensor completion

“…Also the approaches which themselves retreat to alternating least squares [20] treat the iterations as necessity for the minimization of an objective function with regularization term. The penalty term is, as usual, based on the singular values of the output of a microstep (a posteriori), as it is also the case for the work [40] on tensor completion through Riemannian optimization. Although their term may appear similar to the term we derive (cf.…”

“…• SALSA (Algorithm 4, the algorithm proposed in this work) • RTTC (Riemannian cg for tensor train completion [40]) We explain how ranks are adapted for ALS in Section 9.1, shortly present the idea behind RTTC in Section 9.2, give details for data acquisition and measurements in Section 9.3 as well as tuning parameters in Section 9.4. We analyze the results in the latter Section 9.9.…”

“…The article [40], in which RTTC is derived and explained in detail, focuses exclusively on tensor completion using the tensor train format as well (but it can likewise be assumed that it is generalizable to other problems). Instead of alternating optimization, RTTC provides a nonlinear conjugate gradient scheme based on Riemannian optimization, which has comparable computational complexity per sweep.…”

“…Naturally, the problem of rank adaption also poses a challenge in that setting. Therefore, a heuristic rank adaption is introduced (Algorithm 3 in [40]) which successively tests if a single rank increase yields a tolerable change of the residual on the validation set (based on a parameter ρ ≥ 0). If so, it continues normally with the next test; otherwise, the algorithm priorly resets to the iterate with previous rank.…”

Rank adaptive tensor recovery based model reduction for partial differential equations with high-dimensional random inputs

“…, A d } is the solution of (21) with rank R ≥ 1, we consider one higher rank, i.e., R + 1. While an analogous approach for tensor completion using tensor train decomposition can be found in [29], we here focus on CP decomposition and give the following scheme of rank-one update. The initial factor matrices for rank R + 1 is set to the rank-one updates of the factor matrices of X (R) , i.e.,…”