Provable Tensor-Train Format Tensor Completion by Riemannian Optimization

Abstract

The tensor train (TT) format enjoys appealing advantages in handling structural high-order tensors. The recent decade has witnessed the wide applications of TT-format tensors from diverse disciplines, among which tensor completion has drawn considerable attention. Numerous fast algorithms, including the Riemannian gradient descent (RGrad) algorithm, have been proposed for the TT-format tensor completion. However, the theoretical guarantees of these algorithms are largely missing or sub-optimal, partly due to the complicated and recursive algebraic operations in TT-format decomposition. Moreover, existing results established for the tensors of other formats, for example, Tucker and CP, are inapplicable because the algorithms treating TT-format tensors are substantially different and more involved. In this paper, we provide, to our best knowledge, the first theoretical guarantees of the convergence of RGrad algorithm for TT-format tensor completion, under a nearly optimal sample size condition. The RGrad algorithm converges linearly with a constant contraction rate that is free of tensor condition number without the necessity of re-conditioning. We also propose a novel approach, referred to as the sequential second-order moment method, to attain a warm initialization under a similar sample size requirement. As a byproduct, our result even significantly refines the prior investigation of RGrad algorithm for matrix completion. Numerical experiments confirm our theoretical discovery and showcase the computational speedup gained by the TT-format decomposition.

Computationally Efficient and Statistically Optimal Robust Low-rank Matrix and Tensor Estimation

“…Recently, it is discovered that optimizing (2) directly on the low-rank manifold M r , called Riemannian, enjoys the fast computational speed of factorization-based approaches and, meanwhile, converges linearly regardless of the condition number of M * . See, e.g., Cai et al (2021a) and Cai et al (2021b), for the convergence of Riemannian gradient descent (RGrad) algorithms in minimizing strongly convex and smooth functions with tensor-related applications. We note that RGrad is similar to the projected gradient descent (PGD, Chen and Wainwright (2015)) except that RGrad utilizes the Riemannian gradient while PGD takes the vanilla one.…”

Tensor train completion: local recovery guarantees via Riemannian optimization

“…A recent paper [34] addresses Riemannian TT completion from the theoretical point of view. There, the authors use a fixed step size and apply an additional trimming procedure before TT-SVD: it ensures that all the elements of the tensor before the retraction do not exceed a certain threshold and that the projected tensor is incoherent.…”

“…We leave aside the question of generating an initial estimate that lies close enough to the true solution and focus instead on reducing the required number of samples. On the contrary, in [34] the main concern is in enlarging the basin of attraction and providing a constructive initialization procedure.…”

“…A different version of RGD for TT completion was proposed in [34] with an extra trimming step placed before the TT-SVD projection. The authors showed that such algorithm produces a locally convergent sequence and, in addition, provided a constructive initialization scheme.…”

“…In [34], the interface coherence was used as well and allowed the authors to prove a result similar to Theorem 5.4. However, it was not sufficient to prove local convergence of their version of RGD.…”