Variants of Alternating Least Squares Tensor Completion in the Tensor Train Format

Grasedyck, Lars; Kluge, Melanie; Krämer, Sebastian

doi:10.1137/130942401

Cited by 63 publications

(65 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Schatten norm, or trace norm), which is defined as the sum of singular values of a matrix and it is the most popular convex surrogate for rank regularization. Based on differ-ent definitions of tensor rank, various nuclear norm regularized algorithms have been proposed (Liu et al 2013;Imaizumi, Maehara, and Hayashi 2017;Liu et al 2014;Liu et al 2015). Rank minimization based methods do not need to specify the rank of the employed tensor decompositions beforehand, and the rank of the recovered tensor will be automatically learned from the limited observations.…”

Section: Introductionmentioning

confidence: 99%

Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion

Yuan

Mandic

et al. 2019

AAAI

120

View full text Add to dashboard Cite

In tensor completion tasks, the traditional low-rank tensor decomposition models suffer from the laborious model selection problem due to their high model sensitivity. In particular, for tensor ring (TR) decomposition, the number of model possibilities grows exponentially with the tensor order, which makes it rather challenging to find the optimal TR decomposition. In this paper, by exploiting the low-rank structure of the TR latent space, we propose a novel tensor completion method which is robust to model selection. In contrast to imposing the low-rank constraint on the data space, we introduce nuclear norm regularization on the latent TR factors, resulting in the optimization step using singular value decomposition (SVD) being performed at a much smaller scale. By leveraging the alternating direction method of multipliers (ADMM) scheme, the latent TR factors with optimal rank and the recovered tensor can be obtained simultaneously. Our proposed algorithm is shown to effectively alleviate the burden of TR-rank selection, thereby greatly reducing the computational cost. The extensive experimental results on both synthetic and real-world data demonstrate the superior performance and efficiency of the proposed approach against the state-of-the-art algorithms.

show abstract

Section: Introductionmentioning

confidence: 99%

Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion

Yuan

Mandic

et al. 2019

AAAI

120

View full text Add to dashboard Cite

show abstract

“…The most popular tensor network is the Tensor Train (TT) representation, which for a order-d tensor with each dimension of size n requires O(dnr 2 ) parameters, where r is the rank of each of the factors, and thus allows for the efficient data representation [10]. Tensor completion based on tensor train decompositions have been recently considered in [11], [12]. The authors of [11] considered the completion of data based on the alternating least square method.Although the TT format has been widely applied in numerical analysis, its applications to image classification and completion are rather limited [4], [11], [12].…”

mentioning

confidence: 99%

Efficient Low Rank Tensor Ring Completion

Wang

Aggarwal

Aeron

2017

2017 IEEE International Conference on Computer Vision (ICCV)

150

119

View full text Add to dashboard Cite

Using the matrix product state (MPS) representation of the recently proposed tensor ring decompositions, in this paper we propose a tensor completion algorithm, which is an alternating minimization algorithm that alternates over the factors in the MPS representation. This development is motivated in part by the success of matrix completion algorithms that alternate over the (low-rank) factors. In this paper, we propose a spectral initialization for the tensor ring completion algorithm and analyze the computational complexity of the proposed algorithm. We numerically compare it with existing methods that employ a low rank tensor train approximation for data completion and show that our method outperforms the existing ones for a variety of real computer vision settings, and thus demonstrate the improved expressive power of tensor ring as compared to tensor train. I. INTRODUCTIONTensor decompositions for representing and storing data have recently attracted considerable attention due to their effectiveness in compressing data for statistical signal processing [1]- [5]. In this paper we focus on Tensor Ring (TR) decomposition [6] and in particular its relation to Matrix Product States (MPS) [7] representation for tensor representation and use it for completing data from missing entries. In this context our algorithm is motivated by recent work in matrix completion where under a suitable initialization an alternating minimization algorithm [8], [9] over the low rank factors is able to accurately predict the missing data.Recently, tensor networks, considered as the generalization of tensor decompositions, have emerged as the potentially powerful tools for analysis of large-scale tensor data [7]. The most popular tensor network is the Tensor Train (TT) representation, which for a order-d tensor with each dimension of size n requires O(dnr 2 ) parameters, where r is the rank of each of the factors, and thus allows for the efficient data representation [10]. Tensor completion based on tensor train decompositions have been recently considered in [11], [12]. The authors of [11] considered the completion of data based on the alternating least square method.Although the TT format has been widely applied in numerical analysis, its applications to image classification and completion are rather limited [4], [11], [12]. As outlined in [6], TT decomposition suffers from the following limitations. Namely, (i) TT model requires rank-1 constraints on the border factors, (ii) TT ranks are typically small for near-border factors and large for the middle factors, and (iii) the multiplications of the TT factors are not permutation invariant. In order to alleviate those drawbacks, a tensor ring (TR) decomposition has been proposed in [6]. TR decomposition removes the unit rank constraints for the boundary tensor factors and utilizes a

show abstract

“…The essential adaption of the rank however, including similar matrix approaches, is rarely considered, all the less in numerical tests, and remains an open problem in this setting. A mentionable approach so far is the rank increasing strategy [14,44] and its regularization properties are a first starting point for this article.…”

Section: Relation To Other Matrix and Tensor Methodsmentioning

confidence: 99%

Stable ALS approximation in the TT-format for rank-adaptive tensor completion

Grasedyck

Krämer

2019

Numer. Math.

Self Cite

View full text Add to dashboard Cite

Low rank tensor completion is a highly ill-posed inverse problem, particularly when the data model is not accurate, and some sort of regularization is required in order to solve it. In this article we focus on the calibration of the data model. For alternating optimization, we observe that existing rank adaption methods do not enable a continuous transition between manifolds of different ranks. We denote this characteristic as instability (under truncation). As a consequence of this property, arbitrarily small changes in the iterate can have arbitrarily large influence on the further reconstruction. We therefore introduce a singular value based regularization to the standard alternating least squares (ALS), which is motivated by averaging in microsteps. We prove its stability and derive a natural semi-implicit rank adaption strategy. We further prove that the standard ALS microsteps for completion problems are only stable on manifolds of fixed ranks, and only around points that have what we define as internal tensor restricted isometry property, iTRIP. In conclusion, numerical experiments are provided that show improvements of the reconstruction quality up to orders of magnitude in the new Stable ALS Approximation (SALSA) compared to standard ALS and the well known Riemannian optimization RTTC.

show abstract

Variants of Alternating Least Squares Tensor Completion in the Tensor Train Format

Cited by 63 publications

References 23 publications

Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion

Tensor Ring Decomposition with Rank Minimization on Latent Space: An Efficient Approach for Tensor Completion

Efficient Low Rank Tensor Ring Completion

Stable ALS approximation in the TT-format for rank-adaptive tensor completion

Contact Info

Product

Resources

About