Alternating Least Squares as Moving Subspace Correction

Oseledets, Ivan; Rakhuba, Maxim; Uschmajew, André

doi:10.1137/17m1148712

Cited by 15 publications

(18 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The argument for Ŝ2 is analogous. The above invariance of the AO method allows us to interpret it as a method X +1 = S(X ) in the full matrix space R m×n , or more precisely on the subset of matrices of rank at most k. This viewpoint has been taken in [13] and will be helpful in this work, too. From an algorithmic perspective, the AO viewpoint (2.2) is more useful since it operates on the smaller matrices U and V instead of the full matrix X.…”

Section: Standard Ao Methodsmentioning

confidence: 99%

“…While the study of global convergence of the AO method (1.3) is usually difficult, its local convergence properties are well-understood [16,14,13]. The local analysis is based on the fact that the linearized version of the method at a critical point (U * , V * ) takes the form of a block Gauss-Seidel method for the Hessian ∇ 2 F (U * , V * ).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Local convergence of alternating low-rank optimization methods with overrelaxation

Oseledets¹,

Rakhuba²,

Uschmajew³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The local convergence of alternating optimization methods with overrelaxation for lowrank matrix and tensor problems is established. The analysis is based on the linearization of the method which takes the form of an SOR iteration for a positive semidefinite Hessian and can be studied in the corresponding quotient geometry of equivalent low-rank representations. In the matrix case, the optimal relaxation parameter for accelerating the local convergence can be determined from the convergence rate of the standard method. This result relies on a version of Young's SOR theorem for positive semidefinite 2 × 2 block systems.

show abstract

Section: Standard Ao Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Local convergence of alternating low-rank optimization methods with overrelaxation

Oseledets¹,

Rakhuba²,

Uschmajew³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…We focus here on its accuracy in comparison with the two other methods via numerical experiments. We emphasize that for tensors of order at least 3, convergence can be shown for the (inner) iteration (see [11,25,29,30]). This limit, however, is not a global minimum in general.…”

Section: Methods 2: An Alternating Least Squares Type Iterationmentioning

confidence: 99%

Numerical Approximation of Poisson Problems in Long Domains

et al. 2021

View full text Add to dashboard Cite

In this paper, we consider the Poisson equation on a “long” domain which is the Cartesian product of a one-dimensional long interval with a (d − 1)-dimensional domain. The right-hand side is assumed to have a rank-1 tensor structure. We will present and compare methods to construct approximations of the solution which have tensor structure and the computational effort is governed by only solving elliptic problems on lower-dimensional domains. A zero-th order tensor approximation is derived by using tools from asymptotic analysis (method 1). The resulting approximation is an elementary tensor and, hence has a fixed error which turns out to be very close to the best possible approximation of zero-th order. This approximation can be used as a starting guess for the derivation of higher-order tensor approximations by a greedy-type method (method 2). Numerical experiments show that this method is converging towards the exact solution. Method 3 is based on the derivation of a tensor approximation via exponential sums applied to discretized differential operators and their inverses. It can be proved that this method converges exponentially with respect to the tensor rank. We present numerical experiments which compare the performance and sensitivity of these three methods.

show abstract

“…Step 1: Given [45]. The modified ALS iteration has recently been considered in Oseledets et al [38] under the name "simultaneous orthogonal iterations". It is shown there that the modified version is equivalent to ALS and, therefore, has the same rate of convergence.…”

Section: The Alternating Least Squares (Als) Methodsmentioning

confidence: 99%

The Equivalence between Orthogonal Iterations and Alternating Least Squares

Dax¹

2020

ALAMT

View full text Add to dashboard Cite

This note explores the relations between two different methods. The first one is the Alternating Least Squares (ALS) method for calculating a rank-k approximation of a real m n × matrix, A. This method has important applications in nonnegative matrix factorizations, in matrix completion problems, and in tensor approximations. The second method is called Orthogonal Iterations. Other names of this method are Subspace Iterations, Simultaneous Iterations, and block-Power method. Given a real symmetric matrix, G, this method computes k dominant eigenvectors of G. To see the relation between these methods we assume that T G A A =. It is shown that in this case the two methods generate the same sequence of subspaces, and the same sequence of low-rank approximations. This equivalence provides new insight into the convergence properties of both methods.

show abstract

Alternating Least Squares as Moving Subspace Correction

Cited by 15 publications

References 15 publications

Local convergence of alternating low-rank optimization methods with overrelaxation

Local convergence of alternating low-rank optimization methods with overrelaxation

Numerical Approximation of Poisson Problems in Long Domains

The Equivalence between Orthogonal Iterations and Alternating Least Squares

Contact Info

Product

Resources

About