New ALS Methods With Extrapolating Search Directions and Optimal Step Size for Complex-Valued Tensor Decompositions

Abstract

In signal processing, data analysis and scientific computing, one often encounters the problem of decomposing a tensor into a sum of contributions. To solve such problems, both the search direction and the step size are two crucial elements in numerical algorithms, such as alternating least squares algorithm (ALS). Owing to the nonlinearity of the problem, the often used linear search direction is not always powerful enough. In this paper, we propose two higher-order search directions. The first one, geometric search direction, is constructed via a combination of two successive linear directions. The second one, algebraic search direction, is constructed via a quadratic approximation of three successive iterates. Then, in an enhanced line search along these directions, the optimal complex step size contains two arguments: modulus and phase. A current strategy is ELSCS that finds these two arguments alternately. So it may suffer from a local optimum. We broach a direct method, which determines these two arguments simultaneously, so as to obtain the global optimum. Finally, numerical comparisons on various search direction and step size schemes are reported in the context of blind separation-equalization of convolutive DS-CDMA mixtures. The results show that the new search directions have greatly improve the efficiency of ALS and the new step size strategy is competitive.

From basis components to complex structural patterns

“…For two simplified tensor decompositions in Lemmas 2.1 and 2.2, CPD-like algorithms can be efficiently used such as the ALS algorithm [6] (often with line search extrapolation methods [15][16][17]) or the fast damped Gauss-Newton algorithm [18]. For space reasons, we present the ALS algorithm for the structured CPD in Lemma 2.2 which sequentially updates U (n) and V (n) …”

Generalized generating function with tucker decomposition and alternating least squares for underdetermined blind identification

“…The standard way for computing the tensor decomposition is by using an ' ALS' algorithm [20]. Several improved versions, such as the enhanced line search (ELS) [21] and extrapolating search direction (ESD) [22], are proposed to accelerate the rate of convergence of the ALS. Hence, the ALS is chosen here to compute the CAND.…”

Low rank canonical polyadic decomposition of tensors based on group sparsity

“…Several modifications have been proposed in order to improve its behavior, especially in the presence of bottlenecks [9]. For instance, Enhanced Line Search (ELS) procedures, based on a sophisticated extrapolation scheme, using information on nonlinear trends in the parameters, was designed [10] [11]. Despite the practical good results of the ELS-ALS technique, no global minimization of the used data-fit objective function is guaranteed.…”