Low Complexity Damped Gauss--Newton Algorithms for CANDECOMP/PARAFAC

Abstract

Abstract. The damped Gauss-Newton (dGN) algorithm for CANDECOMP/PARAFAC (CP) decomposition can handle the challenges of collinearity of factors and different magnitudes of factors; nevertheless, for factorization of an N-D tensor of size I 1 × . . . × I N with rank R, the algorithm is computationally demanding due to construction of large approximate Hessian of size (RT × RT ) and its inversion where T = n I n . In this paper, we propose a fast implementation of the dGN algorithm which is based on novel expressions of the inverse approximate Hessian in block form. The new implementation has lower computational complexity, besides computation of the gradient (this part is common to both methods), requiring the inversion of a matrix of size NR 2 × NR 2 , which is much smaller than the whole approximate Hessian, if T ≫ NR. In addition, the implementation has lower memory requirements, because neither the Hessian nor its inverse never need to be stored in their entirety. A variant of the algorithm working with complex valued data is proposed as well. Complexity and performance of the proposed algorithm is compared with those of dGN and ALS with line search on examples of difficult benchmark tensors.

EEG extended source localization: Tensor-based vs. conventional methods

“…The CP decomposition can be computed using Alternating Least Squares (ALS) (Bro, 1998), efficient gradient-based algorithms (Phan et al, 2013;Sorber et al, 2013), or semialgebraic methods as presented in (Luciani and Albera, 2011;Haardt, 2008, 2013). Because of its good performance and robustness to collinear factors, overestimation of the number of CP components, and initialization, we employ the DIAG (Direct Algorithm for canonical polyadic decomposition) algorithm Albera, 2011, 2014).…”

Research on bispectrum analysis of secondary feature for vehicle exterior noise based on nonnegative tucker3 decomposition

“…Meanwhile, NTD is of success to overcome the problem of overfitting in theory. Related conclusions are also discussed in [24].…”

Breaking the Curse of Dimensionality Using Decompositions of Incomplete Tensors: Tensor-based scientific computing in big data analysis

“…These methods exploit the structure in the objective function's approximate Hessian. Due to the NLS framework, second order convergence can be attained under certain circumstances [25], [27]. The latter two methods are both guaranteed to converge to a stationary point, which can be a local optimum, however.…”