Extrapolated Alternating Algorithms for Approximate Canonical Polyadic Decomposition

Ang, Andersen; Cohen, Jérémy E.; Hien, Le Thi Khanh; Gillis, Nicolas

doi:10.1109/icassp40776.2020.9053849

Cited by 7 publications

(16 citation statements)

References 16 publications

(19 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Another possible extension is support for other loss functions, such as the KL-divergence for count data or weighted least squares for missing data. It may also be beneficial to use Nesterovtype extrapolation, which has been successfully used to speed up ALS schemes for CP [36,44] as well as ALS and flexible coupling schemes for PARAFAC2 [50].…”

Section: Discussionmentioning

confidence: 99%

An AO-ADMM approach to constraining PARAFAC2 on all modes

Roald,

Schenker,

Calhoun

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Analyzing multi-way measurements with variations across one mode of the dataset is a challenge in various fields including data mining, neuroscience and chemometrics. For example, measurements may evolve over time or have unaligned time profiles. The PARAFAC2 model has been successfully used to analyze such data by allowing the underlying factor matrices in one mode (i.e., the evolving mode) to change across slices. The traditional approach to fit a PARAFAC2 model is to use an alternating least squares-based algorithm, which handles the constant cross-product constraint of the PARAFAC2 model by implicitly estimating the evolving factor matrices. This approach makes imposing regularization on these factor matrices challenging. There is currently no algorithm to flexibly impose such regularization with general penalty functions and hard constraints. In order to address this challenge and to avoid the implicit estimation, in this paper, we propose an algorithm for fitting PARAFAC2 based on alternating optimization with the alternating direction method of multipliers (AO-ADMM). With numerical experiments on simulated data, we show that the proposed PARAFAC2 AO-ADMM approach allows for flexible constraints, recovers the underlying patterns accurately, and is computationally efficient compared to the state-of-the-art. We also apply our model to a realworld chromatography dataset, and show that constraining the evolving mode improves the interpretability of the extracted patterns.

show abstract

Section: Discussionmentioning

confidence: 99%

An AO-ADMM approach to constraining PARAFAC2 on all modes

Roald,

Schenker,

Calhoun

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Furthermore, it can be worthwhile to consider Nesterov-type acceleration to increase efficiency of the AO-ADMM framework. Accelerations of this type have previously been applied to ALS for fitting CP decompositions [74,75].…”

Section: Discussionmentioning

confidence: 99%

A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplings

Schenker,

Cohen,

Acar

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Coupled matrix and tensor factorizations (CMTF) are frequently used to jointly analyze data from multiple sources, also called data fusion. However, different characteristics of datasets stemming from multiple sources pose many challenges in data fusion and require to employ various regularizations, constraints, loss functions and different types of coupling structures between datasets. In this paper, we propose a flexible algorithmic framework for coupled matrix and tensor factorizations which utilizes Alternating Optimization (AO) and the Alternating Direction Method of Multipliers (ADMM). The framework facilitates the use of a variety of constraints, loss functions and couplings with linear transformations in a seamless way. Numerical experiments on simulated and real datasets demonstrate that the proposed approach is accurate, and computationally efficient with comparable or better performance than available CMTF methods for Frobenius norm loss, while being more flexible. Using Kullback-Leibler divergence on count data, we demonstrate that the algorithm yields accurate results also for other loss functions.

show abstract

“…[13][14][15][16] Recently, several accelerated versions of ALS exploit the idea of extrapolation to accelerate the convergence of ALS and they have demonstrated strong performance against ALS. 17,18 A key to these algorithms is the use of sophisticated adaptive extrapolation, but they lack a mechanism to directly address the convergence bottlenecks. Our work proposes an adaptive random perturbation mechanism to explicitly overcome the convergence bottlenecks.…”

Section: Motivationmentioning

confidence: 99%

“…1 The alternating least squares (ALS) algorithm is often used for computing the CP decomposition, but it suffers from slow convergence in many cases. Following a number of previous works, [13][14][15][16][17] we propose a new accelerated ALS algorithm.…”

Section: Introductionmentioning

confidence: 99%

Blockwise acceleration of alternating least squares for canonical tensor decomposition

Evans

Yang

2023

Numerical Linear Algebra App

View full text Add to dashboard Cite

The canonical polyadic (CP) decomposition of tensors is one of the most important tensor decompositions. While the well‐known alternating least squares (ALS) algorithm is often considered the workhorse algorithm for computing the CP decomposition, it is known to suffer from slow convergence in many cases and various algorithms have been proposed to accelerate it. In this article, we propose a new accelerated ALS algorithm that accelerates ALS in a blockwise manner using a simple momentum‐based extrapolation technique and a random perturbation technique. Specifically, our algorithm updates one factor matrix (i.e., block) at a time, as in ALS, with each update consisting of a minimization step that directly reduces the reconstruction error, an extrapolation step that moves the factor matrix along the previous update direction, and a random perturbation step for breaking convergence bottlenecks. Our extrapolation strategy takes a simpler form than the state‐of‐the‐art extrapolation strategies and is easier to implement. Our algorithm has negligible computational overheads relative to ALS and is simple to apply. Empirically, our proposed algorithm shows strong performance as compared to the state‐of‐the‐art acceleration techniques on both simulated and real tensors.

show abstract

Extrapolated Alternating Algorithms for Approximate Canonical Polyadic Decomposition

Cited by 7 publications

References 16 publications

An AO-ADMM approach to constraining PARAFAC2 on all modes

An AO-ADMM approach to constraining PARAFAC2 on all modes

A Flexible Optimization Framework for Regularized Matrix-Tensor Factorizations with Linear Couplings

Blockwise acceleration of alternating least squares for canonical tensor decomposition

Contact Info

Product

Resources

About