Block-randomized Stochastic Proximal Gradient for Constrained Low-rank Tensor Factorization

Fu, Xiao; Gao, Cheng; Wai, Hoi-To; Huang, Kejun

doi:10.1109/icassp.2019.8682465

Cited by 6 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Part of the work was submitted to ICASSP 2019 [35]. In this new version, we have additionally included detailed convergence proofs and the new adaptive stepsize based algorithm.…”

Section: Introductionmentioning

confidence: 99%

Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization

Ibrahim

Wai

et al. 2020

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

This work considers the problem of computing the canonical polyadic decomposition (CPD) of large tensors. Prior works mostly leverage data sparsity to handle this problem, which is not suitable for handling dense tensors that often arise in applications such as medical imaging, computer vision, and remote sensing. Stochastic optimization is known for its low memory cost and per-iteration complexity when handling dense data. However, exisiting stochastic CPD algorithms are not flexible enough to incorporate a variety of constraints/regularizations that are of interest in signal and data analytics. Convergence properties of many such algorithms are also unclear. In this work, we propose a stochastic optimization framework for large-scale CPD with constraints/regularizations. The framework works under a doubly randomized fashion, and can be regarded as a judicious combination of randomized block coordinate descent (BCD) and stochastic proximal gradient (SPG). The algorithm enjoys lightweight updates and small memory footprint. In addition, this framework entails considerable flexibility-many frequently used regularizers and constraints can be readily handled under the proposed scheme. The approach is also supported by convergence analysis. Numerical results on large-scale dense tensors are employed to showcase the effectiveness of the proposed approach.

show abstract

“…Part of the work was submitted to ICASSP 2019 [35]. In this new version, we have additionally included detailed convergence proofs and the new adaptive stepsize based algorithm.…”

Section: Introductionmentioning

confidence: 99%

Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization

Ibrahim

Wai

et al. 2020

IEEE Trans. Signal Process.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Low-rankness is an important prior of the underlying HSI and has been widely used in image processing [47]- [56]. NGMeet unfolds the original HSI along the spectral dimension as a matrix and optimizes the matrix rank while the unfolding operation breaks the spatial structure.…”

Section: B Proposed Gnlr Model and Algorithmmentioning

confidence: 99%

Low-Rank Tensor Optimization With Nonlocal Plug-and-Play Regularizers for Snapshot Compressive Imaging

Li¹,

Zhao²,

Lin³

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

The increasing volume of hyperspectral images (HSIs) brings great challenges to storage and transmission. Recently, snapshot compressive imaging (SCI), which compresses 3D HSIs into 2D measurements, has received increasing attention. Since the original HSIs can be naturally represented as thirdorder tensors, in this work, we reformulate the degradation model in the SCI systems as a tensor-based form, which friendly allows us to explore the underlying low-rank tensor structure of HSIs. To address the ill-posed SCI reconstruction problem, we suggest a global low-rank tensor optimization model with nonlocal plugand-play (PnP) regularizers (GNLR) to reconstruct the HSI from the 2D measurement, which flexibly and collaboratively integrates three-directional tensor nuclear norm (3DTNN) and two implicit nonlocal regularizers. More concretely, 3DTNN characterizes the global correlation of the underlying HSIs. Two implicit regularizers under the PnP framework exploit the benefits of the transformed sparse and low-rank priors on similar patches of the coefficient tensor, respectively. Based on the alternating direction method of multipliers (ADMM) algorithm, we develop an efficient algorithm to tackle the proposed model. Extensive experiments on remotely sensed HSIs and real-world HSIs demonstrate the superiority of the proposed GNLR method.

show abstract

“…An alternate approach to speeding up CP computations is by reducing the tensor size either via sampling or compression. A large body of work exists for randomized tensor methods [5,11,43,56] which are recently being extended to the constrained problem [12,13]. The second approach is to first compress the tensor using a different decomposition, like Tucker, and then compute CP on this reduced array.…”

Section: Related Workmentioning

confidence: 99%

PLANC: Parallel Low Rank Approximation with Non-negativity Constraints

Eswar,

Hayashi,

Ballard

et al. 2019

Preprint

View full text Add to dashboard Cite

We consider the problem of low-rank approximation of massive dense non-negative tensor data, for example to discover latent patterns in video and imaging applications. As the size of data sets grows, single workstations are hitting bottlenecks in both computation time and available memory. We propose a distributed-memory parallel computing solution to handle massive data sets, loading the input data across the memories of multiple nodes and performing efficient and scalable parallel algorithms to compute the low-rank approximation. We present a software package called PLANC (Parallel Low Rank Approximation with Non-negativity Constraints), which implements our solution and allows for extension in terms of data (dense or sparse, matrices or tensors of any order), algorithm (e.g., from multiplicative updating techniques to alternating direction method of multipliers), and architecture (we exploit GPUs to accelerate the computation in this work). We describe our parallel distributions and algorithms, which are careful to avoid unnecessary communication and computation, show how to extend the software to include new algorithms and/or constraints, and report efficiency and scalability results for both synthetic and real-world data sets.

show abstract

Block-randomized Stochastic Proximal Gradient for Constrained Low-rank Tensor Factorization

Cited by 6 publications

References 16 publications

Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization

Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization

Low-Rank Tensor Optimization With Nonlocal Plug-and-Play Regularizers for Snapshot Compressive Imaging

PLANC: Parallel Low Rank Approximation with Non-negativity Constraints

Contact Info

Product

Resources

About