Tensor Deflation for CANDECOMP/PARAFAC— Part I: Alternating Subspace Update Algorithm

Phan, Anh-Huy; Tichavský, Petr; Cichocki, Andrzej

doi:10.1109/tsp.2015.2458785

Cited by 29 publications

(31 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An obvious approach would be to replace the TT-tensor G = G 1 • G 2 • · · · • G N in (14) by an equivalent Kruskal tensor for which the factor matrices are found acording to Lemma 2 in the form…”

Section: Sequential Conversion Based On Best Rank-1 Matrix Approximentioning

confidence: 99%

“…Finally, to complete the algorithm, we introduce an efficient expression for the computation of the cost function in (14) or (21) in the form…”

Section: B Progressive Computation Of the Contraction Matrices ψ >N mentioning

confidence: 99%

“…In addition to the ALS algorithm, we can derive other iterative algorithms for the optimization problem in (14), e.g., the Levenberg-Marquardt (LM) algorithm [35]. The results in Lemma 4 are useful for fast computation of the gradients of the objective function with respect to the factor matrices, i.e., the product of the mode-n matricization of the TT-tensor G and the Khatri-Rao product of all-but-one factor matrices A (k) , k n. Following (22), these terms are given by…”

Section: Extensions Of Other Optimization Algorithms For Cpdmentioning

confidence: 99%

“…Since the 1990's, the CPD has attracted attention from signal processing researchers as e.g., the receiving signals in telecommunication and blind source separation often admit the model [5]- [8]. The recent rapid development in machine learning has opened up new applications of CPD in feature extraction, data reconstruction, image completion [9]- [12] and various tracking scenarios [13], [14]. Compared to other tensor decompositions, the CPD exhibits a great advantage in dimensionality reduction.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Tensor Networks for Latent Variable Analysis: Higher Order Canonical Polyadic Decomposition

Phan

Cichocki

Oseledets

et al. 2020

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

View full text Add to dashboard Cite

The Canonical Polyadic decomposition (CPD) is a convenient and intuitive tool for tensor factorization; however, for higher-order tensors, it often exhibits high computational cost and permutation of tensor entries, these undesirable effects grow exponentially with the tensor order. Prior compression of tensor in-hand can reduce the computational cost of CPD, but this is only applicable when the rank R of the decomposition does not exceed the tensor dimensions. To resolve these issues, we present a novel method for CPD of higher-order tensors, which rests upon a simple tensor network of representative interconnected core tensors of orders not higher than 3. For rigour, we develop an exact conversion scheme from the core tensors to the factor matrices in CPD, and an iterative algorithm with low complexity to estimate these factor matrices for the inexact case. Comprehensive simulations over a variety of scenarios support the approach.

show abstract

Section: Sequential Conversion Based On Best Rank-1 Matrix Approximentioning

confidence: 99%

“…Finally, to complete the algorithm, we introduce an efficient expression for the computation of the cost function in (14) or (21) in the form…”

Section: B Progressive Computation Of the Contraction Matrices ψ >N mentioning

confidence: 99%

Section: Extensions Of Other Optimization Algorithms For Cpdmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Tensor Networks for Latent Variable Analysis: Higher Order Canonical Polyadic Decomposition

Phan

Cichocki

Oseledets

et al. 2020

IEEE Trans. Neural Netw. Learning Syst.

Self Cite

View full text Add to dashboard Cite

show abstract

“…It is known that the greedy algorithms cannot generalize to tensor fields straightly [16]. In [17], the authors proposed a deflation method with a constraining procedure after each step. In [18], a similar method was proposed based on successive rank-1 approximations and an iterative process followed for eliminating the residue.…”

Section: Introductionmentioning

confidence: 99%

Joint Range‐Doppler‐Angle Estimation for OFDM‐Based RadCom System via Tensor Decomposition

Kong

Wang

Deng

et al. 2018

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

Radar and communication (RadCom) systems have received increasing attention due to their high energy efficiency and spectral efficiency. They have been identified as green communications. This paper is concerned with a joint estimation of range-Dopplerangle parameters for an orthogonal frequency division multiplexing (OFDM) based RadCom system. The key idea of the proposed method is to derive different factor matrices by the tensor decomposition method and then extract parameters of the targets from these factor matrices. Different from the classical tensor decomposition method via alternating least squares or higher-order singular value decomposition, we adopt a greedy based method with each step constituted by a rank-1 approximation subproblem. To avoid local extremum, the rank-1 approximation is solved by using a multiple random initialized tensor power method with a comparison procedure followed. A parameterized rectification method is also proposed to incorporate the inherent structures of the factor matrices. The proposed algorithm can estimate all the parameters simultaneously without parameter pairing requirement. The numerical experiments demonstrate superior performance of the proposed algorithm compared with the existing methods.

show abstract

Stable Low-Rank Tensor Decomposition for Compression of Convolutional Neural Network

Phan

Sobolev

Sozykin

et al. 2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Most state-of-the-art deep neural networks are overparameterized and exhibit a high computational cost. A straightforward approach to this problem is to replace convolutional kernels with its low-rank tensor approximations, whereas the Canonical Polyadic tensor Decomposition is one of the most suited models. However, fitting the convolutional tensors by numerical optimization algorithms often encounters diverging components, i.e., extremely large rank-one tensors but canceling each other. Such degeneracy often causes the non-interpretable result and numerical instability for the neural network ne-tuning. This paper is the first study on degeneracy in the tensor decomposition of convolutional kernels. We present a novel method, which can stabilize the low-rank approximation of convolutional kernels and ensure efficient compression while preserving the highquality performance of the neural networks. We evaluate our approach on popular CNN architectures for image classification and show that our method results in much lower accuracy degradation and provides consistent performance.

show abstract

Tensor Deflation for CANDECOMP/PARAFAC— Part I: Alternating Subspace Update Algorithm

Cited by 29 publications

References 35 publications

Tensor Networks for Latent Variable Analysis: Higher Order Canonical Polyadic Decomposition

Tensor Networks for Latent Variable Analysis: Higher Order Canonical Polyadic Decomposition

Joint Range‐Doppler‐Angle Estimation for OFDM‐Based RadCom System via Tensor Decomposition

Stable Low-Rank Tensor Decomposition for Compression of Convolutional Neural Network

Contact Info

Product

Resources

About