Vectorial Dimension Reduction for Tensors Based on Bayesian Inference

Ju, Fujiao; Sun, Yanfeng; Gao, Junbin; Hu, Yongli; Yin, Baocai

doi:10.1109/tnnls.2017.2739131

Cited by 12 publications

(9 citation statements)

References 39 publications

(54 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…CP-based multilinear PPCAs such as TBVDR [26] use the CP model for preserving the tensor structures. They have a more flexible subspace representation, whereas are more prone to overfitting than Tucker-based PPCAs.…”

Section: Cp-based Multilinear Ppcasmentioning

confidence: 99%

“…Connections with CP-based PPCAs: To the best of our knowledge, TBVDR [26] is the only existing CP-based PPCA. It introduces an additional linear projection W h ∈ R P ×Q into the CP model (3) and defines z = W h h, where h ∈ R Q ∼ N (0, I) serves as the latent features.…”

Section: B Connections With Existing Ppcasmentioning

confidence: 99%

“…where γ is the regularization parameter, τ ≡ 1/σ 2 is the precision (inverse of the noise variance), and τ is the expectation obtained from the variational posterior q(τ ) shown in (26). The above prior distribution provides a probabilistic implementation of moment-based CR, which essentially leads to a similar likelihood function as (18) is constrained to be zero.…”

Section: E Prota With Bayesian Crmentioning

confidence: 99%

“…Algorithms and their settings: PROTA is compared against linear baselines: PCA, PPCA; Tucker-based PCA: MPCA [16]; CP-based PCAs: TROD [19], UMPCA [20]; Tucker-based PPCAs: PSOPCA, BPPCA; Bayesian CPDs: BCPF [28] and VBTCP [31]; and CP-based PPCA: TBVDR [26]. BPPCA has both MLE and MAP implementations.…”

Section: B Classification On 2d Imagesmentioning

confidence: 99%

“…Compared with Tucker-based approaches, CP-based PPCAs are relatively under-developed. To the best of our knowledge, Tensor Bayesian Vectorial Dimension Reduction (TBVDR) [26] is the only existing CP-based multilinear PPCA. It introduces an additional linear projection into the CP model, so that the model complexity and the number of extracted features can be controlled separately.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Probabilistic Rank-One Tensor Analysis With Concurrent Regularizations

Zhou

Cheung

2021

IEEE Trans. Cybern.

View full text Add to dashboard Cite

Subspace learning for tensors attracts increasing interest in recent years, leading to the development of multilinear extensions of Principal Component Analysis (PCA) and Probabilistic PCA (PPCA). Existing multilinear PPCAs are based on the Tucker or CANDECOMP/PARAFAC (CP) models. Although both kinds of multilinear PPCAs have shown their effectiveness in dealing with tensors, they also have their own limitations. Tucker-based multilinear PPCAs have a restrictive subspace representation and suffer from rotational ambiguity, while CPbased ones are more prone to overfitting. To address these problems, we propose Probabilistic Rank-One Tensor Analysis (PROTA), a CP-based multilinear PPCA. PROTA has a more flexible subspace representation than Tucker-based PPCAs, and avoids rotational ambiguity. To alleviate overfitting for CPbased PPCAs, we propose two simple and effective regularization strategies, named as concurrent regularizations. By adjusting the noise variance or the moments of latent features, our strategies concurrently and coherently penalize the whole subspace. This relaxes unnecessary scale restrictions and gains more flexibility in regularizing CP-based PPCAs. To take full advantage of the probabilistic framework, we further propose a Bayesian treatment of PROTA, which achieves both automatic feature determination and robustness against overfitting. Experiments on synthetic and real-world datasets demonstrate the superiority of PROTA in subspace estimation and classification, as well as the effectiveness of concurrent regularizations in alleviating overfitting.

show abstract

Section: Cp-based Multilinear Ppcasmentioning

confidence: 99%

Section: B Connections With Existing Ppcasmentioning

confidence: 99%

Section: E Prota With Bayesian Crmentioning

confidence: 99%

Section: B Classification On 2d Imagesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Probabilistic Rank-One Tensor Analysis With Concurrent Regularizations

Zhou

Cheung

2021

IEEE Trans. Cybern.

View full text Add to dashboard Cite

show abstract

A latent variable model for two-dimensional canonical correlation analysis and the variational inference

Safayani¹,

Momenzadeh²

2020

Soft Comput

View full text Add to dashboard Cite

Describing the dimension reduction (DR) techniques by means of probabilistic models has recently been given special attention. Probabilistic models, in addition to a better interpretability of the DR methods, provide a framework for further extensions of such algorithms. One of the new approaches to the probabilistic DR methods is to preserving the internal structure of data. It is meant that it is not necessary that the data first be converted from the matrix or tensor format to the vector format in the process of dimensionality reduction. In this paper, a latent variable model for matrix-variate data for canonical correlation analysis (CCA) is proposed. Since in general there is not any analytical maximum likelihood solution for this model, we present two approaches for learning the parameters. The proposed methods are evaluated using the synthetic data in terms of convergence and quality of mappings. Also, real data set is employed for assessing the proposed methods with several probabilistic and none-probabilistic CCA based approaches. The results confirm the superiority of the proposed methods with respect to the competing algorithms. Moreover, this model can be considered as a framework for further extensions.

show abstract

Principal Tensor Embedding for Unsupervised Tensor Learning

2020

View full text Add to dashboard Cite

Tensors and multiway analysis aim to explore the relationships between the variables used to represent the data and find a summarization of the data with models of reduced dimensionality. However, although in this context a great attention was devoted to this problem, dimension reduction of high-order tensors remains a challenge. The aim of this paper is to provide a nonlinear dimensionality reduction approach, named principal tensor embedding (PTE), for unsupervised tensor learning, that is able to derive an explicit nonlinear model of data. As in the standard manifold learning (ML) technique, it assumes multidimensional data lie close to a low-dimensional manifold embedded in a high-dimensional space. On the basis of this assumption a local parametrization of data that accurately captures its local geometry is derived. From this mathematical framework a nonlinear stochastic model of data that depends on a reduced set of latent variables is obtained. In this way the initial problem of unsupervised learning is reduced to the regression of a nonlinear input-output function, i.e. a supervised learning problem. Extensive experiments on several tensor datasets demonstrate that the proposed ML approach gives competitive performance when compared with other techniques used for data reconstruction and classification.

show abstract

Vectorial Dimension Reduction for Tensors Based on Bayesian Inference

Cited by 12 publications

References 39 publications

Probabilistic Rank-One Tensor Analysis With Concurrent Regularizations

Probabilistic Rank-One Tensor Analysis With Concurrent Regularizations

A latent variable model for two-dimensional canonical correlation analysis and the variational inference

Principal Tensor Embedding for Unsupervised Tensor Learning

Contact Info

Product

Resources

About