The probabilistic tensor decomposition toolbox

Abstract

This article introduces the probabilistic tensor decomposition toolbox -a MATLAB toolbox for tensor decomposition using Variational Bayesian inference and Gibbs sampling. An introduction and overview of probabilistic tensor decomposition and its connection with classical tensor decomposition methods based on maximum likelihood is provided. We subsequently describe the probabilistic tensor decomposition toolbox which encompasses the Canonical Polyadic, Tucker, and Tensor Train decomposition models. Currently, unconstrained, non-negative, orthogonal, and sparse factors are supported. Bayesian inference forms a principled way of incorporating prior knowledge, prediction of held-out data, and estimating posterior probabilities. Furthermore, it facilitates automatic model order determination, automatic regularization on factors (e.g. sparsity), and inherently penalizes model complexity which is beneficial when inferring hierarchical models, such as heteroscedastic noise modelling. The toolbox allows researchers to easily apply Bayesian tensor decomposition methods without the need to derive or implement these methods themselves. Furthermore, it serves as a reference implementation for comparing existing and new tensor decomposition methods. The software is available from https://github.com/JesperLH/prob-tensor-toolbox/. © 2020 The Author(s). Published by IOP Publishing Ltd Mach. Learn.: Sci. Technol. 1 (2020) 025011 J L Hinrich et al Bayesian tensor decompositionFor comprehensive reviews of maximum likelihood based tensor decomposition methods the reader is referred to existing tensor decomposition reviews [6-9, 22, 23]. We presently provide a short overview of Bayesian tensor decomposition. For brevity, this section will drop the prefix Bayesian and unless otherwise stated all models are based on fully Bayesian inference.Currently, probabilistic models are primarily based on the Tucker and CP decomposition. The earliest works using Tucker decomposition are found in [24,25] where both the core array and factors follow a normal distribution. The latter also considered enforcing sparsity on the core array to automatically learn the most prominent multi-linear interactions. However, neither of these methods were fully Bayesian as they relied on maximum-a-posteriori estimation which provides a point estimate of the posterior distribution. The first fully Bayesian Tucker model with normal factors using Gibbs sampling was proposed in [26] and the indeterminacy and structure of the core array was explored in [27]. The Tucker decomposition was extended to handle missing values and sparse noise based on variational Bayesian (VB) inference [28] whereas the Tucker model with an infinite core size was explored in [29][30][31] with factors specified as Gaussian processes and inference based on VB. Extensions of the Tucker model to count data was explored in [32,33] using the Poisson likelihood and either Gamma or Dirichlet factors. For categorical data, a Tucker model with multinomial likelihood and factors was considered in [34]....

Bayesian estimation of group event-related potential components (BEGEP): testing a model for synthetic and real datasets

“…The first one is based on the assumption that the matrix A = {a d,n } is sparse, that is, the set of its elements is equal to zero. This technique, known as the automatic relevance determination (ARD), is used in many applications [25]. In our case, it was assumed that the parameters λ (a) d,n for the VB algorithm are not constant, but have a suitable gamma distribution that provides a sparse estimate of the matrix A [12,26].…”

Alternating Linear Scheme in a Bayesian Framework for Low-Rank Tensor Approximation

“…The first literature about the probabilistic treatment of the TT decomposition using von-Mises--Fisher priors on the orthogonal cores and variational approximation with evidence lower bound is introduced by [18]. Recently, the authors of [17] published the probabilistic TD toolbox for MATLAB, providing inference with variational Bayes and with Gibbs sampling.…”

Alternating linear scheme in a Bayesian framework for low-rank tensor approximation

“…The first literature about the probabilistic treatment of the tensor train decomposition using von-Mises-Fisher priors on the orthogonal cores and variational approximation with evidence lower bound is introduced by [17]. Recently, [16] published the probabilistic tensor decomposition toolbox for MATLAB, providing inference with variational Bayes and with Gibbs sampling.…”