Bayesian Methods for Tensor Regression

Abstract

For many applications pertaining to neuroimaging, social science, international relations, chemometrics, genomics, and molecular omics, datasets often involve variables which are best represented in the form of a multidimensional array or tensor , which extends the familiar two‐way data matrix into higher dimensions. Rather than vectorizing tensor‐valued variables prior to analysis which results in loss of inference, new methods have emerged developing regression relationships between variables with either tensor‐valued response(s) or predictor(s). Bayesian approaches, in particular, have shown great promise in applications pertaining to tensor regressions. A remarkable feature of fully Bayesian approaches is that they allow flexible modeling of tensor‐valued parameters in the regressions involving tensor variables and naturally offer characterization of uncertainty in the parametric and predictive inferences. This article provides a review of some relevant Bayesian models on tensor regressions developed in recent years. We divide methods according to the objective of the analysis. We begin with tensor regression approaches with a scalar response and a tensor‐valued covariate and discuss both parametric and nonparametric modeling options and applications in this framework. We then address the problem of making inference with a tensor response and a vector of covariates, with applications including task‐related brain activation and connectivity studies. Finally, we offer discussion on Bayesian models involving a tensor response and a tensor covariate. Discussion of each model is accompanied by available results on its posterior contraction properties, laying out restrictions on key model parameters (such as the tensor dimensions) to draw accurate posterior inference.

Tensor Linear Regression: Degeneracy and Solution

“…We briefly review the tensor linear regression model [1]- [3], [6], [8]- [10], [12]- [17], [31]- [35]. Suppose we have a data set {(y i , X i ) : i = 1, .…”

“…Tensor regression [1]- [12] has recently received an increasing amount of attention in many applications. In most existing work, low-rank constraints are imposed to overcome the curse of dimensionality [2], [12]- [17]. One popular low-rank modeling is based on CANDECOMP/PARAFAC (CP) decomposition [18].…”

Linear Tensor Projection Revealing Nonlinearity

“…Tucker decomposition [16] is a well-known tensor decomposition using unsupervised methods. Several studies on tensor regression task have been reported [7,8,9,10]. Most of these methods reduce the number of parameters by expressing the partial regression coefficients in a low rank and can be regarded as the multi-linear projection into lower-dimensional tensor subspace.…”

“…The issues described above also apply to the tensor data. Several tensor-regression methods have been proposed for the data represented by matrices or tensors that describe the relationships between variables [7,8,9,10]. However, these methods cannot cope with strong nonlinearities.…”

Efficient Tensor Contraction via Fast Count Sketch

“…Many real-world data exhibit multi-dimensional form and can be naturally represented by tensors. Tensor decomposition is an essential tool for multi-way data analysis [1,2,3,4]. As a generalization of the matrix product, tensor contraction is widely used in tensor factorization based methods [5,6,7,8].…”