Learning with tensors: a framework based on convex optimization and spectral regularization

Abstract

We present a framework based on convex optimization and spectral regularization to perform learning when feature observations are multidimensional arrays (tensors). We give a mathematical characterization of spectral penalties for tensors and analyze a unifying class of convex optimization problems for which we present a provably convergent and scalable template algorithm. We then specialize this class of problems to perform learning both in a transductive as well as in an inductive setting. In the transductive case one has an input data tensor with missing features and, possibly, a partially observed matrix of labels. The goal is to both infer the missing input features as well as predict the missing labels. For induction, the goal is to determine a model for each learning task to be used for out of sample prediction. Each training pair consists of a multidimensional array and a set of labels each of which corresponding to related but distinct tasks. In either case the proposed technique exploits precise low multilinear rank assumptions over unknown multidimensional arrays; regularization is based on composite spectral penalties and connects to the concept of Multilinear Singular Value Decomposition. As a by-product of using a tensor-based formalism, our approach allows one to tackle the multi-task case in a natural way. Empirical studies demonstrate the merits of the proposed methods.Editor: Massimiliano Pontil.

Folded-concave penalization approaches to tensor completion

“…In addition, we believe that the folded-concave penalization framework should also yield efficient and robust approaches to more general tensor optimization problems, such as low-rank tensor learning problems [14], and tensor robust principal analysis (tensor RPCA) [41].…”

“…Kressner et al [13] proposed a new algorithm that performs Riemannian optimization techniques on the manifold of tensors of fixed multi-linear rank. Signoretto et al [14] studied a learning framework with tensors and developed a hard completion algorithm for the tensor completion problem. Krishnamurthy and Singh [15] developed an efficient algorithm for tensor completion using the adaptive sampling technique.…”

Least Square Support Tensor Regression Machine Based on Submatrix of the Tensor

“…That means, each tensor data sample can be regarded as an abstract vector [13] whose elements are submatrix types of features. Gathering together the same feature information of different 2 Mathematical Problems in Engineering tensor data sample, we can construct new submatrix training sets and the same number of related training models, from which we can get an equal amount of weight submatrix.…”

A Weighted Nuclear Norm Method for Tensor Completion

“…Their algorithms are based on the Douglas-Rachford splitting technique and its dual variant, the alternating direction method of multipliers. Several other efficient algorithms can be found in [2,6,7].…”