Low-Rank Approximation and Completion of Positive Tensors

Abstract

Abstract. Unlike the matrix case, computing low-rank approximations of tensors is NP-hard and numerically ill-posed in general. Even the best rank-1 approximation of a tensor is NP-hard. In this paper, we use convex optimization to develop polynomial-time algorithms for low-rank approximation and completion of positive tensors. Our approach is to use algebraic topology to define a new (numerically well-posed) decomposition for positive tensors, which we show is equivalent to the standard tensor decomposition in important cases. Though computing this decomposition is a nonconvex optimization problem, we prove it can be exactly reformulated as a convex optimization problem. This allows us to construct polynomial-time randomized algorithms for computing this decomposition and for solving low-rank tensor approximation problems. Among the consequences is that best rank-1 approximations of positive tensors can be computed in polynomial time. Our framework is next extended to the tensor completion problem, where noisy entries of a tensor are observed and then used to estimate missing entries. We provide a polynomial-time algorithm that for specific cases requires a polynomial (in tensor order) number of measurements, in contrast to existing approaches that require an exponential number of measurements. These algorithms are extended to exploit sparsity in the tensor to reduce the number of measurements needed. We conclude by providing a novel interpretation of statistical regression problems with categorical variables as tensor completion problems, and numerical examples with synthetic data and data from a bioengineered metabolic network show the improved performance of our approach on this problem.

Inverse Optimization with Noisy Data

“…Estimation consistency in any statistical setting (including inverse optimization with noisy data) requires that an identifiability condition holds, and such identifiability conditions can be stated under a variety of different mathematical formulations (Wald 1949, Jennrich 1969, Bartlett and Mendelson 2002, Greenshtein and Ritov 2004, Bickel and Doksum 2006, Chatterjee 2014, Aswani 2015). The intuition for these different formulations is the same: Essentially, an identifiability condition states that the output of the model is different for two distinct sets of model parameters.…”

Nonnegative Tensor Completion via Integer Optimization

“…Tensor completion is the problem of observing (possibly with noise) a subset of entries of a tensor and then estimating the remaining entries based on an assumption of low-rankness. The tensor completion problem is encountered in a number of important applications, including computer vision (Liu et al, 2012;Zhang et al, 2019), regression with only categorical variables (Aswani, 2016), healthcare (Gandy et al, 2011;Dauwels et al, 2011), and many other domains (Song et al, 2019).…”

“…However, there are a few special cases of tensors where algorithms that achieve the information-theoretic rate have been developed. Completion of nonnegative rank-1 tensors can be exactly written as a convex optimization problem (Aswani, 2016). For symmetric orthogonal tensors, a variant of the Frank-Wolfe algorithm has been proposed (Rao et al, 2015).…”

“…These notions can be used to measure the complexity of sets of matrices (Srebro and Shraibman, 2005) or tensors (Aswani, 2016). The idea is to interpret each nonnegative tensor as a function ψ : R → R + from a set of indices x ∈ R to the corresponding entry of the tensor ψ x .…”

Multilayer tensor factorization with applications to recommender systems

“…Methods for non-negative CPD are proposed to address this issue [e.g. ; 35,50,11,4], as such decompositions make results meaningful and interpretable. In recommender system problems, however, the direct interpretation of latent factors is less critical, as the relative scale or ranking is important for recommendation.…”