Tensor Graphical Model: Non-Convex Optimization and Statistical Inference

Abstract

We consider the estimation and inference of graphical models that characterize the dependency structure of high-dimensional tensor-valued data. To facilitate the estimation of the precision matrix corresponding to each way of the tensor, we assume the data follow a tensor normal distribution whose covariance has a Kronecker product structure. A critical challenge in the estimation and inference of this model is the fact that its penalized maximum likelihood estimation involves minimizing a non-convex objective function. To address it, this paper makes two contributions: (i) In spite of the non-convexity of this estimation problem, we prove that an alternating minimization algorithm, which iteratively estimates each sparse precision matrix while fixing the others, attains an estimator with an optimal statistical rate of convergence. (ii) We propose a de-biased statistical inference procedure for testing hypotheses on the true support of the sparse precision matrices, and employ it for testing a growing number of hypothesis with false discovery rate (FDR) control. The asymptotic normality of our test statistic and the consistency of FDR control procedure are established. Our theoretical results are backed up by thorough numerical studies and our real applications on neuroimaging studies of Autism spectrum disorder and users' advertising click analysis bring new scientific findings and business insights. The proposed methods are encoded into a publicly available R package Tlasso.

A general framework for tensor screening through smoothing

“…If we are fitting a specific kind of tensor model, it is also possible to modify the screening utilities to further take advantage of the tensor structure (with or without the smoothness assumption). For example, under the TDA model, the variance σ kJ can be estimated much more accurately [33,42,40] than the sample estimate. The improved estimation could benefit screening.…”

“…3), Σ 2 = I 64 , D is a diamond area with 25 variables where the four vertexes are located at (30,20), (36,20), (33,17), (33,23) and…”

SG-PALM: a Fast Physically Interpretable Tensor Graphical Model

“…For instance, other popular algorithms for tensor-variate graphical models, such as the TG-ISTA presented in Greenewald et al (2019) and the Tlasso proposed in Lyu et al (2019) both require inversion of d k × d k matrices, which is non-parallelizable and requires O(d 3 k ) operations for each k. In particular, TeraLasso's TG-ISTA algorithm requires…”

“…Figure 2 shows the root mean squared error normalized by the difference between maximum and minimum pixels (NRMSE) over the testing samples, for the forecasts based on the SG-PALM estimator, TeraLasso estimator (Greenewald et al, 2019), Tlasso estimator (Lyu et al, 2019), and IndLasso estimator. Here, the TeraLasso and the Tlasso are estimation algorithms for a KS and a KP tensor precision matrix model, respectively; the IndLasso denotes an estimator obtained by applying independent and separate 1 -penalized regressions to each pixel in y t .…”

“…To address such sample complexity challenges, sparsity is often imposed on the covariance Σ or the inverse covariance Ω, e.g., by using a sparse Kronecker product (KP) or Kronecker sum (KS) decomposition of Σ or Ω. The earliest and most popular form of sparse structured precision matrix estimation approaches represent Ω, equivalently Σ, as the KP of smaller precision/covariance matrices (Allen & Tibshirani, 2010;Leng & Tang, 2012;Yin & Li, 2012;Tsiligkaridis et al, 2013;Zhou, 2014;Lyu et al, 2019). The KP structure induces a generative representation for the tensor-variate data via a separable covariance/inverse covariance model.…”