Sparse inverse covariance matrix estimation via the $ \newcommand{\e}{{\rm e}} \ell_{0}$ -norm with Tikhonov regularization

Abstract

Sparse inverse covariance matrix estimation is a fundamental problem in a Gaussian network model and has attracted much attention in the last decade. A widely used approach for estimating the sparse inverse covariance matrix is the regularized negative log-likelihood minimization, where the regularization term promotes the sparsity of the inverse covariance matrix. Although the -norm is the most popular sparsity promoting function due to its convexity, it may not result in sufficiently satisfied estimations. The -norm is the most natural function to measure sparsity and the -norm regularized negative log-likelihood minimization has been demonstrated to produce favorable results. However, when the sample covariance matrix is singular, the -norm problem has no optimal solution. In this paper, we propose to estimate the sparse inverse covariance matrix via simultaneously using the -norm and the Tikhonov regularization. The resulting minimization problem is guaranteed to have optimal solutions. To overcome the numerical difficulty caused by the -norm, we utilize the penalty decomposition approach. We prove the asymptotic approximation and exact approximation to local (resp., global) minimizers of the original problem by those of the penalty decomposition problem. Then, local minimizers of the penalty decomposition problem are characterized as fixed-points of a system of fixed-point equations with the help of proximity operators. Based on these theoretical results, we design for solving the proposed minimization problem a penalty decomposition fixed-point proximity algorithm and provide the convergence analysis of the algorithm. Numerical results indicate that the proposed algorithm outperforms the compared state-of-the-art algorithms in terms of accuracy.

A novel probabilistic generator for large-scale gene association networks

“…A review of recent methodological papers showed that the Erdo ¨s-Re ´nyi [22][23][24][32][33][34][35][36], Watts-Strogatz [15,33,35,37,38], and Baraba ´si-Albert [23-25, 32, 33, 35, 37, 39-46] models were most commonly used to generate networks, so these will form the basis of comparison for the proposed generator. Block diagonal network structures were also found in some papers [47][48][49], but these are used for specific comparisons rather than assessing general performance.…”

Gaussian graphical models with graph constraints for magnetic moment interaction in high entropy alloys

“…graphical Lasso (Jerome et al, 2007), or gLasso, and a recent Penalized Likelihood method with Tikhonov regularization under an ℓ 0 constraint (Liu and Zhang, 2019), or PLT 0 . Graphical lasso estimates Σ −1 by min {− log( det X) + tr(SX) + λ X 1 } (23) over non-negative definite matrices X, where λ > 0 is the regularization parameter.…”

Representation and reconstruction of covariance operators in linear inverse problems

“…In the discrete setting, also other forms of regularization have been adopted, e.g. sparsity on the inverse covariance matrix (Friedman et al 2008, Liu andZhang 2019).…”