An inexact interior point method for L 1-regularized sparse covariance selection

Abstract

Sparse covariance selection problems can be formulated as log-determinant (log-det) semidefinite programming (SDP) problems with large numbers of linear constraints. Standard primal-dual interior-point methods that are based on solving the Schur complement equation would encounter severe computational bottlenecks if they are applied to solve these SDPs. In this paper, we consider a customized inexact primal-dual path-following interior-point algorithm for solving large scale log-det SDP problems arising from sparse covariance selection problems. Our inexact algorithm solves the large and ill-conditioned linear system of equations in each iteration by a preconditioned iterative solver. By exploiting the structures in sparse covariance selection problems, we are able to design highly effective preconditioners to efficiently solve the large and illconditioned linear systems. Numerical experiments on both synthetic and real covariance selection problems show that our algorithm is highly efficient and outperforms other existing algorithms.keywords: log-determinant semidefinite programming, sparse inverse covariance selection, inexact interior point method, inexact search direction, iterative solver

Inverse covariance estimation from data with missing values using the Concave-Convex Procedure

“…Following experiments done by [14] and [24], we use the biology datasets preprocessed by [16] to compare the performance of our algorithm with EM, in the hypothetical case that values were missing from data. This is an interesting case in practice, as collecting hundreds of biological parameters for each experiment may become expensive.…”

A Proximal Point Algorithm for Log-Determinant Optimization with Group Lasso Regularization

“…Unfortunately, it is impossible to solve (1.2) efficiently on a common PC via IPMs when is large, say more than 200. As a result, many customized algorithms for solving (1.2) and related problems have been designed in the literature, e.g., block coordinate descent method [21,2,61], projected subgradient method [16], Nesterov's first-order methods [41,42] and their variants [2,34,35], alternating direction method (ADM) [60], Newton-CG based proximal point algorithm (PPA) [53], and inexact IPM with effective preconditioners [33]. In general, first-order algorithms (block coordinate descent, projected subgradient, ADM, Nesterov's methods and their variants) are easily implementable and fast to obtain low/moderate accuracy solutions.…”

Learning a common substructure of multiple graphical Gaussian models

“…Several efficient optimization techniques are available for solving this problem. Examples include GLasso (Friedman et al, 2008), PSM (Duchi et al, 2008a), IPM (Li and Toh, 2010), SINCO (Scheinberg and Rish, 2010), ADMM (Yuan, 2009; and QUIC (Hsieh et al, 2011).…”

“…In several previous studies, synthetic sparse precision matrices are generated in a naive manner, that is, just adding a properly scaled identity matrix to a sparse symmetric matrix so that the resulting matrix is sparse and positive definite (Banerjee et al, 2008;Wang et al, 2009;Li and Toh, 2010). In our simulations, we take a different approach to generating a sparse precision matrix for compatibility with the next step.…”

“…Since classical algorithms for this do not scale to high dimensional data, the scope of studies has shifted to a relaxed setting (Meinshausen and Bühlmann, 2006;Yuan and Lin, 2007;Banerjee et al, 2008), where Covariance Selection is formulated as a convex optimization problem using a ℓ 1 -regularization that induces zeros in the resulting matrix. Because of the effectiveness of the relaxed formulation, several related optimization techniques have also been studied (Friedman et al, 2008;Duchi et al, 2008a;Li and Toh, 2010;Scheinberg and Rish, 2010;Yuan, 2009;Hsieh et al, 2011).…”