The cluster graphical lasso for improved estimation of Gaussian graphical models

Abstract

The task of estimating a Gaussian graphical model in the high-dimensional setting is considered. The graphical lasso, which involves maximizing the Gaussian log likelihood subject to a lasso penalty, is a well-studied approach for this task. A surprising connection between the graphical lasso and hierarchical clustering is introduced: the graphical lasso in effect performs a two-step procedure, in which (1) single linkage hierarchical clustering is performed on the variables in order to identify connected components, and then (2) a penalized log likelihood is maximized on the subset of variables within each connected component. Thus, the graphical lasso determines the connected components of the estimated network via single linkage clustering. The single linkage clustering is known to perform poorly in certain finite-sample settings. Therefore, the cluster graphical lasso, which involves clustering the features using an alternative to single linkage clustering, and then performing the graphical lasso on the subset of variables within each cluster, is proposed. Model selection consistency for this technique is established, and its improved performance relative to the graphical lasso is demonstrated in a simulation study, as well as in applications to a university webpage and a gene expression data sets.

Block-Diagonal Covariance Selection for High-Dimensional Gaussian Graphical Models

“…[HSDR14] have improved the computational cost of the two-step procedure by using a quadratic approximation, and have proved the superlinear convergence of their algorithm. [TWS15] have noticed that the first step, i.e. the detection of connected components by thresholding, is equivalent to performing a single linkage clustering on the variables.…”

An overview of large‐dimensional covariance and precision matrix estimators with applications in chemometrics

“…Several algorithms have been proposed to solve the problem Equation . To date, the most popular approach is the graphical lasso (glasso), where a solution to (Equation ) is found by solving a series of coupled regression problems in an iterative fashion . A special property of glasso is that the estimated precision matrix is always positive definite as long as the algorithm is initialized with a positive definite matrix such as a shrinkage estimator .…”

“…To date, the most popular approach is the graphical lasso (glasso), where a solution to (Equation 19) is found by solving a series of coupled regression problems in an iterative fashion. 27,[75][76][77] A special property of glasso is that the estimated precision matrix is always positive definite as long as the algorithm is initialized with a positive definite matrix such as a shrinkage estimator. 23 Hsieh et al propose a quadratic approximation to the objective in Equation 19 to reduce the computational load.…”

A component lasso

“…Furthermore, there is an interesting connection to hierarchical clustering. Specifically the connected components correspond to the subtrees from when we apply single linkage agglomerative clustering to S and then cut the dendrogram at level (Tan et al., ). Single linkage clustering is sometimes not very attractive in practice, as it can produce long and stringy clusters and hence components of very unequal size.…”