Joint estimation of multiple graphical models

Abstract

Gaussian graphical models explore dependence relationships between random variables, through the estimation of the corresponding inverse covariance matrices. In this paper we develop an estimator for such models appropriate for data from several graphical models that share the same variables and some of the dependence structure. In this setting, estimating a single graphical model would mask the underlying heterogeneity, while estimating separate models for each category does not take advantage of the common structure. We propose a method that jointly estimates the graphical models corresponding to the different categories present in the data, aiming to preserve the common structure, while allowing for differences between the categories. This is achieved through a hierarchical penalty that targets the removal of common zeros in the inverse covariance matrices across categories. We establish the asymptotic consistency and sparsity of the proposed estimator in the high-dimensional case, and illustrate its performance on a number of simulated networks. An application to learning semantic connections between terms from webpages collected from computer science departments is included.

Dynamic Covariance Models

“…Yuan (2010) and estimated the static precision matrix under a sparsity assumption via linear programming. Guo et al (2011) and Danaher et al (2014) studied the estimation of multiple graphical models when several datasets are available.…”

Sharing and specificity of co-expression networks across 35 human tissues

“…We provide a schematic illustration of our algorithm in Fig 2. Previous work suggests the promise of using transfer learning to learn multiple genetic networks [18,20,21,23]; hierarchical models have also been used more broadly throughout biology, for example to study phylogenies [24]. [18] used prior knowledge of a hierarchy of cancer cell types to learn a network for each cell type.…”

Learning a common substructure of multiple graphical Gaussian models

“…There are several approaches for choosing these, such as cross-validation (Yuan and Lin, 2007;Guo et al, 2011) or the Bayesian information criterion (Guo et al, 2011). Apart from selection techniques, the following result gives us some insight into ρ and γ, and is helpful for analyzing the data more intensively.…”

“…In both cases, the problem is formulated under the assumption that all matrices share the same zero patterns. Guo et al (2011) considered a method to avoid this additional assumption, although the problem then loses convexity. Though these approaches achieved some success in improving the estimation accuracy of graphical models, this does not necessarily mean that they are suitable for finding commonness across datasets as we will see in the simulation.…”