Variational Bayesian inference and complexity control for stochastic block models

Abstract

It is now widely accepted that knowledge can be acquired from networks by clustering their vertices according to connection profiles. Many methods have been proposed and in this paper we concentrate on the Stochastic Block Model (SBM). The clustering of vertices and the estimation of SBM model parameters have been subject to previous work and numerous inference strategies such as variational Expectation Maximization (EM) and classification EM have been proposed. However, SBM still suffers from a lack of criteria to estimate the number of components in the mixture. To our knowledge, only one model based criterion, ICL, has been derived for SBM in the literature. It relies on an asymptotic approximation of the Integrated Complete-data Likelihood and recent studies have shown that it tends to be too conservative in the case of small networks. To tackle this issue, we propose a new criterion that we call ILvb, based on a non asymptotic approximation of the marginal likelihood. We describe how the criterion can be computed through a variational Bayes EM algorithm.

How Many Communities Are There?

“…Apart from CL-BIC and BIC, we also consider the Integrated Likelihood Variational Bayes (VB) approach of Latouche et al (2012). To estimate the true community number, their method selects the candidate value k which maximizes a variational Bayes approximation to the observed-data log-likelihood.…”

“…The remaining entries of θ are set to 0.05. Hence, following Latouche et al (2012), vertices from community 4 connect with probability 0.35 to any other vertices in the network, forming a community of only hubs. Community sizes are the same as in Simulation 1, with results collected in Table 3.…”

“…For the literature addressing the problem of selecting K, besides the block-wise edge splitting method of Chen and Lei (2014), a common practice is to use BIC-type criteria (Airoldi et al, 2008;Daudin et al, 2008) or a variational Bayes approach (Latouche et al, 2012;Hunter et al, 2012). An inherently related problem is that of selecting the number of components in mixture models, where the birth-and-death point process of Stephens (2000) and the allocation sampler of Nobile and Fearnside (2007) provide two fully Bayesian approaches in the case where K is finite but unknown.…”

“…Using techniques from the composite likelihood paradigm (Lindsay, 1988), we develop a composite likelihood BIC (CL-BIC) approach (Gao and Song, 2010) for selecting the community number in the situation where assumed independencies in the stochastic blockmodel and other exchangeable graph models do not hold. The procedure is tested on simulated and real data, and is shown to outperform two competitors -traditional BIC and the variational Bayes criterion of Latouche et al (2012), in terms of model selection consistency.…”

Community detection in networks: A user guide

“…Other techniques to extract the number of groups have been proposed (Côme and Latouche, 2015;Daudin et al, 2008;Handcock et al, 2007;Latouche et al, 2012;Newman and Reinert, 2016).…”