Distribution-free model for community detection

Abstract

Community detection for unweighted networks has been widely studied in network analysis, but the case of weighted networks remains a challenge. This paper proposes a general Distribution-Free Model (DFM) for weighted networks in which nodes are partitioned into different communities. DFM can be seen as a generalization of the famous stochastic blockmodels from unweighted networks to weighted networks. DFM does not require prior knowledge of a specific distribution for elements of the adjacency matrix but only the expected value. In particular, signed networks with latent community structures can be modeled by DFM. We build a theoretical guarantee to show that a simple spectral clustering algorithm stably yields consistent community detection under DFM. We also propose a four-step data generation process to generate adjacency matrices with missing edges by combining DFM, noise matrix, and a model for unweighted networks. Using experiments with simulated and real datasets, we show that some benchmark algorithms can successfully recover community membership for weighted networks generated by the proposed data generation process.

Estimating the Number of Communities in Weighted Networks

“…DCDFM includes several previous models. For example, when for all , DCDFM reduces to the distribution-free model [ 55 ]; when is Bernoulli distribution and P ’s elements are non-negative, DCDFM reduces to the classical degree-corrected stochastic blockmodels [ 57 ]; when is Bernoulli distribution, all elements of are the same, and P ’s elements are non-negative, DCDFM reduces to the popular stochastic blockmodels [ 24 ], i.e., SBM, DCSBM, and DFM are sub-models of DCDFM. As analyzed in [ 56 ], can be any distribution as long as A ’s expectation matrix is under distribution .…”

“…A significant drawback of the above SBM-based and DCSBM-based methods is that they ignore the impact of edge weights which are common in network data and could help us to understand the community structure of a network better [ 16 ]. Recently, community detection in weighted networks has become a hot topic and many statistical models have been developed to fit weighted networks, such as the weighted stochastic blockmodels (WSBM) proposed in [ 48 , 49 , 50 , 51 , 52 , 53 , 54 ], the distribution-free model (DFM) of [ 55 ], and the degree-corrected distribution-free model (DCDFM) introduced in [ 56 ]. Among these models, DFM and its extension DCDFM stand out as they allow edge weights to follow any distribution as long as the expected adjacency matrix follows a block structure related to community partition.…”