Generic Sample Splitting for Refined Community Recovery in Degree Corrected Stochastic Block Models

Abstract

We propose and analyze a generic method for community recovery in stochastic block models and degree corrected block models. This approach can exactly recover the hidden communities with high probability when the expected node degrees are of order log n or higher. Starting from a roughly correct community partition given by some conventional community recovery algorithm, this method refines the partition in a cross clustering step. Our results simplify and extend some of the previous work on exact community recovery, discovering the key role played by sample splitting. The proposed method is simple and can be implemented with many practical community recovery algorithms.

Consistency of spectral hypergraph partitioning under planted partition model

“…It also is known that one can iteratively refine the solution of a spectral algorithm to exactly recover the partitions (Vu, 2014;Lei and Zhu, 2014). Such an approach usually constructs an embedding of the nodes based on the adjacency matrix.…”

“…This not the best known error rate as exact recovery of the partitions are known to be possible using other approaches (Amini and Levina, 2014). Recent results (Vu, 2014;Lei and Zhu, 2014;Gao et al, 2015) show that an additional refinement process can improve the partitioning of spectral clustering to exactly recover the partitions, thereby achieving strong consistency. The condition on minimum node degree can be also relaxed by considering alternative spectral techniques (Krzakala et al, 2013;Le et al, 2015), and these algorithms can detect partitions in sparse random graphs that are close to the algorithmic barrier for community detection (Decelle et al, 2011).…”

Model assisted variable clustering: Minimax-optimal recovery and algorithms

“…The problem of variable clustering that we consider in this work is fundamentally different from that of variable clustering from network data. The latter, especially in the context of the Stochastic Block Model (SBM), has received a large amount of attention over the past years, for instance [21,28,16,29,2,33,27]. The most important difference stems from the nature of the data: the data analyzed via the SBM is a p×p binary matrix A, called the adjacency matrix, with entries assumed to have been generated as independent Bernoulli random variables; its expected value is assumed to have a block structure.…”

A goodness-of-fit test for stochastic block models

“…The definition of consistent community recovery can be satisfied by several methods. For example, in the case of fixed finite K (n) = K and B (n) = B, the profile likelihood method (Bickel and Chen, 2009) is consistent for all (g (n) : n ≥ 1) satisfying (A1) and all B ∈ B K ; the spectral clustering method can be made consistent, with slight modification, for all (g (n) : n ≥ 1) satisfying (A1) and B ∈ B K with full rank (McSherry, 2001, Vu, 2014, Lei and Zhu, 2014. In the case of slowly growing K (n) , consistent community recovery can be achieved in some special cases such as the planted partition model (Chaudhuri, Chung andTsiatas, 2012, Amini andLevina, 2014).…”