Engineering Parallel Algorithms for Community Detection in Massive Networks

Abstract

The amount of graph-structured data has recently experienced an enormous growth in many applications. To transform such data into useful information, fast analytics algorithms and software tools are necessary. One common graph analytics kernel is disjoint community detection (or graph clustering). Despite extensive research on heuristic solvers for this task, only few parallel codes exist, although parallelism will be necessary to scale to the data volume of real-world applications. We address the deficit in computing capability by a flexible and extensible community detection framework with shared-memory parallelism. Within this framework we design and implement efficient parallel community detection heuristics: A parallel label propagation scheme; the first large-scale parallelization of the well-known Louvain method, as well as an extension of the method adding refinement; and an ensemble scheme combining the above. In extensive experiments driven by the algorithm engineering paradigm, we identify the most successful parameters and combinations of these algorithms. We also compare our implementations with state-of-the-art competitors. The processing rate of our fastest algorithm often reaches 50M edges/second. We recommend the parallel Louvain method and our variant with refinement as both qualitatively strong and fast. Our methods are suitable for massive data sets with billions of edges. 1

Director Networks and Credit Ratings

“…The whole graph with direct and indirect connections (such as between director 2 and director 4) is projected. Using these projected graphs, we compute the four centrality measures using the Stanford Network Analysis Platform (Leskovec and Sosič, 2016) and Networkit (Staudt and Meyerhenke, 2016) software. One could argue that multiple directorships of directors could be a proxy for social capital.…”

Distributed Graph Clustering Using Modularity and Map Equation

“…For modularity, we compare against our own implementation of the sequential Louvain algorithm [5] and the shared-memory parallel PLM [22]. For map equation, we compare against the sequential Infomap [21], the shared-memory parallel RelaxMap [2] and the distributed GossipMap [3] implementations.…”

A scalable community detection algorithm for large graphs using stochastic block models

“…Bhowmick and Srinivasan used a shared memory interface for parallelizing the Louvain method [40] . Staudt and Meyerhenke developed a parallel framework for multiple algorithms, including label propagation and the Louvain method with refinement [13]. Gregori et al devised an parallel algorithm for k-clique community detection [41].…”

“…Many community detection algorithms handle such a problem [8,9,10,3,11,12,13,14]. However, they come along with limitations for large graphs, for example, in handling community heterogeneity [15,16,17].…”