Parallel heuristics for scalable community detection

Abstract

a b s t r a c tCommunity detection has become a fundamental operation in numerous graph-theoretic applications. It is used to reveal natural divisions that exist within real world networks without imposing prior size or cardinality constraints on the set of communities. Despite its potential for application, there is only limited support for community detection on large-scale parallel computers, largely owing to the irregular and inherently sequential nature of the underlying heuristics. In this paper, we present parallelization heuristics for fast community detection using the Louvain method as the serial template. The Louvain method is a multi-phase, iterative heuristic for modularity optimization. Originally developed by Blondel et al. (2008), the method has become increasingly popular owing to its ability to detect high modularity community partitions in a fast and memoryefficient manner. However, the method is also inherently sequential, thereby limiting its scalability. Here, we observe certain key properties of this method that present challenges for its parallelization, and consequently propose heuristics that are designed to break the sequential barrier. For evaluation purposes, we implemented our heuristics using OpenMP multithreading, and tested them over real world graphs derived from multiple application domains (e.g., internet, citation, biological). Compared to the serial Louvain implementation, our parallel implementation is able to produce community outputs with a higher modularity for most of the inputs tested, in comparable number or fewer iterations, while providing absolute speedups of up to 16Â using 32 threads.

A Systematic Approach to Bacterial Phylogeny Using Order Level Sampling and Identification of HGT Using Network Science

“…The total number of protein sequences in each of the datasets varied with the largest containing approximately 1.2 M. The pClust pipeline was employed to group the protein sequences into homologous clusters. pClust is an open source software package that uses the highly efficient software package Parasail [11] for sequence alignment and Grappolo [12] for clustering to achieve fast and efficient clustering of protein sequences. Parasail employs a filtering approach to reduce the number of pair-wise sequence alignments to approximately 0.1% of the original number and allows the user to choose between local, semi-global, and global alignment.…”

Hierarchical ktjet clustering for parallel architectures

“…Because of this the modularity is a monotonically increasing. By reaching convergence in a given phase, the system is getting reduced, by assigning a single "meta-vertex" [24] in the place of all the nodes belonging to the same community. The new nodes can have loops and the weight of these new edges will be the sum of the weights of all the edges that are connecting the inner nodes of the group.…”

“…The challenges to parallelize the Louvain method were explored in [24]. To solve those issues multiple heuristics were introduced, that can be used to leverage the performance of the parallel systems in a basically sequential algorithm.…”

“…In the parallel algorithm, if at any given iteration vertex i which is in a community by itself (C(i) = i, singlet community [24]), in hope for modularity gain might decide to move into another community, that holds only vertex j. This transition will only be applied if l(C(j) < l(C(i))).…”

Community Detection Using Modularity

“…To reduce the computational complexity and to improve the modularity measure several algorithms are proposed [2][3][4][5][6][7][8][9][10][11][12]. These algorithms are measured the modularity by substractithe expected number of community edges from the actual number of intra-community edges [3], which mathematically given as:…”