Ensemble clustering for graphs: comparisons and applications

Abstract

We recently proposed a new ensemble clustering algorithm for graphs (ECG) based on the concept of consensus clustering. We validated our approach by replicating a study comparing graph clustering algorithms over benchmark graphs, showing that ECG outperforms the leading algorithms. In this paper, we extend our comparison by considering a wider range of parameters for the benchmark, generating graphs with different properties. We provide new experimental results showing that the ECG algorithm alleviates the well-known resolution limit issue, and that it leads to better stability of the partitions. We also illustrate how the ensemble obtained with ECG can be used to quantify the presence of community structure in the graph, and to zoom in on the sub-graph most closely associated with seed vertices. Finally, we illustrate further applications of ECG by comparing it to previous results for community detection on weighted graphs, and community-aware anomaly detection.Most networks that arise in nature exhibit complex structure [1, 2] with subsets of vertices densely interconnected relative to the rest of the network, which we call communities or clusters. Binary relational data-sets are typically represented as graphs G = (V, E), where vertices v ∈ V represent the entities, and edges e ∈ E represent the relations between pairs of entities. Graph clustering aims at finding a partition of the vertices V = C 1 ∪ . . . ∪ C l into good clusters. This is an ill-posed problem [3], as there is no universal definition of good clusters, leading to a wide variety of graph clustering algorithms [1, 4-10], with different objective functions. In a recent study [11], several state-of-the art algorithms implemented in the igraph [12] package were compared over a wide range of artificial networks generated via the LFR benchmark [13]. We recently introduced a new ensemble clustering algorithm for graphs (ECG), which compared favorably with leading algorithms from that study [14].The ECG algorithm is based on the concept of co-association consensus clustering. It is similar to other consensus clustering algorithms, in particular [15], but differs in two major points: (1) the choice of an algorithm that alleviates the resolution limit issue for the generation step, and (2) the restriction to endpoints of edges for co-occurrences of vertex pairs, which keeps low computational complexity.The rest of the paper is organized as follows. We briefly describe the ECG algorithm in Section 2, where we also recall some results from the previous comparison study. New results are included in the following three sections.In Section 3, we extend our study to a wider variety of graphs by varying the power law exponents of the LFR benchmark. Some of the advantages of ECG are its stability, and its ability to alleviate the well known resolution limit issue. We illustrate those properties in Section 4. We also take a closer look at the edge weights generated by the ECG algorithm, showing that they can be good indicators of the presence (or absence) of comm...

Non-linear Dimensionality Reduction on Extracellular Waveforms Reveals Cell Type Diversity in Premotor Cortex

“…Ensemble clustering for graphs (ECG) 128,129 was used to validate the clusters found in Fig. 2A (see Fig.…”

“…That is, results from successive runs on the same data can show considerable variation on some datasets 129 . To test whether these eight clusters consistently contained the same constituent data points run-to-run, we used ensemble clustering for graphs (ECG) 128,129 . ECG generates k randomized level-1 (one round of Louvain clustering, Fig.…”

“…Stability: Louvain clustering is sometimes unstable. That is, results from successive runs on the same data can show considerable variation on some datasets (Poulin and Théberge, 2019).…”

An unsupervised framework for comparing graph embeddings

“…An important part of our algorithm consists of building a reasonable partition of the vertices, over which we build the representative vectors (1) and (2). The ECG algorithm was shown to generally yield good and stable clusters [25], and we used it throughout. We re-ran all embeddings on the College Football graph using respectively the Louvain [26] and InfoMap [27] clustering algorithms, and in each case, we produced a ranking of the embeddings with respect to our method.…”

A multi-objective adaptive evolutionary algorithm to extract communities in networks

“…Many approaches have been proposed to employ multi-objective techniques for data clustering. Most of these approaches cluster objects in metric spaces [5,6,7], though a method for partitioning graphs has been presented in [8] and a graph clustering algorithm of web user sessions is described in [9].…”