Comparing Two Clusterings Using Matchings between Clusters of Clusters

Abstract

Clustering is a fundamental problem in data science, yet, the variety of clustering methods and their sensitivity to parameters make clustering hard. To analyze the stability of a given clustering algorithm while varying its parameters, and to compare clusters yielded by different algorithms, several comparison schemes based on matchings, information theory and various indices (Rand, Jaccard) have been developed. We go beyond these by providing a novel class of methods computing meta-clusters within each clustering-a meta-cluster is a group of clusters, together with a matching between these. Let the intersection graph of two clusterings be the edge-weighted bipartite graph in which the nodes represent the clusters, the edges represent the non empty intersection between two clusters, and the weight of an edge is the number of common items. We introduce the so-called D-family-matching problem on intersection graphs, with D the upper-bound on the diameter of the graph induced by the clusters of any meta-cluster. First we prove NP-completeness results and unbounded approximation ratio of simple strategies. Second, we design exact polynomial time dynamic programming algorithms for some classes of graphs (in particular trees). Then, we prove spanning-tree based efficient algorithms for general graphs. Our experiments illustrate the role of D as a scale parameter providing information on the relationship between clusters within a clustering and in-between two clusterings. They also show the advantages of our built-in mapping over classical cluster comparison measures such as the variation of information (VI). Comparer deux clusterings en utilisant des clusters de clustersRésumé :Le clustering est une tâche essentielle en analyse de données, mais la variété des méthodes disponibles rend celle-ci ardue. Diverses stratégies ont été proposées pour analyser la stabilité d'un clustering en fonction des paramètres de l'algorithme l'ayant généré, ou bien comparer des clusterings produits par des algorithmes différents. Nous allons au delà de celles-ci, en proposant une nouvelle classe de méthodes formant des groupes de clusters (meta-clusters) dans chaque clustering, et établissant une correspondance entre ceux-ci.Plus spécifiquement, définissons le graphe intersection de deux clusterings comme le graphe biparti dont les sommets sont les clusters, chaque arête étant pondérée par le nombre de points communs à deux clusters. Nous définissons le D-family-matching problème à partir du graphe intersection, D étant une borne supérieure sur le diamètre du graphe induit par les clusters des metaclusters. Dans un premier temps, nous établissons des résultats de difficulté et d'inaproximabilité. Dans un second temps, nous développons des algorithmes de programmation dynamique pour certaines classes de graphes (arbres en particulier). Enfin, nous concevons des algorithmes efficaces, basés sur des arbres couvrants, pour des graphes généraux.Nos résultats expérimentaux illustrent le rôle de D comme un paramètre d'échelle fournissant de l'i...

Clusters of COVID-19 Indicators in India: Characterization, Correspondence and Change Analysis

“…Several research papers [ 10 , 11 ] have analyzed the stability of a given clustering algorithm while varying its parameters, and to compare clusters yielded by different algorithms, using comparison schemes based on matchings, information theory, and use of various indices (Rand, Jaccard). This was generalized to accommodate many-to-many matchings between clusters, via the D-family-matching on the intersection graph , with D as the upper bound on the diameter of the graph induced by the clusters of any meta-cluster by [ 12 ]. While this problem is NP-complete and hard to approximate, a polynomial time, spanning tree based heuristic was presented.…”

“…In this paper, we are dealing with clusters obtained through two different datasets which are therefore, non-uniform in size and could represent different metrics. We present a novel way of using a specialized edge-weighing formulation in intersection graph and to find the correspondences through maximum-weighted bipartite matching of the graph in section [5.2, 5.3], figure [ 12 , 15 ]. It is noteworthy that our edge-weight metric can be generalized to compute similarity between two sets containing objects at different levels of hierarchies in a hierarchical dataset.…”

User‐guided global explanations for deep image recognition: A user study

“…For this purpose, we applied a cluster matching framework, called D-family matching. 22 It first defines the "intersection graph" G of N i and N j as a bipartite graph where the vertices in the two partite sets correspond to the clusters of N i and N j . Each pair of clusters of N i and N j has an edge with the weight equal to the size of their intersection.…”

MaxMin clustering for historical analogy

“…In addition, studies comparing algorithms or generated clusters have been performed [37]. Cazals et al proposed a framework to analyze the stability of clustering algorithms and compare clusters by introducing meta-clusters [10]. They defined the family-matching problems on an intersection graph.…”