sARI: a soft agreement measure for class partitions incorporating assignment probabilities

Abstract

Agreement indices are commonly used to summarize the performance of both classification and clustering methods. The easy interpretation/intuition and desirable properties that result from the Rand and adjusted Rand indices, has led to their popularity over other available indices. While more algorithmic clustering approaches like k-means and hierarchical clustering produce hard partition assignments (assigning observations to a single cluster), other techniques like model-based clustering include information about the certainty of allocation of objects through class membership probabilities (soft partitions). To assess performance using traditional indices, e.g. the adjusted Rand index (ARI), the soft partition is mapped to a hard set of assignments, which commonly overstates the certainty of correct assignments. This paper proposes an extension of the ARI, the soft adjusted Rand index (sARI), with similar intuition and interpretation but also incorporating information from one or two soft partitions. It can be used in conjunction with the ARI, comparing the similarities of hard to soft, or soft to soft, partitions to the similarities of the mapped hard partitions. Simulation study results support the intuition that, in general, mapping to hard partitions tends to increase the measure of similarity between partitions. In applications, the sARI more accurately reflects the cluster boundary overlap commonly seen in real data. Keywords adjusted Rand index • model-based clustering • mixture models • soft partition • posterior probabilities • class membership probabilities Mathematics Subject Classification (2000) 62H30 • 91C20 • 62H86 Address(es) of author(s) should be given

Semiparametric finite mixture of regression models with Bayesian P-splines

“…Regarding the clustering performance, a comparison between true allocations and inferred ones is made in terms of Adjusted Rand Index (ARI) (Hubert and Arabie 1985) and soft ARI (sARI) (Flynt et al 2019). For each method and each value of G, 4000 MCMC draws are simulated after a burn-in of as many draws.…”

Flexible Bayesian modelling of concomitant covariate effects in mixture models

“…, n} are the grid points taken evenly in the range of each covariate. Regarding the clustering performance, a comparison between the true allocations and the estimated ones is made in terms of both Adjusted Rand Index (ARI) and soft ARI (Flynt et al, 2019). While the ARI measures the similarity between two data clusterings (hard partitions), the sARI can incorporate the posterior allocation distributions (soft partitions).…”