A generalized Bayes framework for probabilistic clustering

Abstract

Loss-based clustering methods, such as k-means and its variants, are standard tools for finding groups in data. However, the lack of quantification of uncertainty in the estimated clusters is a disadvantage. Model-based clustering based on mixture models provides an alternative, but such methods face computational problems and large sensitivity to the choice of kernel. This article proposes a generalized Bayes framework that bridges between these paradigms through the use of Gibbs posteriors. In conducting Bayesian updating, the loglikelihood is replaced by a loss function for clustering, leading to a rich family of clustering methods. The Gibbs posterior represents a coherent updating of Bayesian beliefs without needing to specify a likelihood for the data, and can be used for characterizing uncertainty in clustering. We consider losses based on Bregman divergence and pairwise similarities, and develop efficient deterministic algorithms for point estimation along with sampling algorithms for uncertainty quantification. Several existing clustering algorithms, including k-means, can be interpreted as generalized Bayes estimators under our framework, and hence we provide a method of uncertainty quantification for these approaches; for example, allowing calculation of the probability a data point is well clustered.

An enriched mixture model for functional clustering

“…In those situations, even the definition of a “cluster” becomes less clear, so the choice of $$$ {H}_{\ell } $$$ becomes problematic as well. We adhere to the philosophical viewpoint detailed in Rigon et al, 42 which is summarized in the following. We argue that it is often appropriate to treat the number of clusters as the resolution at which one wants to summarize the data rather than an unknown quantity one wishes to estimate.…”

Bayesian cluster analysis

“…Partition-based clustering algorithms aim to minimize a specific loss function and are widely adopted but lack any quantification of uncertainty in the clustering solution. To address this and bridge the gap between partition-based and model-based approaches, the recent work of Rigon et al [ 79 ] employs a generalized Bayesian framework through the use of Gibbs posteriors [ 80 ]. Specifically, the generalized posterior is defined as where the loss function has the form with denoting the observations belonging to the th cluster and quantifying the discrepancy of from the th cluster.…”

“…Specifically, the generalized posterior is defined as where the loss function has the form with denoting the observations belonging to the th cluster and quantifying the discrepancy of from the th cluster. A simple example is the -means loss which sets (additional examples can be found in Rigon et al [ 79 ]). Fixing the number of clusters , the prior is chosen to be uniform over the set of partitions with clusters.…”

“…Partition-based clustering algorithms aim to minimize a specific loss function and are widely adopted but lack any quantification of uncertainty in the clustering solution. To address this and bridge the gap between partition-based and model-based approaches, the recent work of Rigon et al [79] employs a generalized Bayesian framework through the use of Gibbs posteriors [80]. Specifically, the generalized posterior is defined as πfalse(bold-italicz|bold-italicyfalse)∝exp⁡false(−λℓfalse(bold-italicz,bold-italicyfalse)false)πfalse(bold-italiczfalse),where the loss function has the form ℓfalse(bold-italicz,bold-italicyfalse)=∑j=1k∑i:zi=jDfalse(yi,yjfalse) with yj=falsefalse{yi:i=zjfalsefalse} denoting the observations belonging to the jth cluster and Dfalse(yi,yjfalse)≥0 quantifying the discrepancy of yi from the jth cluster.…”

“…(additional examples can be found in Rigon et al [79]). Fixing the number of clusters k, the prior π (z) is chosen to be uniform over the set of partitions with k clusters.…”