On Potts Model Clustering, KernelK-Means and Density Estimation

Abstract

Many clustering methods, such as K -means, kernel K -means, and MNcut clustering, follow the same recipe: (i) choose a measure of similarity between observations; (ii) define a figure of merit assigning a large value to partitions of the data that put similar observations in the same cluster; and (iii) optimize this figure of merit over partitions. Potts model clustering represents an interesting variation on this recipe. Blatt, Wiseman, and Domany defined a new figure of merit for partitions that is formally similar to the Hamiltonian of the Potts model for ferromagnetism, extensively studied in statistical physics. For each temperature T , the Hamiltonian defines a distribution assigning a probability to each possible configuration of the physical system or, in the language of clustering, to each partition. Instead of searching for a single partition optimizing the Hamiltonian, they sampled a large number of partitions from this distribution for a range of temperatures. They proposed a heuristic for choosing an appropriate temperature and from the sample of partitions associated with this chosen temperature, they then derived what we call a consensus clustering: two observations are put in the same consensus cluster if they belong to the same cluster in the majority of the random partitions. In a sense, the consensus clustering is an "average" of plausible configurations, and we would expect it to be more stable (over different samples) than the configuration optimizing the Hamiltonian.The goal of this article is to contribute to the understanding of Potts model clustering and to propose extensions and improvements: (1) We show that the Hamiltonian used in Potts model clustering is closely related to the kernel K -means and MNCut criteria. (2) We propose a modification of the Hamiltonian penalizing unequal cluster sizes and show that it can be interpreted as a weighted version of the kernel K -means criterion. (3) We introduce a new version of the Wolff algorithm to simulate configurations from the distribution defined by the penalized Hamiltonian, leading to penalized Potts model clustering. (4) We note a link between kernel based clustering methods and nonparametric density estimation and exploit it to automatically determine locally adaptive kernel bandwidths. (5) We propose a new simple rule for selecting a good temperature T .As an illustration we apply Potts model clustering to gene expression data and compare our results to those obtained by model based clustering and a nonparametric dendrogram sharpening method.

EvoluCode: Evolutionary Barcodes as a Unifying Framework for Multilevel Evolutionary Data

“…The Potts clustering approach, also known as super-paramagnetic clustering, is based on the physical behavior of an inhomogeneous ferromagnet 37. No assumptions are made about the underlying distribution of the data.…”

The Conditional-Potts Clustering Model

“…MAP + PMC and iPrior + PMC stand for the procedures with clustering evidence drawn from the MAP (σ M , T M ) and the datadriven prior maximizer (σ p , T p these scores, the reader may get an idea of how difficult it is to cluster some datasets into the groups selected by some experts (see, e.g., the Yeast cycle data below). The artificial datasets were (a) a 5-clump-3-arc dataset (Murua, Stanberry, and Stuetzle 2008) whose clusters present high variation in shape and distribution and are not very well separated; (b) a three-ring version of the Bull's eye data (Blatt, Domany, and Wiseman 1997), which are a real challenge for most clustering methods; and (c) a 50-Gaussian mixture dataset whose differences in cluster volume may produce difficulties when choosing the appropriate temperature-bandwidth parameters. The data are plotted in Figure 3.…”

“…Its impact has reached the medical (Stanberry, Murua, and Cordes 2008), bioinformatics (Getz et al 2000;Einav et al 2005), and the computer science and machine learning communities as well (Domany et al 1999;Quiroga, Nadasdy, and Ben-Shaul 2004). It also has been mentioned in the statistical literature, but as Potts model clustering (Murua, Stanberry, and Stuetzle 2008), where its link with other kernel-based methods and nonparametric density estimation was presented. A similar, simpler model has also been used as a probabilistic framework for K-nearest-neighbor classification (Cucala et al 2009).…”

“…It is usually set to the mean of the data point similarities Wiseman 1996, 1997). However, Murua, Stanberry, and Stuetzle (2008) showed that treating it as variable may give better clustering results. Although they suggest the use of a data-driven adaptive bandwidth, we believe that its incorporation as a parameter of the model is more appropriate.…”

“…This procedure is heavily based on a data-driven estimate of a very informative prior, which is derived from random graph theory and the connection between kernel-based methods and kernel density estimation (Murua, Stanberry, and Stuetzle 2008). We refer to this latter procedure as the informed conditional-Potts clustering.…”