Data spectroscopy: Eigenspaces of convolution operators and clustering

Abstract

This paper focuses on obtaining clustering information about a distribution from its i.i.d. samples. We develop theoretical results to understand and use clustering information contained in the eigenvectors of data adjacency matrices based on a radial kernel function with a sufficiently fast tail decay. In particular, we provide population analyses to gain insights into which eigenvectors should be used and when the clustering information for the distribution can be recovered from the sample. We learn that a fixed number of top eigenvectors might at the same time contain redundant clustering information and miss relevant clustering information. We use this insight to design the data spectroscopic clustering (DaSpec) algorithm that utilizes properly selected eigenvectors to determine the number of clusters automatically and to group the data accordingly. Our findings extend the intuitions underlying existing spectral techniques such as spectral clustering and Kernel Principal Components Analysis, and provide new understanding into their usability and modes of failure. Simulation studies and experiments on real-world data are conducted to show the potential of our algorithm. In particular, DaSpec is found to handle unbalanced groups and recover clusters of different shapes better than the competing methods.Comment: Published in at http://dx.doi.org/10.1214/09-AOS700 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Noise resistant graph ranking for improved web image search

“…[ 19] pointed out that when the data points have formed clusters, each high density region implicitly corresponds to some low-frequency (smooth) eigenvector which takes rela-tively large absolute values for points in the region (cluster) and whose values are close to zero elsewhere. Note that we exclude the first eigenvector u 1 because it is nearly constant and does not form clusters.…”

OLYMPUS: An automated hybrid clustering method in time series gene expression. Case study: Host response after Influenza A (H1N1) infection

“…This procedure gives the opportunity to pre-design (and hence know beforehand) the structure of the data that the clustering procedure aims to recover. At first level, we compared the ability of OLYMPUS in determining the correct cluster number against other known unsupervised clustering algorithms such as Density-Based Spatial Clustering of Applications with Noise (DBSCAN) [32], Data Spectroscopic (DaSpec) [33] and Gaussian Mixture Model-BIC, as proposed in [34], as well as with Evolutionary kprototype (EKP) [35], an unsupervised hybrid method that integrates evolutionary optimization with clustering categories. At second level, we tested the accuracy of OLYMPUS against the original FSTS and other hybrid approaches such as the evolutionary fuzzy algorithm of Anand et al [18] -called hereafter Anand -and the Improved Differential Evolutionary Fuzzy Clustering (IDEFC) [19].…”

Spectral Clustering Based on k-Nearest Neighbor Graph

“…Some authors propose a global value of σ for the whole data set e.g. [12] and [14] while the others suggest using a local parameter e.g. [18].…”

“…Another open issue of key importance in spectral clustering is that of choosing a proper number of groups. Usually this number is a user defined parameter [12], but sometimes it is estimated -with varying success rate [14] -in a heuristically motivated way. In this paper we present a spectral clustering algorithm Speclus that can simultaneously address both of the above mentioned challenges for a variety of data sets.…”

“…In this paper we present a spectral clustering algorithm Speclus that can simultaneously address both of the above mentioned challenges for a variety of data sets. It adopts the idea derived by Shi et al from their analysis of the relationship between a probability distribution and spectrum of the corresponding distribution-dependent convolution operator, [14]. Their DaSpec (i.e.…”