Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization

Abstract

We provide evidence that non-linear dimensionality reduction, clustering and data set parameterization can be solved within one and the same framework. The main idea is to define a system of coordinates with an explicit metric that reflects the connectivity of a given data set and that is robust to noise. Our construction, which is based on a Markov random walk on the data, offers a general scheme of simultaneously reorganizing and subsampling graphs and arbitrarily shaped data sets in high dimensions using intrinsic geometry.We show that clustering in embedding spaces is equivalent to compressing operators. The objective of data partitioning and clustering is to coarse-grain the random walk on the data while at the same time preserving a diffusion operator for the intrinsic geometry or connectivity of the data set up to some accuracy. We show that the quantization distortion in diffusion space bounds the error of compression of the operator, thus giving a rigorous justification for k-means clustering in diffusion space and a precise measure of the performance of general clustering algorithms.

Locally Discriminant Diffusion Projection and Its Application in Speech Emotion Recognition

“…Diffusion Maps is originally proposed in [9][10] to improve learning performance by 'coarse-graining' structure on samples. The method of DM brings a solution in controlling noise data and it can also achieve different scales of a given data set.…”

“…According to the idea of diffusion maps in [9][10], the feature mapping of a certain sample x in the original feature space can be represented as the form of (2). In addition, the diffusion distance of two data points x and z is shown as (3).…”

“…LE (LPP) [6][7], LLE [8], DM (Diffusion Maps) [9][10][11][12], Isomap [13], MFA (LDE) [14][15], SDE [16], MLE [17] etc. In addition, some algorithm frameworks, graph Locally Discriminant Diffusion Projection and Its Application in Speech Emotion Recognition X. Xu, C. Huang, C. Wu, L. Zhao embedding [14] and least-squares [18] frameworks, have been proposed recently, and successfully applied to nonlinear dimensionality reduction and manifold learning.…”

DifFUZZY: a fuzzy clustering algorithm for complex datasets

“…For very small values of γ 2 we have P ∼ I, for which the probabilities of transition between different data points is close to zero, therefore there will not be any diffusion during one time step. The parameter γ 3 ∈ (0, ∞) is the number of time steps the random walk is going to be run or propagated, capturing information of higher order neighbourhood structure (Lafon and Lee [16]). Small values of γ 3 give us a few time steps, whereas large values of γ 3 give us a large number of time steps.…”

Graph-based tools for microscopic cellular image segmentation

“…Moreover, recent data sets analysis and machine learning methods have been developed. They are based on graph Laplacian diffusion processes and have been used to perform data sets classification [5,6], dimensionality reduction [7] or interactive image segmentation based on label diffusion methods [8,9]. If we consider an image as a set of pixels, graph Laplacian classification is difficult to use, due to the great mass of data to analyze.…”