Diffusion Maps - a Probabilistic Interpretation for Spectral Embedding and Clustering Algorithms

Abstract

Summary. Spectral embedding and spectral clustering are common methods for non-linear dimensionality reduction and clustering of complex high dimensional datasets. In this paper we provide a diffusion based probabilistic analysis of algorithms that use the normalized graph Laplacian. Given the pairwise adjacency matrix of all points in a dataset, we define a random walk on the graph of points and a diffusion distance between any two points. We show that the diffusion distance is equal to the Euclidean distance in the embedded space with all eigenvectors of the normalized graph Laplacian. This identity shows that characteristic relaxation times and processes of the random walk on the graph are the key concept that governs the properties of these spectral clustering and spectral embedding algorithms. Specifically, for spectral clustering to succeed, a necessary condition is that the mean exit times from each cluster need to be significantly larger than the largest (slowest) of all relaxation times inside all of the individual clusters. For complex, multiscale data, this condition may not hold and multiscale methods need to be developed to handle such situations.

Research literature clustering using diffusion maps

“…Thus, the high-dimensional data points become embedded in a lower-dimensional space. The dimensionality reduction yields a space where the Euclidean distance corresponds to the diffusion distance in the original space (Coifman & Lafon, 2006;Nadler et al, 2008).…”

“…n is used to normalize the W matrix: P = D −1 W. This matrix represents the transition probabilities between the data points. The conjugate matrixP = D This normalized graph Laplacian (Chung, 1997) preserves the eigenvalues (Nadler et al, 2008). Singular value decomposition (SVD)P = UΛU * finds the eigenvalues Λ = diag([λ 1 , λ 2 , .…”

“…The eigenvalues for P are the same as forP. The eigenvectors for P are found with Nadler et al, 2008). The low-dimensional coordinates Ψ are created using Ψ = VΛ.…”

A variational approach to the consistency of spectral clustering

“…For example, the spectrum of N rw is shown to be connected to the Ncut problem, whereas the spectrum of L is connected to RatioCut (see [47]). A probabilistic interpretation of the spectrum of N rw may be found in [28]. In addition, connections between normalized graph Laplacians, data parametrization and dimensionality reduction via diffusion maps are developed in [25].…”

Anomaly Detection from Network Logs Using Diffusion Maps

“…The matrix describes the distances between the points. This study uses the common Gaussian kernel with Euclidean distance measure, as in equation 1 [6,16].…”