The Principal Components Analysis of a Graph, and Its Relationships to Spectral Clustering

Abstract

Abstract. This work presents a novel procedure for computing (1) distances between nodes of a weighted, undirected, graph, called the Euclidean Commute Time Distance (ECTD), and (2) a subspace projection of the nodes of the graph that preserves as much variance as possible, in terms of the ECTD -a principal components analysis of the graph. It is based on a Markov-chain model of random walk through the graph. The model assigns transition probabilities to the links between nodes, so that a random walker can jump from node to node. A quantity, called the average commute time, computes the average time taken by a random walker for reaching node j for the first time when starting from node i, and coming back to node i. The square root of this quantity, the ECTD, is a distance measure between any two nodes, and has the nice property of decreasing when the number of paths connecting two nodes increases and when the "length" of any path decreases. The ECTD can be computed from the pseudoinverse of the Laplacian matrix of the graph, which is a kernel. We finally define the Principal Components Analysis (PCA) of a graph as the subspace projection that preserves as much variance as possible, in terms of the ECTD. This graph PCA has some interesting links with spectral graph theory, in particular spectral clustering.

Fast Katz and Commuters: Efficient Estimation of Social Relatedness in Large Networks

“…A straightforward means of computing them is to solve the linear system (I − αA)x = e j and (L + 1 n ee T )y = e i − e j . Then [19]). Solving these linear systems is an effective method to compute only the pairwise scores.…”

“…Other uses of Katz scores and commute time are anomalous link detection [18], recommendation [20], and clustering [19].…”

Fast Gaussian Pairwise Constrained Spectral Clustering

“…Spectral clustering can also be understood in terms of the spectral embedding of the graph, the change of representation of the data represented by nodes. Indeed, the spectral decomposition of the graph Laplacian gives a projection of the data in a new feature space in which Euclidean distance corresponds to a similarity given by the graph (e.g., the resistance distance [15,27]). …”

Diffusion Maps - a Probabilistic Interpretation for Spectral Embedding and Clustering Algorithms

“…In recent years various spectral methods to perform these tasks, based on the eigenvectors of adjacency matrices of graphs on the data have been developed, see for example [1][2][3][4][5][6][7][8][9][10][11][12] and references therein. In the simplest version, known as the normalized graph Laplacian, given n data points {x i } n i=1 where each x i ∈ R p (or some other normed vector space), we define a pairwise similarity matrix between points, for example using a Gaussian kernel with width σ 2 ,…”

“…A different theoretical analysis of the eigenvectors of the matrix M , based on the fact that M is a stochastic matrix representing a random walk on the graph was described by Meilǎ and Shi [14], who considered the case of piecewise constant eigenvectors for specific lumpable matrix structures. Additional notable works that considered the random walk aspects of spectral clustering are [10,15], where the authors suggest clustering based on the average commute time between points, [16,17] which considered the relaxation process of this random walk, and [18,19] which suggested random walk based agglomerative clustering algorithms.…”