Affinity Learning with Diffusion on Tensor Product Graph

Abstract

Abstract-In many applications, we are given a finite set of data points sampled from a data manifold and represented as a graph with edge weights determined by pairwise similarities of the samples. Often the pairwise similarities (which are also called affinities) are unreliable due to noise or due to intrinsic difficulties in estimating similarity values of the samples. As observed in several recent approaches, more reliable similarities can be obtained if the original similarities are diffused in the context of other data points, where the context of each point is a set of points most similar to it. Compared to the existing methods, our approach differs in two main aspects. First, instead of diffusing the similarity information on the original graph, we propose to utilize the tensor product graph (TPG) obtained by the tensor product of the original graph with itself. Since TPG takes into account higher order information, it is not a surprise that we obtain more reliable similarities. However, it comes at the price of higher order computational complexity and storage requirement. The key contribution of the proposed approach is that the information propagation on TPG can be computed with the same computational complexity and the same amount of storage as the propagation on the original graph. We prove that a graph diffusion process on TPG is equivalent to a novel iterative algorithm on the original graph, which is guaranteed to converge. After its convergence we obtain new edge weights that can be interpreted as new, learned affinities. We stress that the affinities are learned in an unsupervised setting. We illustrate the benefits of the proposed approach for data manifolds composed of shapes, images, and image patches on two very different tasks of image retrieval and image segmentation. With learned affinities, we achieve the bull's eye retrieval score of 99.99 percent on the MPEG-7 shape dataset, which is much higher than the state-of-the-art algorithms. When the data points are image patches, the NCut with the learned affinities not only significantly outperforms the NCut with the original affinities, but it also outperforms state-of-the-art image segmentation methods.

A Comprehensive Insight towards Research Direction in Information Propagation

“…Various forms of user's behavior, structures pertaining to typical propagation, etc were investigated in this study. Yang et al [76] have presented a unique discussion of the unreliability problems associated with pairwise similarities. The authors have solved this problem using tensor product graph in order to regain the better amount of reliability using superiorly compressed video dataset.…”

Consensus of k-NNs for Robust Neighborhood Selection on Graph-Based Manifolds

“…Once P is calculated, we can now perform diffusion using any of the graph diffusion procedures (Ex: LCDP [16], or TPG [18]). In our experiments, we primarily use TPG diffusion as it takes into account higher-order similarity relations for the same space and time complexity as classical diffusion on the original graph.…”

“…Many recent papers make use of such contextual information to learn new affinity scores between pairs of data points [1,4,15,16,18]. The similarity information is usually propagated as a diffusion process on the graph.…”

“…The similarity information is usually propagated as a diffusion process on the graph. Yang et al [16] perform diffusion on a locally constrained sparse graph, while in [18], the diffusion process is performed on a tensor product graph, thus allowing the capture of higher order information. The diffusion process is susceptible to noise [3], and hence the affinity propagation is not performed on a fully connected graph.…”

“…The diffusion process is susceptible to noise [3], and hence the affinity propagation is not performed on a fully connected graph. Both [16] and [18] follow different styles of graph sparsification (k-nearest neighbors and dominant neighbors, respectively). The selection of a proper neighborhood is critical for diffusion to work.…”