Efficient Diffusion on Region Manifolds: Recovering Small Objects with Compact CNN Representations

Abstract

Query expansion is a popular method to improve the quality of image retrieval with both conventional and CNN representations. It has been so far limited to global image similarity. This work focuses on diffusion, a mechanism that captures the image manifold in the feature space. The diffusion is carried out on descriptors of overlapping image regions rather than on a global image descriptor like in previous approaches. An efficient off-line stage allows optional reduction in the number of stored regions. In the on-line stage, the proposed handling of unseen queries in the indexing stage removes additional computation to adjust the precomputed data. We perform diffusion through a sparse linear system solver, yielding practical query times well below one second.Experimentally, we observe a significant boost in performance of image retrieval with compact CNN descriptors on standard benchmarks, especially when the query object covers only a small part of the image. Small objects have been a common failure case of CNN-based retrieval.

SIFT Meets CNN: A Decade Survey of Instance Retrieval

“…Other effective methods include burstiness handling [77] (discussed in Section 3.4.3), considering the different inlier ratios between the query and target objects [121], etc. In the second type of methods, effective region proposals [122] or multi-scale image patches [123] can be used as object region candidates. In [123], a recent state-of-the-art method, a regional diffusion mechanism based on neighborhood graphs is proposed to further improve the recall of small objects.…”

“…In the second type of methods, effective region proposals [122] or multi-scale image patches [123] can be used as object region candidates. In [123], a recent state-of-the-art method, a regional diffusion mechanism based on neighborhood graphs is proposed to further improve the recall of small objects.…”

Local Features and Visual Words Emerge in Activations

“…We observed that the step ∆ [27]. MAC: max-pooling [40]; GeM: generalized-mean pooling [29]; D: diffusion [12]. All results with supervised whitening [21].…”

Genetic Algorithms for the Optimization of Diffusion Parameters in Content-Based Image Retrieval

“…Recently, Iscen et al [21] and Yang et al [39] outperformed the state of the art on several public image retrieval datasets through the application of the diffusion process to R-MAC descriptors [15]. The reason for the success of diffusion for retrieval [40] is that it permits to find more neighbors that are close to the query using the manifold representation, than using the Euclidean one.…”