Fine-Tuning CNN Image Retrieval with No Human Annotation

Abstract

Image descriptors based on activations of Convolutional Neural Networks (CNNs) have become dominant in image retrieval due to their discriminative power, compactness of representation, and search efficiency. Training of CNNs, either from scratch or fine-tuning, requires a large amount of annotated data, where a high quality of annotation is often crucial. In this work, we propose to fine-tune CNNs for image retrieval on a large collection of unordered images in a fully automated manner. Reconstructed 3D models obtained by the state-of-the-art retrieval and structure-from-motion methods guide the selection of the training data. We show that both hard-positive and hard-negative examples, selected by exploiting the geometry and the camera positions available from the 3D models, enhance the performance of particular-object retrieval. CNN descriptor whitening discriminatively learned from the same training data outperforms commonly used PCA whitening. We propose a novel trainable Generalized-Mean (GeM) pooling layer that generalizes max and average pooling and show that it boosts retrieval performance. Applying the proposed method to the VGG network achieves state-of-the-art performance on the standard benchmarks: Oxford Buildings, Paris, and Holidays datasets.

Challenging Deep Image Descriptors for Retrieval in Heterogeneous Iconographic Collections

“…They also propose the regional version RMAC, by sampling windows at different scales and describing them separately. Radenović et al [19] generalize the preceding approaches with a generalized mean pooling (GeM) including a learnable parameter.…”

Weakly-supervised object detection via mining pseudo ground truth bounding-boxes

“…expectation maximization (EM) [33] , curriculum learning [34] , self-paced learning [35] , etc. ) are widely used in the weakly-supervised tasks [9,[36][37][38][39][40][41] . For example, [36] adopts the expectation maximization (EM) algorithm to dynamically predict semantic foreground and background pixels by using an alternative training procedure.…”

Co-occurrence of deep convolutional features for image search

“…In Table 7, we present the results in ROxford and RParis datasets of stateof-the-art methods which uses VGG as feature extractor. In the pre-trained single pass category we improve the state-of-the-art performance with ChCO-SC T + SpT D based in the linear aggregation of co-occurrences against well known image retrieval methods like crow [7], SPoC [5], MAC and R-MAC [6] and GeM [31]. Moreover, with bilinear pooling we can obtain a final vector representation with higher dimensions than the number of channels of the last VGG layer.…”

“…In this section is evaluated the co-occurrence representation after the cooccurrence filter training process in a CoOcNET pipeline. The evaluation is performed similar to [31], with its same whitening procedure, alpha query expansion method αQE, and testing also each query in multiscale, ms, (1, 1…”

“…The simplest method, called average query expansion AQE , is the average of the n nearest descriptors after the first query. The second method, denoted alpha query expansion αQE, presented by Radenovic et al [31], which is a weighted aggregation of the nearest descriptors.…”