Two-stage Discriminative Re-ranking for Large-scale Landmark Retrieval

Abstract

We propose an efficient pipeline for large-scale landmark image retrieval that addresses the diversity of the dataset through two-stage discriminative re-ranking. Our approach is based on embedding the images in a feature-space using a convolutional neural network trained with a cosine softmax loss. Due to the variance of the images, which include extreme viewpoint changes such as having to retrieve images of the exterior of a landmark from images of the interior, this is very challenging for approaches based exclusively on visual similarity. Our proposed re-ranking approach improves the results in two steps: in the sort-step, k-nearest neighbor search with soft-voting to sort the retrieved results based on their label similarity to the query images, and in the insert-step, we add additional samples from the dataset that were not retrieved by image-similarity. This approach allows overcoming the low visual diversity in retrieved images. In-depth experimental results show that the proposed approach significantly outperforms existing approaches on the challenging Google Landmarks Datasets. Using our methods, we achieved 1st place in the Google Landmark Retrieval 2019 challenge and 3rd place in the Google Landmark Recognition 2019 challenge on Kaggle. Our code is publicly available here: https://github.com/lyakaap/ Landmark2019-1st-and-3rd-Place-Solution

Google Landmarks Dataset v2 -- A Large-Scale Benchmark for Instance-Level Recognition and Retrieval

“…We also compare with models trained on the Landmarks-full Table 5: Baseline results (% mAP@100) for the GLDv2 retrieval task. The bottom three results were reported in [63]. .…”

“…The Google Landmarks Dataset v2 training set presents a realistic crowdsourced setting with diverse types of images for each landmark: e.g., for a specific museum there may be outdoor images showing the building facade, but also indoor images of paintings and sculptures that are on display. Such diversity within a class may pose challenges to the training process, so we consider the pre-processing steps proposed in [63] in order to make each class more visually coherent. Within each class, each image is queried against all others by global descriptor similarity, followed by geometric verification of the top-100 most similar images using local features.…”

All the attention you need: Global-local, spatial-channel attention for image retrieval

“…All models are pre-trained on ImageNet [39] and implemented in PyTorch [31]. For fair comparisons, we set a training environment similar to the those of compared studies [56,53,28,35]. We employ ResNet101 [18] as a backbone model.…”

“…We adopt the batch sampling of Yokoo et al [56] where mini-batch samples with similar aspect ratios are resized to a particular size. Here, we use a batch size of 64.…”

“…Fixed-size vs. group-size sampling Numerous studies have proposed methods for constructing batches according to image size for efficient training. For instance, Gordo et al [16], DELF [29], and Yokoo et al [56] employed different image sizes per batch for training instead of a single fixed size. We adopt the method of Yokoo et al, which constructs a batch with images of similar aspect ratio, so that the images can be resized to a size with an aspect ratio that is similar to their own.…”