Centralized Ranking Loss with Weakly Supervised Localization for Fine-Grained Object Retrieval

Abstract

Fine-grained object retrieval has attracted extensive research focus recently. Its state-of-the-art schemes are typically based upon convolutional neural network (CNN) features. Despite the extensive progress, two issues remain open. On one hand, the deep features are coarsely extracted at image level rather than precisely at object level, which are interrupted by background clutters. On the other hand, training CNN features with a standard triplet loss is time consuming and incapable to learn discriminative features. In this paper, we present a novel fine-grained object retrieval scheme that conquers these issues in a unified framework. Firstly, we introduce a novel centralized ranking loss (CRL), which achieves a very efficient (1,000 times training speedup comparing to the triplet loss) and discriminative feature learning by a "centralized" global pooling. Secondly, a weakly supervised attractive feature extraction is proposed, which segments object contours with top-down saliency. Consequently, the contours are integrated into the CNN response map to precisely extract features "within" the target object. Interestingly, we have discovered that the combination of CRL and weakly supervised learning can reinforce each other. We evaluate the performance of the proposed scheme on widely-used benchmarks including CUB200-2011 and CARS196. We have reported significant gains over the state-of-the-art schemes, e.g., 5.4% over SCDA [Wei et al., 2017] on CARS196, and 3.7% on CUB200-2011.

Towards Optimal Fine Grained Retrieval via Decorrelated Centralized Loss with Normalize-Scale Layer

“…We have conducted extensive experiments on two widelyused FGIR benchmark CUB-200-2011 and CARS196, with comparisons to a set of state-of-the-art methods. Quantitatively, it outperforms CRL-WSL (Zheng et al 2018) by 12.0% in CAR196, and runs 5× faster in training over triplet loss.…”

“…Fine-Grained Image Retrieval (FGIR). FGIR has attracted increasing research focus in recent years (Wei et al 2017;Xie et al 2015;Zhang et al 2016;Zheng et al 2018). It aims to differentiate subordinate classes, where the challenges are two-fold: 1) Most classes are highly correlated and difficult to be distinguished due to their subtle difference i.e., small inter-class variance.…”

Global Optimal Structured Embedding Learning for Remote Sensing Image Retrieval

“…The clustering-based structured losses aim to learn a discriminative embedding space by optimizing clustering metric and are applied in abundant fields of computer vision like face recognition [53,54] and fine-grained image retrieval (FGIR) [55,56]. Clustering loss [57] utilizes the structured prediction framework to realize clustering with higher score for ground truth than others.…”

“…The triple-center loss (TCL) [59] was proposed to learn a center for each category and separate the cluster centers and their relevant samples from different categories. To enhance the performance of FGIR, centralized ranking loss (CRL) [55] was proposed aiming to optimize centers and enlarge the compactness and separability of intraclass and interclass samples. Later, decorrelated global-aware centralized loss (DGCRL) [56] was proposed to optimize the center space by utilizing Gram-Schmidt independent operation and enhance the clustering result by combining softmax loss.…”

A Novel Semantics-Preserving Hashing for Fine-Grained Image Retrieval

“…To some extent, SCDA is not a deep learning method because it just localizes salient regions with a pretrained CNN model. To address the limitation of unsupervised fine-grained image retrieval by pre-trained models, CRL-WSL [24] proposes a unified architecture to jointly learn salient regions and meaningful deep descriptors with labeled data. Recently, DCL-NS [25] obtains state-of-the-art performance on several benchmark datasets.…”