Adversarial Learning for Fine-Grained Image Search

Lin, Kevin; Yang, Fan; Wang, Qiaosong; Piramuthu, Robinson

doi:10.1109/icme.2019.00091

Cited by 9 publications

(7 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In previous works [36], handcrafted features were initially utilized to tackle the FGIR problem. Powered by deep learning techniques, more and more deep learning based FGIR methods [36,42,33,43,31,26,19,32] were proposed. These deep methods can be roughly divided into two parts, i.e., supervised and unsupervised methods.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

ExchNet: A Unified Hashing Network for Large-Scale Fine-Grained Image Retrieval

Cui

Jiang

Wei³

et al. 2020

Preprint

View full text Add to dashboard Cite

Retrieving content relevant images from a large-scale finegrained dataset could suffer from intolerably slow query speed and highly redundant storage cost, due to high-dimensional real-valued embeddings which aim to distinguish subtle visual differences of fine-grained objects.In this paper, we study the novel fine-grained hashing topic to generate compact binary codes for fine-grained images, leveraging the search and storage efficiency of hash learning to alleviate the aforementioned problems. Specifically, we propose a unified end-to-end trainable network, termed as ExchNet. Based on attention mechanisms and proposed attention constraints, it can firstly obtain both local and global features to represent object parts and whole fine-grained objects, respectively. Furthermore, to ensure the discriminative ability and semantic meaning's consistency of these part-level features across images, we design a local feature alignment approach by performing a feature exchanging operation. Later, an alternative learning algorithm is employed to optimize the whole ExchNet and then generate the final binary hash codes. Validated by extensive experiments, our proposal consistently outperforms state-of-the-art generic hashing methods on five fine-grained datasets, which shows our effectiveness. Moreover, compared with other approximate nearest neighbor methods, ExchNet achieves the best speed-up and storage reduction, revealing its efficiency and practicality.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Fine-Grained Image Retrieval (FGIR) [36,42,43,31,26,19] is a practical but challenging computer vision task. It aims to retrieve images belonging to various † Equal contribution.…”

Section: Introductionmentioning

confidence: 99%

ExchNet: A Unified Hashing Network for Large-Scale Fine-Grained Image Retrieval

Cui

Jiang

Wei³

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Recently, due to the breakthrough successes of neural network-based models, significant efforts [22][23][24][25][26][27][28][29][30][31][32][33][34][35][36] have been made in exploring how to deploy popular techniques, such as reinforcement learning and adversarial learning, to solve ranking problems (we refer the reader to Section 2 for a brief overview). Despite the successes achieved by the aforementioned studies, fundamental research questions remain open.…”

Section: Introductionmentioning

confidence: 99%

Diagnostic Evaluation of Policy-Gradient-Based Ranking

Huang

Ren

et al. 2021

Electronics

View full text Add to dashboard Cite

Learning-to-rank has been intensively studied and has shown significantly increasing values in a wide range of domains, such as web search, recommender systems, dialogue systems, machine translation, and even computational biology, to name a few. In light of recent advances in neural networks, there has been a strong and continuing interest in exploring how to deploy popular techniques, such as reinforcement learning and adversarial learning, to solve ranking problems. However, armed with the aforesaid popular techniques, most studies tend to show how effective a new method is. A comprehensive comparison between techniques and an in-depth analysis of their deficiencies are somehow overlooked. This paper is motivated by the observation that recent ranking methods based on either reinforcement learning or adversarial learning boil down to policy-gradient-based optimization. Based on the widely used benchmark collections with complete information (where relevance labels are known for all items), such as MSLRWEB30K and Yahoo-Set1, we thoroughly investigate the extent to which policy-gradient-based ranking methods are effective. On one hand, we analytically identify the pitfalls of policy-gradient-based ranking. On the other hand, we experimentally compare a wide range of representative methods. The experimental results echo our analysis and show that policy-gradient-based ranking methods are, by a large margin, inferior to many conventional ranking methods. Regardless of whether we use reinforcement learning or adversarial learning, the failures are largely attributable to the gradient estimation based on sampled rankings, which significantly diverge from ideal rankings. In particular, the larger the number of documents per query and the more fine-grained the ground-truth labels, the greater the impact policy-gradient-based ranking suffers. Careful examination of this weakness is highly recommended for developing enhanced methods based on policy gradient.

show abstract

“…To overcome the shortcomings of the aforementioned two categories of ranking methods, the listwise methods [9,10,11,12,13,14,15,16,17,18,19,8] appeal to the loss function that is defined over all documents associated with the same query. Recently, inspired by generative adversarial network (GAN) and its variants, significant efforts [20,21,22,23,24,25] have been made to develop meaningful adversarial optimization methods for addressing learning-to-rank problems.…”

Section: Introductionmentioning

confidence: 99%

PT-Ranking: A Benchmarking Platform for Neural Learning-to-Rank

2020

Preprint

View full text Add to dashboard Cite

Deep neural networks has become the first choice for researchers working on algorithmic aspects of learning-to-rank. Unfortunately, it is not trivial to find the optimal setting of hyper-parameters that achieves the best ranking performance. As a result, it becomes more and more difficult to develop a new model and conduct a fair comparison with prior methods, especially for newcomers. In this work, we propose PT-Ranking 1 , an open-source project based on PyTorch for developing and evaluating learning-to-rank methods using deep neural networks as the basis to construct a scoring function. On one hand, PT-Ranking includes many representative learning-to-rank methods. Besides the traditional optimization framework via empirical risk minimization, adversarial optimization framework is also integrated. Furthermore, PT-Ranking's modular design provides a set of building blocks that users can leverage to develop new ranking models. On the other hand, PT-Ranking supports to compare different learning-to-rank methods based on the widely used datasets (e.g., MSLR-WEB30K, Yahoo!LETOR and Istella LETOR) in terms of different metrics, such as precision, MAP, nDCG, nERR. By randomly masking the ground-truth labels with a specified ratio, PT-Ranking allows to examine to what extent the ratio of unlabelled query-document pairs affects the performance of different learning-to-rank methods. We further conducted a series of demo experiments to clearly show the effect of different factors on neural learning-to-rank methods, such as the activation function, the number of layers and the optimization strategy. The experimental results reveal that the aforementioned factors significantly affect the final performance. Careful examinations of these factors are highly recommended. To summarize, PT-Ranking is highly complementary to the previous open-source projects for learningto-rank. We envision that PT-Ranking will lower the technical barrier and provide a convenient open-source platform for evaluating and developing learning-to-rank models in different fields, and thus facilitate researchers from various backgrounds.

show abstract

Adversarial Learning for Fine-Grained Image Search

Cited by 9 publications

References 51 publications

ExchNet: A Unified Hashing Network for Large-Scale Fine-Grained Image Retrieval

ExchNet: A Unified Hashing Network for Large-Scale Fine-Grained Image Retrieval

Diagnostic Evaluation of Policy-Gradient-Based Ranking

PT-Ranking: A Benchmarking Platform for Neural Learning-to-Rank

Contact Info

Product

Resources

About