Improved Deep Hashing With Soft Pairwise Similarity for Multi-Label Image Retrieval

Due to the proliferation of large-scale remotesensing (RS) archives with multiple annotations, multilabel RS scene classification and retrieval are becoming increasingly popular. Although some recent deep learning-based methods are able to achieve promising results in this context, the lack of research on how to learn embedding spaces under the multilabel assumption often makes these models unable to preserve complex semantic relations pervading aerial scenes, which is an important limitation in RS applications. To fill this gap, we propose a new graph relation network (GRN) for multilabel RS scene categorization. Our GRN is able to model the relations between samples (or scenes) by making use of a graph structure which is fed into network learning. For this purpose, we define a new loss function called scalable neighbor discriminative loss with binary cross entropy (SNDL-BCE) that is able to embed the graph structures through the networks more effectively. The proposed approach can guide deep learning techniques (such as convolutional neural networks) to a more discriminative metric space, where semantically similar RS scenes are closely embedded and dissimilar images are separated from a novel multilabel viewpoint. To achieve this goal, our GRN jointly maximizes a weighted leave-one-out K-nearest neighbors (K NN) score in the training set, where the weight matrix describes the contributions of the nearest neighbors associated with each RS image on its class decision, and the likelihood of the class discrimination in

show abstract

“…2) mean average precision (MAP) [68], [69]; and 3) hamming loss (HL). To be specific, WMAP is calculated as…”

Section: ) Multilabel Rs Image Classificationmentioning

confidence: 99%

Graph Relation Network: Modeling Relations Between Scenes for Multilabel Remote-Sensing Image Classification and Retrieval

Kang

Fernández-Beltrán

Hong

et al. 2021

IEEE Trans. Geosci. Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Once feature points are detected, feature descriptions need to be established for matching. Feature descriptors are numeric vectors that encode the characteristics of the local region of the feature points [34]. Traditional feature descriptors are formulated on intensity or gradient domains such as the scale-invariant feature transform (SIFT) [14], speeded-up robust features (SURF) [15], and binary robust independent elementary feature (BRIEF) [35], perform well on single spectrum images, but they behave poorly when dealing with multispectral data [18].…”

Section: Related Workmentioning

confidence: 99%

Automatic Matching of Multispectral Images Based on Nonlinear Diffusion of Image Structures

Ruixiang

Zhao

Zhang

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

Self Cite

View full text Add to dashboard Cite

“…However, it only takes the multi-label information to supervise the label network training, and the original images or text are still measured by single-label. Improved deep hashing network (IDHN) ( Zhang et al, 2019 ) introduces pairwise similarity metrics to fit the multi-label instances applications. In contrast, this method concentrates on the single modality hashing retrieval.…”

Section: Related Workmentioning

confidence: 99%

Hierarchical semantic interaction-based deep hashing network for cross-modal retrieval

Chen

Wang

2021

PeerJ Computer Science

View full text Add to dashboard Cite

Due to the high efficiency of hashing technology and the high abstraction of deep networks, deep hashing has achieved appealing effectiveness and efficiency for large-scale cross-modal retrieval. However, how to efficiently measure the similarity of fine-grained multi-labels for multi-modal data and thoroughly explore the intermediate layers specific information of networks are still two challenges for high-performance cross-modal hashing retrieval. Thus, in this paper, we propose a novel Hierarchical Semantic Interaction-based Deep Hashing Network (HSIDHN) for large-scale cross-modal retrieval. In the proposed HSIDHN, the multi-scale and fusion operations are first applied to each layer of the network. A Bidirectional Bi-linear Interaction (BBI) policy is then designed to achieve the hierarchical semantic interaction among different layers, such that the capability of hash representations can be enhanced. Moreover, a dual-similarity measurement (“hard” similarity and “soft” similarity) is designed to calculate the semantic similarity of different modality data, aiming to better preserve the semantic correlation of multi-labels. Extensive experiment results on two large-scale public datasets have shown that the performance of our HSIDHN is competitive to state-of-the-art deep cross-modal hashing methods.

show abstract

Improved Deep Hashing With Soft Pairwise Similarity for Multi-Label Image Retrieval

Cited by 127 publications

References 55 publications

Graph Relation Network: Modeling Relations Between Scenes for Multilabel Remote-Sensing Image Classification and Retrieval

Graph Relation Network: Modeling Relations Between Scenes for Multilabel Remote-Sensing Image Classification and Retrieval

Automatic Matching of Multispectral Images Based on Nonlinear Diffusion of Image Structures

Hierarchical semantic interaction-based deep hashing network for cross-modal retrieval

Contact Info

Product

Resources

About