Learning to Hash with Convolutional Network for Multi-label Remote Sensing Image Retrieval

Abstract

Recently, deep hashing dominated single label image retrieval approaches. However, the complex nature of remote sensing images, which likely contains multi-labels, hardly benefits from the above approaches. To overcome single-label image retrieval limitations in remote sensing domain, we address this problem by proposing a multi-label remote sensing image retrieval (MLRSIR-NET) framework. Specifically, the proposed MLRSIR-NET composed of two main sub-networks: multi-level feature extraction and deep hash. The multi-level feature extraction network predicts multi-level features to exploit different levels of Convolution Neural Network (CNN (characteristics. To suppress discriminative feature representation, the multi-level features are aggregated and feed to Convolutional Block Attention Module (CBAM) to amplify the representation of relevant multi-label features. CBAM is flexibly integrated into our network with end-to-end training. The hash network stacked two fully connected layers aimed to learn multiple hashing functions to encode the feature vector into a compact hash code. Finally, we conduct experiments on two benchmarks for multi-label images: Dense Labelling Remote Sensing Dataset (DLRSD) and Wuhan Dense Labeling Dataset (WHDLD) to systematically assess the performance. The results show that the proposed framework improved the accuracy in terms of Mean Average Precision (MAP) by a considerable margin of 85.4%, 87.2%, 90.8% and 92.9% for 12-bit, 24-bit, 32-bit and 48-bit code lengths respectively on DLRSD. For WHDLD, it can be noted that the proposed framework supers the DCH by 93.8%, 98.7%, 91.9%, and 94.6% on average respectively.

ICIHRN: An Interpretable Multilabel Hash Retrieval Method for Satellite Cloud Images

“…In order to verify the validity of the proposed model, we further compared the performance of our method with the existing methods. The method chosen for comparison include ResNet-50, Transformer [49], Swim-Transformer [50], FAH [34], FDRL [47] and MLRSIR-NET [48]. Swim-Transformer adopts a hierarchical structure for adapting images of different scales and implements a linear complexity attention computation using a sliding window approach to optimize the Transformer.…”

Attention-based localized hash retrieval for large-scale remote sensing images using deep feature splitting strategy

“…As it is identified that the single-labeled RSIR methods cannot meet the demand for flexibility and accuracy, and the image is often associated with several real-world concepts, multilabel hashing methods are experimented with. 14,17 A kernel-based feature fusion using supervised hashing is designed to express high-resolution RSI's highly complex geometrical structures and spatial patterns. 29 A hash retrieval strategy that combines hash learning with proxy-based metric learning in a convolutional neural network is presented.…”

“…This issue is rectified by the exceptional breakthrough of deep learning (DL) frameworks with the powerful representational ability for features extracted. As a result of the advancement of DL approaches, deep features are utilized in numerous RSIR tasks 14 – 18 …”

“…As a result of the advancement of DL approaches, deep features are utilized in numerous RSIR tasks. [14][15][16][17][18] The exhaustive matching and retrieval of similar images based on a visual query are computationally demanding for a huge volume RSI collection 1 using a computationally expensive similarity function. Also, the rapid growth of RSI volume poses a significant challenge regarding fast and efficient retrieval from massive RSI archives.…”

Hyperspectral change detection based on modification of UNet neural networks

“…The availability of satellite instruments, the enormous amount of data acquired, and the availability of computational power has enabled a deeper neural network to introduce a new challenges in the earth science domain. 15,16 Recent advances in DL have demonstrated state-of-the-art results in pattern recognition tasks, mainly in image processing and speech recognition. 17,18 Modern convolutional neural network (CNN) architectures [19][20][21] tend to contain enormous hidden layers and millions of neurons, allowing them to concurrently learn hierarchical features for a broad class of patterns from data and achieve well-tailored models for the targeted application.…”