Enhancing remote sensing image retrieval using a triplet deep metric learning network

Abstract

With the rapid growing of remotely sensed imagery data, there is a high demand for effective and efficient image retrieval tools to manage and exploit such data. In this letter, we present a novel content-based remote sensing image retrieval method based on Triplet deep metric learning convolutional neural network (CNN). By constructing a Triplet network with metric learning objective function, we extract the representative features of the images in a semantic space in which images from the same class are close to each other while those from different classes are far apart. In such a semantic space, simple metric measures such as Euclidean distance can be used directly to compare the similarity of images and effectively retrieve images of the same class. We also investigate a supervised and an unsupervised learning methods for reducing the dimensionality of the learned semantic features. We present comprehensive experimental results on two publicly available remote sensing image retrieval datasets and show that our method significantly outperforms state-of-the-art.Index Terms-Deep learning, metric learning, content-based image retrieval (CBIR), remote sensing image retrieval (RSIR).

Similarity Retention Loss (SRL) Based on Deep Metric Learning for Remote Sensing Image Retrieval

“…When using the ResNet50 network framework, on the UCMD dataset, the experimental results achieve +8.38% growth compared to MiLaN in mAP and achieve mAP of 99.41%, P@10 of 100 and offer over 73.11%, 95.53% gain over the GCN on PATTERNNET dataset. At the same time, we find that although the effect of the EDML (Enhancing Remote Sensing Image Retrieval with Triplet Deep Metric Learning Network) [53] on the PatternNet dataset is slightly higher than our SRL, for example, the EDML achieves a gain of +1.40% and +0.14% in mAP on PatternNet database, which trained respectively on the VGG16 network and ResNet50. But based on comprehensive experimental results, our SRL is the best.…”

Global Optimal Structured Embedding Learning for Remote Sensing Image Retrieval

“…More and more elegant works prefer to apply DML in the field of remote sensing images to enhance the effectiveness of RSIR [30,[33][34][35][36][37]. Roy et al proposed a metric and hash-code learning network (MHCLN) which could be used to learn semantic embedding space and produce hash codes at the same time [33].…”

“…It aims to realize accurate and fast retrieval in the task of RSIR. Cao et al presented a novel triplet deep metric learning network for RSIR, the remote sensing images are embedded into the learned embedding space where the positive sample pairs closer and negative ones far away from each other [34]. Subhanker et al presented a novel hashing framework which is based on metric learning [35].…”

“…With the development of deep learning research, it has been introduced to the tasks of RSIA like recognition [26,27], classification [1,5,[28][29][30] and retrieval [2,5,31,32]. The high-level discriminative feature representations extracted from CNN with metric learning are more and more frequently used to boost the performance of RSIR [30,[33][34][35][36][37]. Deep metric learning (DML) is an efficient approach which forces the images close to the similar visual information and pushes the dissimilar ones far away from each other [38][39][40].…”

“…Although ranked list loss [46] has made full use of structured information inside the training mini-batch, it fails to consider the relationship between positive and negative sample pairs. Recently, there are some attempts tending to utilize contrastive and triplet loss to fine-tune the network model for the task of RSIR and have obtained appreciable performance [33][34][35].…”