Cross-Domain Distance Metric Learning Framework With Limited Target Samples for Scene Classification of Aerial Images

IEEE Trans. Geosci. Remote Sensing

Fernández-Beltrán

et al. 2020

With the development of convolutional neural networks (CNN), the semantic understanding of remote sensing scenes has been significantly improved based on their prominent feature encoding capabilities. Whereas many existing deep-learning models focus on designing different architectures, only few works in the remote sensing field have focused on investigating the performance of the learned feature embeddings and the associated metric space. In particular, two main loss functions have been exploited: the contrastive and the triplet loss. However, the straightforward application of these techniques to remote sensing images may not be optimal in order to capture their neighborhood structures in the metric space, due to the insufficient sampling of image pairs or triplets during the training stage, and to the inherent semantic complexity of remotely sensed data. To solve these problems, we propose a new deep metric learning approach, which overcomes the limitation on the class discrimination by means of two different components: 1) scalable neighborhood component analysis (SNCA), which aims at discovering the neighborhood structure in the metric space; and 2) the cross entropy loss, which aims at preserving the class discrimination capability based on the learned class prototypes. Moreover, in order to preserve feature consistency among all the mini-batches during training, a novel optimization mechanism based on momentum update is introduced for minimizing the proposed loss. An extensive experimental comparison (using several state-of-the-art models and two different benchmark datasets) has been conducted to validate the effectiveness of the proposed method from different perspectives, including: 1) classification; 2) clustering; and 3) image retrieval. The related codes of this paper will be made publicly available for reproducible research by the community.

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Deep Metric Learning Based on Scalable Neighborhood Components for Remote Sensing Scene Characterization

IEEE Trans. Geosci. Remote Sensing

Fernández-Beltrán

et al. 2020

“…Despite the impressive results achieved by the CNN architectures that are incorporated into the attention mechanism, they still pose various challenges. First of all, they still suffer from high intra-class variations existing in scene images of HRRSI [22]. That is because diverse seasons, locations or sensors may lead to highly different spectral characteristics of scene images with the same category [23], as shown in Figure 2.…”

Section: Introductionmentioning

confidence: 99%

RETRACTED: Attention-Based Deep Feature Fusion for the Scene Classification of High-Resolution Remote Sensing Images

Zhu

Yan

et al. 2019

Remote Sensing

Scene classification of highresolution remote sensing images (HRRSI) is one of the most important means of landcover classification. Deep learning techniques, especially the convolutional neural network (CNN) have been widely applied to the scene classification of HRRSI due to the advancement of graphic processing units (GPU). However, they tend to extract features from the whole images rather than discriminative regions. The visual attention mechanism can force the CNN to focus on discriminative regions, but it may suffer from the influence of intraclass diversity and repeated texture. Motivated by these problems, we propose an attention-based deep feature fusion (ADFF) framework that constitutes three parts, namely attention maps generated by Gradientweighted Class Activation Mapping (GradCAM), a multiplicative fusion of deep features and the centerbased cross-entropy loss function. First of all, we propose to make attention maps generated by GradCAM as an explicit input in order to force the network to concentrate on discriminative regions. Then, deep features derived from original images and attention maps are proposed to be fused by multiplicative fusion in order to consider both improved abilities to distinguish scenes of repeated texture and the salient regions. Finally, the centerbased cross-entropy loss function that utilizes both the cross-entropy loss and center loss function is proposed to backpropagate fused features so as to reduce the effect of intraclass diversity on feature representations. The proposed ADFF architecture is tested on three benchmark datasets to show its performance in scene classification. The experiments confirm that the proposed method outperforms most competitive scene classification methods with an average overall accuracy of 94% under different training ratios.

“…More recently, deep-learning methods (which directly summarize high-level semantics from largescale RS data through end-to-end neural networks) have exhibited prominent performance for intelligently interpreting RS scenes [28], [29]. Specifically, deep metric learning methods have received particular attention in this context, owing to their good performance in the task of discriminating interclass features and discovering the inherent structure of intraclass features [30]- [32]. The main goal of these methods is to separate and group the features extracted from semantically similar and dissimilar images, respectively.…”

Section: Introductionmentioning

confidence: 99%

Noise-Tolerant Deep Neighborhood Embedding for Remotely Sensed Images With Label Noise

Fernández-Beltrán

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

et al. 2021

Recently, many deep learning-based methods have been developed for solving remote sensing (RS) scene classification or retrieval tasks. Most of the adopted loss functions for training these models require accurate annotations. However, the presence of noise in such annotations (also known as label noise) cannot be avoided in large-scale RS benchmark archives, resulting from geo-location/registration errors, land-cover changes, and diverse knowledge background of annotators. To overcome the influence of noisy labels on the learning process of deep models, we propose a new loss function called noise-tolerant deep neighborhood embedding which can accurately encode the semantic relationships among RS scenes. Specifically, we target at maximizing the leave-one-out K-NN score for uncovering the inherent neighborhood structure among the images in feature space. Moreover, we down-weight the contribution of potential noisy images by learning their localized structure and pruning the images with low leaveone-out K-NN scores. Based on our newly proposed loss function, classwise features can be more robustly discriminated. Our experiments, conducted on two benchmark RS datasets, validate the effectiveness of the proposed approach on three different RS scene interpretation tasks, including classification, clustering, and retrieval. The codes of this article will be publicly available from https://github.com/jiankang1991.