SSDAN: Multi-Source Semi-Supervised Domain Adaptation Network for Remote Sensing Scene Classification

Abstract

We present a new method for multi-source semi-supervised domain adaptation in remote sensing scene classification. The method consists of a pre-trained convolutional neural network (CNN) model, namely EfficientNet-B3, for the extraction of highly discriminative features, followed by a classification module that learns feature prototypes for each class. Then, the classification module computes a cosine distance between feature vectors of target data samples and the feature prototypes. Finally, the proposed method ends with a Softmax activation function that converts the distances into class probabilities. The feature prototypes are also divided by a temperature parameter to normalize and control the classification module. The whole model is trained on both the unlabeled and labeled target samples. It is trained to predict the correct classes utilizing the standard cross-entropy loss computed over the labeled source and target samples. At the same time, the model is trained to learn domain invariant features using another loss function based on entropy computed over the unlabeled target samples. Unlike the standard cross-entropy loss, the new entropy loss function is computed on the model’s predicted probabilities and does not need the true labels. This entropy loss, called minimax loss, needs to be maximized with respect to the classification module to learn features that are domain-invariant (hence removing the data shift), and at the same time, it should be minimized with respect to the CNN feature extractor to learn discriminative features that are clustered around the class prototypes (in other words reducing intra-class variance). To accomplish these maximization and minimization processes at the same time, we use an adversarial training approach, where we alternate between the two processes. The model combines the standard cross-entropy loss and the new minimax entropy loss and optimizes them jointly. The proposed method is tested on four RS scene datasets, namely UC Merced, AID, RESISC45, and PatternNet, using two-source and three-source domain adaptation scenarios. The experimental results demonstrate the strong capability of the proposed method to achieve impressive performance despite using only a few (six in our case) labeled target samples per class. Its performance is already better than several state-of-the-art methods, including RevGrad, ADDA, Siamese-GAN, and MSCN.

Semi-supervised bidirectional alignment for Remote Sensing cross-domain scene classification

“…For more comprehensive evaluation, our BSCA is further compared with some SSDA methods reproduced in this study, including: (3) S+T , which is the basic method only using source data and labeled target data for training; (4) ADDA ( Wang et al, 2018 , Tzeng et al, 2017 ), which is the first work on RS-SSDA to our knowledge. It adversarially trains the source data and target data, based on two individual extractors and a shared classifier; (5) RevGrad ( Lasloum et al, 2021 , Lu et al, 2019 ), which was commonly applied to RS cross-domain scene classification, developed from the method of DANN ( Ganin et al, 2016 , Saito et al, 2019 ). It can adversarially decrease the discrimination of the feature classifier on source and target feature via a gradient reverse layer; (6) SSDAN ( Lasloum et al, 2021 ), which is the current state-of-the-art approach for RS-SSDA, derived from the method of MME ( Saito et al, 2019 ).…”

“…In recent years, an increasing number of works have focused on the domain adaptation (DA) of RS image scene classification ( Song et al, 2019 , Lu et al, 2020 , Zheng et al, 2021 , Zhang et al, 2020 , Zhu et al, 2021 , Zheng et al, 2022a , Lasloum et al, 2021 ). In Song et al (2019) , a new subspace alignment layer added into CNN models was proposed for the DA of RS image scene classification to align the source domain and the target domain in the feature subspace; as a result, it can optimize CNN models to adapt to the classification of the target domain.…”

“…In Zheng et al (2022a) , the single-source multiple-target domain adaptation task was explored for RS applications and a new algorithm named two-stage adaptation network (TSAN) was presented, which: (1) utilizes the adversarial learning approach to confuse the classifier between the source domain and the whole mixed multi-target domain, and (2) adopts self-supervised learning to divide the mixed-multiple-target domain with its pseudo domain labels in order to learn intrinsic features of multiple target domains. In Lasloum et al (2021) , the MME algorithm ( Saito et al, 2019 ) mentioned in Section 2.1 was applied to multi-source SSDA for the purpose of RS image scene classification.…”

Self-supervised learning for remote sensing scene classification under the few shot scenario

“…In semi-supervised learning, we optimize the structure of the unlabeled data through a specific loss function. For example, we can use entropy as a loss function in order to make the unlabeled data more uniformly distributed across classes 38 , because, in general, image classification datasets are balanced (meaning that there are almost the same number of images/scenes in each class). Thus, ensuring a uniform distribution across classes preserves the structure of the data and helps in reducing misclassification errors.…”

Domain Adaptation in Remote Sensing Image Classification: A Survey

“…Saha et al developed a graph neural network for multi-target DA in RS classification [175]. Lasloum et al presented a multi-source semi-supervised DA method using a pre-trained CNN for RS scene classification [176].…”