Vision Transformer With Contrastive Learning for Remote Sensing Image Scene Classification

Abstract

Remote sensing images (RSIs) are characterized by complex spatial layouts and ground object structures. ViT can be a good choice for scene classification owing to the ability to capture long-range interactive information between patches of input images. However, due to the lack of some inductive biases inherent to CNNs, such as locality and translation equivariance, ViT cannot generalize well when trained on insufficient amounts of data. Compared with training ViT from scratch, transferring a large-scale pre-trained one is more cost-efficient with better performance even when the target data is small-scale. In addition, In addition, the cross-entropy (CE) loss is frequently utilized in scene classification yet has low robustness to noise labels and poor generalization performances for different scenes. In this paper, a ViT-based model in combination with supervised contrastive learning (CL) is proposed, named ViT-CL. For CL, Supervised contrastive (SupCon) loss which is developed by extending the self-supervised contrastive approach to the fullysupervised setting, can explore the label information of RSIs in embedding space and improve the robustness to common image corruption. In ViT-CL, a joint loss function that combines CE loss and SupCon loss is developed to prompt the model to learn more discriminative features. Also, a two-stage optimization framework is introduced to enhance the controllability of the optimization process of the ViT-CL model. Extensive experiments on the AID, NWPU-RESISC45, and UCM datasets verified the superior performance of ViT-CL, with the highest accuracies of 97.42%, 94.54%, and 99.76% among all competing methods, respectively.

MBC-Net: long-range enhanced feature fusion for classifying remote sensing images

“…E.g., in the study (Ma et al ., 2022), the overlapped categories for AID include 5 pairs and that for NWPU seems to be more than 10. In the study (Bi et al ., 2023), the whole separation of categories for AID is fine, but the pairs of overlapped categories for NWPU are 8. In general, the comparison of t-SNE results is still consistent with the methods' OA performance.…”

“…In general, the comparison of t-SNE results is still consistent with the methods' OA performance. Namely, as evaluated by OAs, MBC-Net is the best, while the last one (Bi et al ., 2023) ranks second. Therefore, based on the above evidence, it validates the effectiveness and superiority of feature representations obtained by MBC-Net.…”

“…To find effective roadmaps, different learning paradigms are being developed presently, such as self-supervised learning (Wang et al, 2022c), knowledge distillation (Chen et al, 2023;Luo et al, 2023) and so on. As applications for RSI-SC, Xu et al (2022a) and Bi et al (2023) Based on all the aforementioned ideas, this work proposes the MBC-Net for RSI-SC. The reasons and differences from previous studies are listed below:…”

“…As applications for RSI-SC, Xu et al. (2022a) and Bi et al. (2023) also proposed two different methods recently: using a ViT teacher to supervise a CNN's learning or applying contrastive learning to a ViT.…”

Quantitative regularization in robust vision transformer for remote sensing image classification

“…Additionally, Sha and Li (2022) introduced a multiple-instance learning method utilising ViTs. Bi et al (2023), X. and Y. proposed innovative ViT-based approaches via contrastive learning. However, despite the increased number of parameters, these ViT-based approaches typically achieve average accuracy when compared to certain CNN-based methods.…”