Best Representation Branch Model for Remote Sensing Image Scene Classification

Abstract

Remote sensing image scene classification is an important method for understanding the high-resolution remote sensing images. Based on Convolutional Neural Network, various classification methods have been applied into this field and achieved remarkable results. These methods mainly rely on the semantic information to improve the classification performance. However, as the network goes deeper, the highly abstract and global semantic information makes it difficult for the network to accurately classify scene images with similar layout and structures, limiting the further improvement of classification accuracy. Relying on the semantic information only is not sufficient to effectively classify these similar scene images and the network needs spatial information to enhance the classification capability. To solve this dilemma, this paper proposes a best representation branch model, which reaches the optimal balance point where the network can make use of both the semantic information and spatial information to improve the final classification accuracy. In the proposed method, ResNet50 pretrained on the ImageNet dataset is first divided into four branches with different depths to extract feature maps and Capsule Network is used as the classifier. The Grad-CAM algorithm is adopted to explain the mechanism of the optimal balance point from the perspective of attention and guide the further feature fusion. In addition, ablation studies are conducted to prove the effectiveness of our method and extensive experiments are conducted on three public benchmark remote sensing datasets. The results demonstrate that the proposed method can achieve competitive classification performance compared to the state-ofthe-art methods.

LPIN: A Lightweight Progressive Inpainting Network for Improving the Robustness of Remote Sensing Images Scene Classification

“…Remote Sens. 2022, 14, x FOR PEER REVIEW 2 of 31 vision and pattern recognition [6,7], the classification methods based on Convolutional Neural Network (CNN) [8][9][10][11][12][13][14][15][16][17][18][19] have been widely investigated for they can learn and extract image features automatically. Hu et al [9] and Du et al [10] used a pretrained CNN to extract image features.…”

“…One is the traditional machine learning-based methods with hand-crafted features, such as models based on Bag of Visual Words (BoVW) [2], Randomized Spatial Partition (RSP) [3], Hierarchical Coding Vector (HCV) [4] and Fisher vectors (FVs) [5]. As deep learning technology has been proved to have excellent performance in computer vision and pattern recognition [6,7], the classification methods based on Convolutional Neural Network (CNN) [8][9][10][11][12][13][14][15][16][17][18][19] have been widely investigated for they can learn and extract image features automatically. Hu et al [9] and Du et al [10] used a pretrained CNN to extract image features.…”

“…Wang et al [13] adopted an attention module and constructed an end-to-end network. Li et al [14], Jiang et al [15], and Zhang et al [16] used Capsule Network (CapsNet) in their model which can preserve the spatial information of RS images. To date, the RSISC tasks have achieved a quite high-level accuracy on standard datasets.…”

“…To date, the RSISC tasks have achieved a quite high-level accuracy on standard datasets. For instance, the overall accuracy (OA) is 94.87% [16] on the NWPU-RESISC45 dataset [20] and 96.05% [16] on the Aerial Image Dataset (AID) [21], both with a training ratio of only 20%. Moreover, the RS image classifications using the red-green-blue band of EuroSAT dataset [22] achieves an accuracy of 99.17% via deep transfer learning [23].…”

An Empirical Study of Remote Sensing Pretraining

“…Among these methods, many flexible and advanced modules have been explored. For example, the attention mechanisms (CBAM [34], EAM [65], MBLANet [33]), where specific channels or spatial positions of the features are highlighted, and multiscale features (F 2 BRBM [66] and GRMANet [67]), where the intermediate features are also employed. In addition, the self-distillation technology combined with specially designed loss functions (ESD-MBENet [25]) and the multibranch siamese networks (IDCCP [68]) have also been applied.…”