A Multiscale Attention Network for Remote Sensing Scene Images Classification

Abstract

The remote sensing scene images classification has been of great value to civil and military fields. Deep learning models, especially the convolutional neural network (CNN), have achieved great success in this task, however, they may suffer from two challenges: firstly, the sizes of the category objects are usually different, but the conventional CNN extracts the features with fixed convolution extractor which could cause the failure in learning the multi-scale features; secondly, some image regions may not be useful during the feature learning process, therefore, how to guide the network to select and focus on the most relevant regions is crucially vital for remote sensing scene image classification. To address these two challenges, we propose a multi-scale attention network (MSA-Network), which integrates a multi-scale (MS) module and a channel and position attention (CPA) module to boost the performance of the remote sensing scene classification. The proposed MS module learns multi-scale features by adopting various sizes of sliding windows from different depths' layers and receptive fields. The CPA module is composed of two parts: the channel attention (CA) module and the position attention (PA) one. The CA module learns the global attention features from channel-level, and the PA module extracts the local attention features from pixel-level. Thus, fusing both of those two attention features, the network is apt to focus on the more critical and salient regions automatically. Extensive experiments on UC Merced, AID, NWPU-RESISC45 datasets demonstrate that the proposed MSA-Network outperforms several state-of-the-art methods.

Homo–Heterogenous Transformer Learning Framework for RS Scene Classification

“…To comprehensively evaluate the classification performance of the proposed HHTL framework, we compare our method with some state-of-the-art CNN-based and transformer-based methods. The CNN-based methods are GoogLeNet [1], [2], VGGNet-16 [1], [2], VGG-16-CapsNet [71], SCCov [24], VGG-VD16+MSCP+MRA [72], GBNet+global feature [40], MIDC-Net CS [73], EFPN-DSE-TDFF [41], DFAGCN [74], EfficientNet-B0-aux [75], SF-CNN with VGGNet [76], MG-CAP (Sqrt-E) [43], ACNet [52], ACR-MLFF [77], MSA-Network [78]. The transformer-based methods are T2T-ViT-12 [62], Pooling-based Vision Transformer-Small (PiT-S) [79], and Pyramid Vision Transformer-Medium (PVT-Medium) [80].…”

“…Like to UCM and AID, we can find that the performance of our HHTL framework is best. Compared with other methods, when 10% scenes are used for training, the enhancements in OA obtained by our HHTL framework are 15.88% (over GoogLeNet), 15.6% (over VGGNet-16), 6.99% (over VGG-16-CapsNet), 2.77% (over SCCov), 4% (over VGG-VD16+MSCP+MRA), 5.95% (over MIDC-Net CS), 0.98% (over ACNet), 2.11% (over EfficientNet-B0-aux), 2.18% (over SF-CNN with VGGNet), 2.06% (over ACR-MLFF), 1.69% (over MSA-Network), 1.24% (over MG- [71] 91.63±0.19 94.74±0.17 SCCov [24] 93.12±0.25 96.10±0.16 VGG-VD16+MSCP+MRA [72] 92.21±0.17 95.56±0.18 GBNet+global feature [40] 92.20±0.23 95.48±0.12 MIDC-Net CS [73] 88.51±0.41 92.95±0.17 EFPN-DSE-TDFF [41] 94.02±0.21 94.50±0.30 ACNet [52] 93.33±0.29 95.38±0.29 DFAGCN [74] -94.88±0.22 EfficientNet-B0-aux [75] 93.69±0.11 96.17±0.16 SF-CNN with VGGNet [76] 93.60±0.12 96.66±0.11 ACR-MLFF [77] 92.73±0.12 95.06±0.33 MSA-Network [78] 93.53±0.21 96.01±0.43 MG-CAP(Sqrt-E) [43] 93.34±0.18 96.12±0.12 HHTL (ours) 95.62±0.13 96.88±0.21…”

“…How to enhance HHTL's behavior in these content-similar categories is our future work. [71] 85.08±0.13 89.18±0.14 SCCov [24] 89.30±0.35 92.10±0.25 VGG-VD16+MSCP+MRA [72] 88.07±0.18 90.81±0.13 MIDC-Net CS [73] 86.12±0.29 87.99±0.18 ACNet [52] 91.09±0.13 92.42±0.16 DFAGCN [74] -89.29±0.28 EfficientNet-B0-aux [75] 89.96±0.27 92.89±0.16 SF-CNN with VGGNet [76] 89.89±0.16 92.55±0.14 ACR-MLFF [77] 90.01±0.33 92.45±0.20 MSA-Network [78] 90.38±0.17 93.52±0.21 MG-CAP(Sqrt-E) [43] 90.83±0.12 92.95±0.13 HHTL (ours) 92.07±0.44 94.21±0.09…”

Classification of hyperspectral images via improved cycle‐MLP

“…Zhang et al. proposed a multiscale attention network (MSA‐Network) for HSI classification, where the model integrates multiple scales of channels and position attention modules to enhance different scales of features [18]. Gong et al.…”

“…Furthermore, Tian et al proposed a multiscale dense convolutional network [17], which enhances multi-scale features and incorporates squeeze-andexcitation to process features of various scenes adaptively, solving the problem of inadequate feature extraction in conventional CNNs. Zhang et al proposed a multiscale attention network (MSA-Network) for HSI classification, where the model integrates multiple scales of channels and position attention modules to enhance different scales of features [18]. Gong et al fused the spectral information with multi-scale spatial information by using two-dimensional convolution with different convolutional kernel sizes [19].…”