Spatial–Spectral Feature Refinement for Hyperspectral Image Classification Based on Attention-Dense 3D-2D-CNN

Zhang, Jin; Feng-yuan, Wei; Feng, Fan; Wang, Chunyang

doi:10.3390/s20185191

Cited by 30 publications

(16 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The points in the red circles in each figure are too small to avoid the reduction in the overall accuracy of the MDAN model. According to studies in the literature, the algorithms for HSI classification are sensitive to unbalanced datasets in the predictor classes [23,38]. A model developed based on unbalanced datasets tends to result in false predictions in small samples, but the overall accuracy of the predictions is not necessarily low.…”

Section: Classification Resultsmentioning

confidence: 99%

“…Moreover, for the problem of small samples, an attention mechanism was applied to an HSI analysis task [21,22]. The attention mechanism is a resource allocation scheme that can improve the performance of a model with a little computational complexity [23], such as the squeeze-and-excitation networks (SENets) [24], the selective kernel networks (SKNets) [25], the convolutional block attention module (CBAM) [26], and the bottleneck attention module (BAM) [27]. Compared with SENet, SKNet, and BAM, CBAM is a lightweight model, and it can extract attention features in spatial-spectral dimensions for adaptive feature refinement.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

An Investigation of a Multidimensional CNN Combined with an Attention Mechanism Model to Resolve Small-Sample Problems in Hyperspectral Image Classification

Liu

Zhang

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

The convolutional neural network (CNN) method has been widely used in the classification of hyperspectral images (HSIs). However, the efficiency and accuracy of the HSI classification are inevitably degraded when small samples are available. This study proposes a multidimensional CNN model named MDAN, which is constructed with an attention mechanism, to achieve an ideal classification performance of CNN within the framework of few-shot learning. In this model, a three-dimensional (3D) convolutional layer is carried out for obtaining spatial–spectral features from the 3D volumetric data of HSI. Subsequently, the two-dimensional (2D) and one-dimensional (1D) convolutional layers further learn spatial and spectral features efficiently at an abstract level. Based on the most widely used convolutional block attention module (CBAM), this study investigates a convolutional block self-attention module (CBSM) to improve accuracy by changing the connection ways of attention blocks. The CBSM model is used with the 2D convolutional layer for better performance of HSI classification purposes. The MDAN model is applied for classification applications using HSI, and its performance is evaluated by comparing the results with the support vector machine (SVM), 2D CNN, 3D CNN, 3D–2D–1D CNN, and CBAM. The findings of this study indicate that classification results from the MADN model show overall classification accuracies of 97.34%, 96.43%, and 92.23% for Salinas, WHU-Hi-HanChuan, and Pavia University datasets, respectively, when only 1% HSI data were used for training. The training and testing times of the MDAN model are close to those of the 3D–2D–1D CNN, which has the highest efficiency among all comparative CNN models. The attention model CBSM is introduced into MDAN, which achieves an overall accuracy of about 1% higher than that of the CBAM model. The performance of the two proposed methods is superior to the other models in terms of both efficiency and accuracy. The results show that the combination of multidimensional CNNs and attention mechanisms has the best ability for small-sample problems in HSI classification.

show abstract

Section: Classification Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

An Investigation of a Multidimensional CNN Combined with an Attention Mechanism Model to Resolve Small-Sample Problems in Hyperspectral Image Classification

Liu

Zhang

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Instead of generating a single 3D attention map, Ref. [28] used a channel attention module and a spatial attention module separately.…”

Section: Literature Reviewmentioning

confidence: 99%

Semantic Segmentation and Edge Detection—Approach to Road Detection in Very High Resolution Satellite Images

Ghandorh

Boulila

Masood³

et al. 2022

Remote Sensing

View full text Add to dashboard Cite

Road detection technology plays an essential role in a variety of applications, such as urban planning, map updating, traffic monitoring and automatic vehicle navigation. Recently, there has been much development in detecting roads in high-resolution (HR) satellite images based on semantic segmentation. However, the objects being segmented in such images are of small size, and not all the information in the images is equally important when making a decision. This paper proposes a novel approach to road detection based on semantic segmentation and edge detection. Our approach aims to combine these two techniques to improve road detection, and it produces sharp-pixel segmentation maps, using the segmented masks to generate road edges. In addition, some well-known architectures, such as SegNet, used multi-scale features without refinement; thus, using attention blocks in the encoder to predict fine segmentation masks resulted in finer edges. A combination of weighted cross-entropy loss and the focal Tversky loss as the loss function is also used to deal with the highly imbalanced dataset. We conducted various experiments on two datasets describing real-world datasets covering the three largest regions in Saudi Arabia and Massachusetts. The results demonstrated that the proposed method of encoding HR feature maps effectively predicts sharp segmentation masks to facilitate accurate edge detection, even against a harsh and complicated background.

show abstract

“…Spatial attention can calculate independent weights for heterogeneous pixels and obtain global spatial information, which may weaken spatial homogeneity and heterogeneity in HSI. In [19,20], the authors added a spatial attention module to model discriminative and representative features. There is another common spatial attention mechanism, namely nonlocal module [21].…”

Section: Introductionmentioning

confidence: 99%

Local Transformer With Spatial Partition Restore for Hyperspectral Image Classification

Xue

Zhang³

2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Convolutional neural network (CNN) has exhibited enormous potentials in hyperspectral image (HSI) classification owing to excellent locally modeling ability. Although excellent performance of CNN-based methods has been witnessed, there still have some limitations of their internal network backbone. On the one hand, modeling long-distance context dependencies is an inborn defect, which leads to receptive field limitation and insufficient feature capture in HSI. On the other hand, CNN-based methods usually need various sample distribution to train and cannot infer dynamically, which may not capture the inherent changes of HSI data well. To overcome the above issues, we propose a novel local transformer with spatial partition restore network (SPRLT-Net) for HSI classification. Firstly, local transformer is introduced to obtain the spatial attention weights dynamically by measuring the similarity between related pixel pairs. Secondly, a spatial partition restore (SPR) module is designed to split the input patch into several overlapping continuous sub-patches as sequential. With the obtained attention weights at hand, the SPR module restores the sequential to the original patch. Finally, a fully connected layer is used for classification. SPRLT-Net can capture global context dependencies, and the dynamical attention weights can adapt the inherent changes of HSI spatial pixels. Experimental results based on spatially disjoint samples and randomly selected samples of five benchmark data sets demonstrate that SPRLT-Net outperforms the other state-of-the-art methods in terms of classification accuracy, generalization performance, and computational complexity.

show abstract

Spatial–Spectral Feature Refinement for Hyperspectral Image Classification Based on Attention-Dense 3D-2D-CNN

Cited by 30 publications

References 42 publications

An Investigation of a Multidimensional CNN Combined with an Attention Mechanism Model to Resolve Small-Sample Problems in Hyperspectral Image Classification

An Investigation of a Multidimensional CNN Combined with an Attention Mechanism Model to Resolve Small-Sample Problems in Hyperspectral Image Classification

Semantic Segmentation and Edge Detection—Approach to Road Detection in Very High Resolution Satellite Images

Local Transformer With Spatial Partition Restore for Hyperspectral Image Classification

Contact Info

Product

Resources

About