3-D Channel and Spatial Attention Based Multiscale Spatial–Spectral Residual Network for Hyperspectral Image Classification

Lu, Zhenyu; Bin, Xu; Sun, Le; Zhan, Tianming; Tang, Songze

doi:10.1109/jstars.2020.3011992

Cited by 57 publications

(31 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sun et al [54] make a attention module that can be embedded anywhere in the spectral module and spatial module for HSIC. Based on this model, Lu et al [55] proposed a 3-D attention module that consists of a channel attention module and a spatial attention module. Swalpa et al [56] proposed an attention-based adaptive spectralspatial kernel module that wes introduced for the first time to learn selective 3-D convolutional kernels for H-SIC.…”

Section: A Attention Mechanismmentioning

confidence: 99%

Siamese Spectral Attention With Channel Consistency for Hyperspectral Image Classification

Wang¹,

Yang²,

Liu³

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Abundant spectral features are the precious wealth of hyperspectral images (HSI). Nevertheless, well-designed spectral feature is still a challenge that affects the performance of the classifier, especially with insufficient number of training samples. To make up the poor discriminability of spectral feature, doublebranch methods are proposed by fusing parallel spectral and spatial branches. However, this structure does nothing to improve the quality of spectral feature, which is regarded as the most valuable information for hyperspectral image information. In this paper, we propose a siamese spectral attention network with channel consistency (SSACC) to focus on obtaining discriminative spectral features, thus improving the generalization ability of the classifier. Two kinds of HSI cubes with different patch sizes are generated as the input of SSACC. The two cubes are divided into top and bottom branches and then be fed into the siamese network to obtain the refined spectral features. Then, self-attention is conducted to interacting with each channel for the spectral features enhancement. Meanwhile, two attention maps are obtained to display the spectral structures of each branch. A channel consistency regularization is performed on the two attention maps by enforcing the two branches to possess similar spectral patterns when identifying the same centric pixel. Extensive experiments conducted on the three HSI datasets verify the superiority of the obtained spectral feature. Furthermore, the proposed method applying convolution only on the spectral domain outperforms the state-of-the-art double-branch methods which integrate the spectral and spatial features simultaneously. Index Terms-spectral siamese; channel consistency; doublebranch; hyperspectral image classification.

show abstract

Section: A Attention Mechanismmentioning

confidence: 99%

Siamese Spectral Attention With Channel Consistency for Hyperspectral Image Classification

Wang¹,

Yang²,

Liu³

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In SSRN, spectral and spatial features were extracted by constructing spectral and spatial residual blocks, which further improved the recognition accuracy. Lu et al [39] proposed a new 3-D channel and spatial attention-based multi-scale spatial spectral residual network (CSMS-SSRN). CSMS-SSRN used a three-layer parallel residual network structure to constantly learn spatial and spectral features from their respective residual blocks by using different 3-D convolution kernels, and then superimposed the extracted multi-scale features and input them into the 3-D attention module.…”

Section: Introductionmentioning

confidence: 99%

Three-Dimensional Convolutional Neural Network Model for Early Detection of Pine Wilt Disease Using UAV-Based Hyperspectral Images

Luo

et al. 2021

Remote Sensing

View full text Add to dashboard Cite

As one of the most devastating disasters to pine forests, pine wilt disease (PWD) has caused tremendous ecological and economic losses in China. An effective way to prevent large-scale PWD outbreaks is to detect and remove the damaged pine trees at the early stage of PWD infection. However, early infected pine trees do not show obvious changes in morphology or color in the visible wavelength range, making early detection of PWD tricky. Unmanned aerial vehicle (UAV)-based hyperspectral imagery (HI) has great potential for early detection of PWD. However, the commonly used methods, such as the two-dimensional convolutional neural network (2D-CNN), fail to simultaneously extract and fully utilize the spatial and spectral information, whereas the three-dimensional convolutional neural network (3D-CNN) is able to collect this information from raw hyperspectral data. In this paper, we applied the residual block to 3D-CNN and constructed a 3D-Res CNN model, the performance of which was then compared with that of 3D-CNN, 2D-CNN, and 2D-Res CNN in identifying PWD-infected pine trees from the hyperspectral images. The 3D-Res CNN model outperformed the other models, achieving an overall accuracy (OA) of 88.11% and an accuracy of 72.86% for detecting early infected pine trees (EIPs). Using only 20% of the training samples, the OA and EIP accuracy of 3D-Res CNN can still achieve 81.06% and 51.97%, which is superior to the state-of-the-art method in the early detection of PWD based on hyperspectral images. Collectively, 3D-Res CNN was more accurate and effective in early detection of PWD. In conclusion, 3D-Res CNN is proposed for early detection of PWD in this paper, making the prediction and control of PWD more accurate and effective. This model can also be applied to detect pine trees damaged by other diseases or insect pests in the forest.

show abstract

“…Liu et al [34] performed transfer learning between different HSI datasets to improve the HSI classification for small-sample conditions. To learn more representative features from the original HSI, sparse representation [35], metric learning [36], and attention techniques [37] are also used for refining the learned spectral-spatial features. Moreover, some learning-based methods introduced the generative adversarial network (GAN) to improve the classification accuracy and mitigate the overfitting risk [38,39].…”

Section: Introductionmentioning

confidence: 99%

Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

Lei

Zhang

Liu

et al. 2021

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Deep learning models have shown excellent performance in the hyperspectral remote sensing image (HSI) classification. In particular, convolutional neural networks (CNNs) have received widespread attention because of their powerful feature-extraction ability. Recently, a capsule network (CapsNet) was introduced to boost the performance of CNNs, marking a remarkable progress in the field of HSI classification. In this paper, we propose a novel deep convolutional capsule neural network (DC-CapsNet) based on spectral-spatial features to improve the performance of CapsNet in the HSI classification, while significantly reducing the computation cost of the model. Specifically, a convolutional capsule layer based on the extension of dynamic routing using 3D convolution is used to reduce the number of parameters and enhance the robustness of the learned spectral-spatial features. Furthermore, a lighter and stronger decoder network composed of deconvolutional layers as a better regularization term and capable of acquiring more spatial relationships is used to further improve the HSI classification accuracy with low computation cost. In this study, we tested the performance of the proposed model on four widely used HSI datasets: the Kennedy Space Center, Indian Pines, Pavia University and Salinas datasets. We found that the DC-CapsNet achieved high classification accuracy with limited training samples and effectively reduced the computation cost.

show abstract

3-D Channel and Spatial Attention Based Multiscale Spatial–Spectral Residual Network for Hyperspectral Image Classification

Cited by 57 publications

References 59 publications

Siamese Spectral Attention With Channel Consistency for Hyperspectral Image Classification

Siamese Spectral Attention With Channel Consistency for Hyperspectral Image Classification

Three-Dimensional Convolutional Neural Network Model for Early Detection of Pine Wilt Disease Using UAV-Based Hyperspectral Images

Hyperspectral Remote Sensing Image Classification Using Deep Convolutional Capsule Network

Contact Info

Product

Resources

About