Mapping Coastal Wetlands Using Transformer in Transformer Deep Network on China ZY1-02D Hyperspectral Satellite Images

Liu, Kai; Sun, Weiwei; Shao, Yijun; Liu, Weiwei; Yang, Gang; Meng, Xiangchao; Peng, Jiangtao; Mao, Dehua; Ren, Kai

doi:10.1109/jstars.2022.3173349

Cited by 39 publications

(6 citation statements)

References 49 publications

(49 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the locally sliding convolution operation of CNN with fixed kernels performs better at characterizing spatial context information and is not In terms of feature extraction, 3D-CNN is the most commonly used structure in the state-of-the-art hyperspectral classification method to integrally mine the spatial-spectral discrimination features [43]. However, the locally sliding convolution operation of CNN with fixed kernels performs better at characterizing spatial context information and is not suitable for the extraction of spectral sequence features, which will lead to information loss and reduce model efficiency [44]. While the characteristic of Transformer to extract sequence features can make up for this.…”

Section: Overview Of the Scstinmentioning

confidence: 99%

Spatial-Convolution Spectral-Transformer Interactive Network for Large-Scale Fast Refined Land Cover Classification and Mapping Based on ZY1-02D Satellite Hyperspectral Imagery

et al. 2023

View full text Add to dashboard Cite

Satellite hyperspectral imagery is an important data source for large-scale refined land cover classification and mapping, but the high spatial heterogeneity and spectral variability at low spatial resolution and the high computation cost for massive data remain challenges in the research community. In recent years, convolutional neural network (CNN) models with the capability for feature extraction have been widely used in hyperspectral image classification. However, incomplete feature extraction, inappropriate feature fusion, and high time consumption are still the major problems for CNN applications in large-scale fine land cover mapping. In this study, a Spatial-Convolution Spectral-Transformer Interactive Network (SCSTIN) was proposed to integrate 2D-CNN and Transformer into a dual-branch network to enhance feature extraction capabilities by exploring spatial context information and spectral sequence signatures in a targeted manner. In addition, spatial-spectral interactive fusion (SSIF) units and category-adaptive weighting (CAW) as two feature fusion modules were also adopted between and after the two feature extraction branches to improve efficiency in feature fusion. The ZY1-02D hyperspectral imagery was collected to conduct the experiments in the study area of the eastern foothills of the Helan Mountains (EFHLM), covering an area of about 8800 km2, which is the largest hyperspectral dataset as far as we know. To explore the potential of the proposed network in terms of accuracy and efficiency, SCSTIN models with different depths (SCSTIN-4 and SCSTIN-2) were performed. The results suggest that compared with the previous eight advanced hyperspectral image classifiers, both SCSTIN models achieved satisfactory performance in accuracy and efficiency aspects with low complexity, where SCSTIN-4 achieved the highest accuracy and SCSTIN-2 obtained higher efficiency. Accordingly, the SCSTIN models are reliable for large-scale fast refined land cover classification and mapping. In addition, the spatial distribution pattern of diverse ground objects in EFHLM is also analyzed.

show abstract

Section: Overview Of the Scstinmentioning

confidence: 99%

Spatial-Convolution Spectral-Transformer Interactive Network for Large-Scale Fast Refined Land Cover Classification and Mapping Based on ZY1-02D Satellite Hyperspectral Imagery

et al. 2023

View full text Add to dashboard Cite

show abstract

“…Considering the above, significant contributions have been made for HSIC, for instance, the Simple and Effective Spatial-Spectral (SESS) approach [31] involves the selection of representative spectral bands, the extraction of spatial characteristics at multiple scales, and the fusion of spatial-spectral features while preserving neighborhood information. In a similar vein, the work by Liu et al [32] introduces HSI transformers, a network that incorporates two transformers to handle both spatial and spectral information. This approach treats HSI pixels as superpixels and maintains spatial information through position encodings.…”

Section: Introductionmentioning

confidence: 99%

EXNet: (2+1)D Extreme Xception Net for Hyperspectral Image Classification

Ghous,

Sarfraz,

Ahmad

et al. 2024

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

3D-CNNs have demonstrated their capability to capture intricate non-linear relationships within Hyperspectral Images (HSIs). However, the computational complexity of 3D CNNs often leads to slower processing speeds, limited generalization, and susceptibility to overfitting. In response to these challenges, this study introduces the concept of depthwise separable convolutions using (2+1)D convolutions as an alternative to traditional 3D convolutions for Hyperspectral Image Classification (HSIC). The study observes that (2+1)D convolutions can effectively approximate the complex relationships represented by 3D convolutions while requiring fewer convolutional operations, thereby reducing the computational overhead associated with classification. Experimental results obtained from benchmark HSI datasets, including Indian Pines, Botswana, Pavia University, and Salinas, demonstrate that the proposed model yields results that are comparable to those achieved by various state-of-the-art models in the existing literature. The source code is available on GitHub github.com/mahmad00/Extreme-Xception-Net-for-HSIC.

show abstract

“…In the past two years, a transformer model combining self-attention has been proposed. In [46], a hyperspectral image transformer in transformer (HSI-TNT) method was proposed. In this method, two transformer deep networks were used to fuse local and global features, and the effectiveness of the method was finally demonstrated through a large number of experiments.…”

Section: Introductionmentioning

confidence: 99%

A Cross-Channel Dense Connection and Multi-Scale Dual Aggregated Attention Network for Hyperspectral Image Classification

Shi

Wang

et al. 2023

Remote Sensing

View full text Add to dashboard Cite

Hyperspectral image classification (HSIC) is one of the most important research topics in the field of remote sensing. However, it is difficult to label hyperspectral data, which limits the improvement of classification performance of hyperspectral images in the case of small samples. To alleviate this problem, in this paper, a dual-branch network which combines cross-channel dense connection and multi-scale dual aggregated attention (CDC_MDAA) is proposed. On the spatial branch, a cross-channel dense connections (CDC) module is designed. The CDC can effectively combine cross-channel convolution with dense connections to extract the deep spatial features of HSIs. Then, a spatial multi-scale dual aggregated attention module (SPA_MDAA) is constructed. The SPA_MDAA adopts dual autocorrelation for attention modeling to strengthen the differences between features and enhance the ability to pay attention to important features. On the spectral branch, a spectral multi-scale dual aggregated attention module (SPE_MDAA) is designed to capture important spectral features. Finally, the spatial spectral features are fused, and the classification results are obtained. The experimental results show that the classification performance of the proposed method is superior to some state-of-the-art methods in small samples and has good generalization.

show abstract

Mapping Coastal Wetlands Using Transformer in Transformer Deep Network on China ZY1-02D Hyperspectral Satellite Images

Cited by 39 publications

References 49 publications

Spatial-Convolution Spectral-Transformer Interactive Network for Large-Scale Fast Refined Land Cover Classification and Mapping Based on ZY1-02D Satellite Hyperspectral Imagery

Spatial-Convolution Spectral-Transformer Interactive Network for Large-Scale Fast Refined Land Cover Classification and Mapping Based on ZY1-02D Satellite Hyperspectral Imagery

EXNet: (2+1)D Extreme Xception Net for Hyperspectral Image Classification

A Cross-Channel Dense Connection and Multi-Scale Dual Aggregated Attention Network for Hyperspectral Image Classification

Contact Info

Product

Resources

About