BS2T: Bottleneck Spatial–Spectral Transformer for Hyperspectral Image Classification

Song, Ruoxi; Feng, Yining; Wei, Cheng; Mu, Zhenhua; Wang, Xianghai

doi:10.1109/tgrs.2022.3185640

Cited by 41 publications

(17 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Comparison Methods: To verify the effectiveness of the proposed S2Former, seven state-of-the-art HSI classifiers are used for comparison, including SVM [38], CDCNN [22], FDSSC [27], DBDA [30], SpectralFormer [34], HSI-Mixer [36], and BS2T [35].…”

Section: Experimental Settingsmentioning

confidence: 99%

“…A new backbone network named SpectralFormer [34] combines the advantages of transformer and CNN, aiming to learn local spectral feature representation and feature transfer between shallow and deep layers. Song et al [35] designed a Bottleneck Spatial-Spectral Transformer (BS2T) to describe the dependencies between HSI pixels over long-term locations and bands. HSI-Mixer [36] uses a simple CNN architecture to simulate the function of transformer, reconsiders the significant inductive bias of convolution.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

S2Former: Parallel Spectral–Spatial Transformer for Hyperspectral Image Classification

Yuan,

Yu,

Qian

et al. 2023

Electronics

View full text Add to dashboard Cite

Due to their excellent representation talent in local features, the convolutional neural network (CNN) has achieved favourable performance in hyperspectral image (HSI) classification tasks. Nevertheless, current CNN models exhibit a marked flaw: they are hard to model the dependencies in long-range distanced positions. This flaw becomes more problematic for the HSI classification task, which targets extracting more discriminative features in local and global dimensions from limited samples. In this paper, we introduce a spatial–spectral transformer (S2Former), which explores spatial and spectral feature extraction in a dual-stream framework for HSI Classification. S2Former, which consists of a spatial transformer and a spectral transformer in parallel branches, extracts the discriminative feature in spatial and spectral dimensions. More specifically, we propose multi-head spatial self-attention to capture the long-range spatial dependency of non-adjacent HSI pixels in a spatial transformer. In the spectral transformer, we propose multi-head covariance spectral attention to mine and represent spectral signatures by computing covariance-based channel maps. Meanwhile, the local activation feed-forward network is developed to complement local details. Extensive experiments conducted on four publicly available datasets indicate that our S2Former achieves state-of-the-art performance for the HSI classification task.

show abstract

Section: Experimental Settingsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

S2Former: Parallel Spectral–Spatial Transformer for Hyperspectral Image Classification

Yuan,

Yu,

Qian

et al. 2023

Electronics

View full text Add to dashboard Cite

show abstract

“…For example, Sun et al 17 proposed Spectral-Spatial Attention Network (SSAN), in which attention was used to search the effective features of HSI cubes. Song's and Wang's researches 18,19 also revealed that spectral-spatial methods can better extract the information in HSI.…”

Section: Introductionmentioning

confidence: 99%

Unsupervised multi-branch capsule for hyperspectral and LiDAR classification

Tang

She

2023

Ninth Symposium on Novel Photoelectronic Detection Technology and Applications

View full text Add to dashboard Cite

With the convenient availability of remote sensing data, how to make models to interpret complex remote sensing data attracts wide attention. In remote sensing data, hyperspectral images contain spectral information and LiDAR contains elevation information. Hence, more explorations are warranted to better fuse the features of different source data. In this paper, we introduce semantic understanding to dynamically fuse data from two different sources, extract features of HSI and LiDAR through different capsule network branches and improve self-supervised loss and random rigid rotation in canonical capsule to a high-dimensional situation. Canonical capsule computes the capsule decomposition of objects by permutation-equivariant attention and the process is self-supervised by training pairs of randomly rotated objects. After fusing the features of HSI and LiDAR with semantic understanding, the unsupervised extraction of spectral-spatial-elevation fusion features is achieved. With two real-world examples of HSI and LiDAR fused, the experimental results show that the proposed multi-branch high-dimensional canonical capsule algorithm can be effective for semantic understanding of HSI and LiDAR. It indicates that the model can extract HSI and LiDAR data features effectively as opposed to existing models for unsupervised extraction of multi-source RS data.

show abstract

“…Recently, Transformer-based methods were proposed to learn long-distance information, which achieves good experimental performance in many computer vision fields, involving semantic segmentation [27][28][29][30], image classification [31][32][33][34], object detection [35][36][37][38], and super-resolution [39][40][41][42]. In ref.…”

Section: Introductionmentioning

confidence: 99%

A hybrid U‐shaped and transformer network for change detection in high‐resolution remote sensing images

Wu,

Yuan,

Zhan

2024

IET Image Processing

View full text Add to dashboard Cite

Deep convolutional neural networks based remote sensing change detection has recently shown significant performance improvement. However, small region changes and global‐local features in high‐resolution remote sensing images are not fully explored. This paper introduces a hybrid U‐shaped and transformer network for change detection in high‐resolution remote sensing images. Specifically, a UNet++‐based backbone to facilitate feature learning across different scales. In addition, we introduce a transformer‐based feature fusion module for extracting long‐range dependencies, which can enhance the representation ability of the network. Furthermore, the introduced efficient channel attention mechanism can efficiently calibrate the feature representation and concentrate on more important feature information. Thanks to the above designs, the proposed method enjoys a strong ability to extract local and global features for remote sensing change detection. Extensive experimental results on different remote sensing images show that our method can achieve superior performance in comparison with state‐of‐the‐art change detection methods.

show abstract

BS2T: Bottleneck Spatial–Spectral Transformer for Hyperspectral Image Classification

Cited by 41 publications

References 50 publications

S2Former: Parallel Spectral–Spatial Transformer for Hyperspectral Image Classification

S2Former: Parallel Spectral–Spatial Transformer for Hyperspectral Image Classification

Unsupervised multi-branch capsule for hyperspectral and LiDAR classification

A hybrid U‐shaped and transformer network for change detection in high‐resolution remote sensing images

Contact Info

Product

Resources

About