S2Former: Parallel Spectral–Spatial Transformer for Hyperspectral Image Classification

Abstract

Due to their excellent representation talent in local features, the convolutional neural network (CNN) has achieved favourable performance in hyperspectral image (HSI) classification tasks. Nevertheless, current CNN models exhibit a marked flaw: they are hard to model the dependencies in long-range distanced positions. This flaw becomes more problematic for the HSI classification task, which targets extracting more discriminative features in local and global dimensions from limited samples. In this paper, we introduce a spatial–spectral transformer (S2Former), which explores spatial and spectral feature extraction in a dual-stream framework for HSI Classification. S2Former, which consists of a spatial transformer and a spectral transformer in parallel branches, extracts the discriminative feature in spatial and spectral dimensions. More specifically, we propose multi-head spatial self-attention to capture the long-range spatial dependency of non-adjacent HSI pixels in a spatial transformer. In the spectral transformer, we propose multi-head covariance spectral attention to mine and represent spectral signatures by computing covariance-based channel maps. Meanwhile, the local activation feed-forward network is developed to complement local details. Extensive experiments conducted on four publicly available datasets indicate that our S2Former achieves state-of-the-art performance for the HSI classification task.

A Hybrid-Scale Feature Enhancement Network for Hyperspectral Image Classification

“…A hyperspectral image (HSI) possesses plentiful spectral and spatial information, which has been widely adopted in extensive application areas, such as precision agriculture [3], environmental monitoring [4], mineral exploration [5], and urban planning [6]. HSI classification has become a research hotspot in pattern recognition and image processing, which is devoted to assigning a unique category label to each spatial pixel [7][8][9][10]. However, HSI classification is still a challenging issue, i.e., especially spatial variability and the curse of dimensionality, thereby increasing the difficulty of classification.…”