Hyperspectral Image Classification via a Novel Spectral–Spatial 3D ConvLSTM-CNN

Abstract

In recent years, deep learning-based models have produced encouraging results for hyperspectral image (HSI) classification. Specifically, Convolutional Long Short-Term Memory (ConvLSTM) has shown good performance for learning valuable features and modeling long-term dependencies in spectral data. However, it is less effective for learning spatial features, which is an integral part of hyperspectral images. Alternatively, convolutional neural networks (CNNs) can learn spatial features, but they possess limitations in handling long-term dependencies due to the local feature extraction in these networks. Considering these factors, this paper proposes an end-to-end Spectral-Spatial 3D ConvLSTM-CNN based Residual Network (SSCRN), which combines 3D ConvLSTM and 3D CNN for handling both spectral and spatial information, respectively. The contribution of the proposed network is twofold. Firstly, it addresses the long-term dependencies of spectral dimension using 3D ConvLSTM to capture the information related to various ground materials effectively. Secondly, it learns the discriminative spatial features using 3D CNN by employing the concept of the residual blocks to accelerate the training process and alleviate the overfitting. In addition, SSCRN uses batch normalization and dropout to regularize the network for smooth learning. The proposed framework is evaluated on three benchmark datasets widely used by the research community. The results confirm that SSCRN outperforms state-of-the-art methods with an overall accuracy of 99.17%, 99.67%, and 99.31% over Indian Pines, Salinas, and Pavia University datasets, respectively. Moreover, it is worth mentioning that these excellent results were achieved with comparatively fewer epochs, which also confirms the fast learning capabilities of the SSCRN.

AgriSen-COG, a Multicountry, Multitemporal Large-Scale Sentinel-2 Benchmark Dataset for Crop Mapping Using Deep Learning

“…Therefore, the network exploits the data's temporal (LSTM) and spatial (convolution) dimensions. This topology is also popular among remote sensing data, in particular for a use case that benefits from both spatial and temporal dimensions such as land cover ( [37][38][39]), soil moisture ( [40,41]), solar radiation ( [42]), air quality [43], and others, including crop type mapping ( [44][45][46]). In [47], the authors propose a more efficient version of ConvLSTM, named ConvSTAR.…”

Decoupled Knowledge Distillation via Spatial Feature Blurring for Hyperspectral Image Classification

“…In IP, PU, Salinas, and WHU-Hi-LongKou datasets, a quantitative evaluation of the comprehensive classification performance was conducted. Evaluation of the overall performance of the DC net (with added SFB module), the Pre_DC net (without the SFB module), and other classical nets: support vector machine (SVM) [57], 2D-CNN [16], 3D-CNN [58], and hybrid spectral CNN (HybridSN) [19]. As shown in Table VII, within the IP datasets, the classes '1 Alfalfa,' '7 Grass-Pasture-mowed,' and '9 Oats' remained as small-sample classes.…”

Hyperspectral Image Classification Based on Fusion of Convolutional Neural Network and Graph Network

“…Hyperspectral image classification aims to classify each pixel point in the image [6]. In the early days, most HSI classification methods mainly relied on some traditional machine learning algorithms [7] which were mainly divided into two processes: traditional manual feature engineering and classifier classification [8]. Feature engineering aims to process data based on knowledge so that the processed features can be better used in subsequent classification algorithms.…”