Hyperspectral Image Classification: An Analysis Employing CNN, LSTM, Transformer, and Attention Mechanism

Abstract

Hyperspectral images contain tens to hundreds of bands, implying a high spectral resolution. This high spectral resolution allows for obtaining a precise signature of structures and compounds that make up the captured scene. Among the types of processing that may be applied to Hyperspectral Images, classification using machine learning models stands out. The classification process is one of the most relevant steps for this type of image. It can extract information using spatial and spectral information and spatial-spectral fusion. Artificial Neural Network models have been gaining prominence among existing classification techniques. They can be applied to data with one, two, or three dimensions. Given the above, this work evaluates Convolutional Neural Network models with one, two, and three dimensions to identify the impact of classifying Hyperspectral Images with different types of convolution. We also expand the comparison to Recurrent Neural Network models, Attention Mechanism, and the Transformer architecture.. Furthermore, a novelty pre-processing method is proposed for the classification process to avoid generating data leaks between training, validation, and testing data. The results demonstrated that using 1 Dimension Convolutional Neural Network (1D-CNN), Long Short-Term Memory (LSTM), and Transformer architectures reduces memory consumption and sample processing time and maintain a satisfactory classification performance up to 99% accuracy on larger datasets. In addition, the Transfomer architecture can approach the 2D-CNN and 3D-CNN architectures in accuracy using only spectral information. The results also show that using two or three dimensions convolution layers improves accuracy at the cost of greater memory consumption and processing time per sample. Furthermore, the pre-processing methodology guarantees the disassociation of training and testing data.

An Evaluation of Convolutional Neural Networks for Lithological Mapping Based on Hyperspectral Images

“…Therefore, the joint integration of spatial-spectral features within neighboring regions as the input can lead to learning more discriminating features than spatial-wise or spectral-wise classification [40][41][42][43]. As an end-to-end deep learning algorithm that can perform representation learning and classification tasks simultaneously, CNNs are designed to learn complex nonlinear distribution features in images by automatically extracting high-level spatial features, which are especially suitable for dealing with high-dimensional hyperspectral images with strong neighboring correlations [44,45]. In this aspect, 3D CNN [46], fully convolutional network [47], convolutional capsule network [48], and multiple deep learning or dual-channel frameworks such as ConvLSTM [49,50], 1D-2D CNN [51], 3D-2D CNN [52,53], convolutional autoencoder [54], graph convolutional network [55], CNN-local discriminant embedding [56], CNN-SAE [57], CNN-transformer learning [58], and joint attention network [59] have been introduced for spectralspatial hyperspectral classification and achieved state-ofthe-art performance.…”

Analisa Perbandingan Algoritma CNN dan LSTM untuk Klasifikasi Pesan Cyberbullying pada Twitter

“…CNN pada dasarnya terdiri dari serangkaian hidden layer yang masing-masing melakukan pemrosesan konvolusi pada data input. Operasi konvolusi juga menambahkan weight atau filter ke data input yang menghasilkan output [30]. Transformer seperti One-Dimensional Convolutional Neural Network (1D-CNN) mulai mendapatkan kepopuleran pada penerapan Natural Language Processing (NLP) [31].…”