Spectral–Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network

Abstract

Abstract:Recent research has shown that using spectral-spatial information can considerably improve the performance of hyperspectral image (HSI) classification. HSI data is typically presented in the format of 3D cubes. Thus, 3D spatial filtering naturally offers a simple and effective method for simultaneously extracting the spectral-spatial features within such images. In this paper, a 3D convolutional neural network (3D-CNN) framework is proposed for accurate HSI classification. The proposed method views the HSI cube data altogether without relying on any preprocessing or post-processing, extracting the deep spectral-spatial-combined features effectively. In addition, it requires fewer parameters than other deep learning-based methods. Thus, the model is lighter, less likely to over-fit, and easier to train. For comparison and validation, we test the proposed method along with three other deep learning-based HSI classification methods-namely, stacked autoencoder (SAE), deep brief network (DBN), and 2D-CNN-based methods-on three real-world HSI datasets captured by different sensors. Experimental results demonstrate that our 3D-CNN-based method outperforms these state-of-the-art methods and sets a new record.

3D Convolutional Neural Networks for Crop Classification with Multi-Temporal Remote Sensing Images

“…The latter may be caused by the lack of more spectral bands or the inevitable noise in a "standard" spectral curve of a certain material, which is a common case in multi-spectral images. However, the processing of spatio-spectral information is not the focus of this study, and other recent articles could be referred [32]. Third, we discuss more on the performance comparison of 2D CNN, 3D CNN, and SVM in pixelwise classification, which is the most common case in remote sensing applications other than the training-test procedure on discrete samples [8,12].…”

“…Recently, some studies have utilized 3D CNN for learning spatio-temporal features from videos [29,30], learning 3D structures from LiDAR point clouds [31], or learning spatio-spectral presentations from hyperspectral images [32]. In general, 3D CNN is not as widely applied as 2D CNN, as the temporal dimension is usually not considered in computer vision and machine learning.…”

A Fast Dense Spectral–Spatial Convolution Network Framework for Hyperspectral Images Classification

“…For an input image with three channels of spectral information (Band1, Band2, and Band3), a 3D convolution processes the data from three channels using two convolution kernels to obtain two characteristic maps, as shown in Figure 1b. Within the neural network, the value V xyz i,j at position (x, y, z) on the jth feature cube in the ith layer can be formulated as follows [12]:…”

“…The key to these operations is the size of the convolution kernel, because features determine accuracy. Taking [12] as an example, a convolution kernel of 3 × 3 × 7 or similar size is used to learn the spectral and spatial features at the same time. Distinguished from obtaining the spectral and spatial features together, the proposed framework uses the CNN to learn the spectral and spatial features separately to extract more discriminative features.…”

“…Moreover, the FDSSC framework uses a smaller training sample size than SAE-LR and Makantasis's method, while achieving higher accuracy. (2) It has a deeper structure than the SSFC framework, and the methods in [11,12] which are CNN-based deep learning methods. A deeper structured CNN can learn more useful deep spatial and spectral features, which leads to extremely high accuracy, and thus the FDSSC framework has better performance than previous methods.…”