Small Sample Hyperspectral Image Classification Based on Cascade Fusion of Mixed Spatial-Spectral Features and Second-Order Pooling

Abstract

Hyperspectral images can capture subtle differences in reflectance of features in hundreds of narrow bands, and its pixel-wise classification is the cornerstone of many applications requiring fine-grained classification results. Although three-dimensional convolutional neural networks (3D-CNN) have been extensively investigated in hyperspectral image classification tasks and have made significant breakthroughs, hyperspectral classification under small sample conditions is still challenging. In order to facilitate small sample hyperspectral classification, a novel mixed spatial-spectral features cascade fusion network (MSSFN) is proposed. First, the covariance structure of hyperspectral data is modeled and dimensionality reduction is conducted using factor analysis. Then, two 3D spatial-spectral residual modules and one 2D separable spatial residual module are used to extract mixed spatial-spectral features. A cascade fusion pattern consisting of intra-block feature fusion and inter-block feature fusion is constructed to enhance the feature extraction capability. Finally, the second-order statistical information of the fused features is mined using second-order pooling and the classification is achieved by the fully connected layer after L2 normalization. On the three public available hyperspectral datasets, Indian Pines, Huston, and University of Pavia, only 5%, 3%, and 1% of the labeled samples were used for training, the accuracy of MSSFN in this paper is 98.52%, 96.30% and 98.83%, respectively, which is far better than the contrast models and verifies the effectiveness of MSSFN in small sample hyperspectral classification tasks.

Low-Rank Constrained Attention-Enhanced Multiple Spatial–Spectral Feature Fusion for Small Sample Hyperspectral Image Classification

“…The single-type CNN models can be improved by using a combination of 3D convolution and 2D convolution to form a mixed CNN model [25,30,41]. The basic structure of a mixed convolutional network can be divided into four basic building blocks, namely, 3D convolution, reshape, 2D convolution, and fully connected layers [21].…”

“…In addition, recently acquired hyperspectral images exhibit high spatial resolution and further enhance the texture details, which presents both opportunities and challenges for HSIC tasks. The literature [41] has validated the effectiveness of factor analysis (FA) as a DR method using some classical hyperspectral datasets. In this paper, FA will be adopted on more hyperspectral datasets with high spatial resolution to further validate its heterogeneous noise modeling capability.…”

“…Feng et al extracted spatial-spectral features using factor analysis and a cascade feature fusion architecture with parallel convolutional kernels of different sizes. Second-order pooling and L2 normalization are utilized to exploit the higher-order statistics of features extracted by CNN models [41]. In addition, the underlying convolution operation is being revisited and improved.…”

An Improved 3D-2D Convolutional Neural Network Based on Feature Optimization for Hyperspectral Image Classification

“…However, it is difficult to obtain samples of hyperspectral images, which requires a lot of manpower and material resources [7]. Therefore, the current research mainly focuses on how to extract more discriminant features from fewer samples [8], [9], [10].…”

“…Therefore, scholars have proposed to use 2D convolution to further extract spatial features to improve feature quality [18]. In this regard, Feng et al [10] used 3D convolution and 2D separable convolution to construct dense blocks, and extracted spatial spectrum and spatial features in turn. Zhang et al [8] optimized the 3D-2D hybrid convolutional neural network model, which uses spectral and spatial attention mechanisms to refine the extracted features, and achieved excellent results in a small number of samples.…”