Combining Spectral Unmixing and 3D/2D Dense Networks with Early-Exiting Strategy for Hyperspectral Image Classification

Abstract

Recently, Hyperspectral Image (HSI) classification methods based on deep learning models have shown encouraging performance. However, the limited numbers of training samples, as well as the mixed pixels due to low spatial resolution, have become major obstacles for HSI classification.To tackle these problems, we propose a resource-efficient HSI classification framework which introduces adaptive spectral unmixing into a 3D/2D dense network with early-exiting strategy. More specifically, on one hand, our framework uses a cascade of intermediate classifiers throughout the 3D/2D dense network that is trained end-to-end. The proposed 3D/2D dense network that integrates 3D convolutions with 2D convolutions is more capable of handling spectral-spatial features, while containing fewer parameters compared with the conventional 3D convolutions, and further boosts the network performance with limited training samples. On another hand, considering the existence of mixed pixels in HSI data, the pixels in HSI classification are divided into hard samples and easy samples. With the early-exiting strategy in these intermediate classifiers, the average accuracy can be improved by reducing the amount of computation cost for easy samples, thus focusing on classifying hard samples. Furthermore, for hard samples, an adaptive spectral unmixing method is proposed as a complementary source of information for classification, which brings considerable benefits to the final performance. Experimental results on four HSI benchmark datasets demonstrate that the proposed method can achieve better performance than state-of-the-art deep learning-based methods and other traditional HSI classification methods.

Patch-Wise Hyperspectral Image Classification Using Composite 3d-2d Convolutional Neural Network Feature Hierarchy

“…The huge feedforward DNN utilizing deep three-dimensional CNN with simulated inputs delivers by far the best results. In recent years, there has been a significant advance Hyperspectral image classification [17], where the spatial characteristics are adjusted by a 2D CNN architecture [18], [19]. However, these spatial elements are often retrieved individually, which negates the need to simultaneously use spatialspectral features for classification of hyperspectral images.…”

Protein–Protein Docking: Past, Present, and Future

“…The efficiency of 2D and 3D CNNs [124][125][126] are proved undoubtedly through many works in image processing. DOcking decoy selection with Voxel-based deep neural nEtwork (DOVE) [127] is the first attempt to test the applicability of 3D CNNs in protein-protein docking.…”

SpectralNET: Exploring Spatial-Spectral WaveletCNN for Hyperspectral Image Classification

“…In [4], 2D-3D CNN has been utilized with multi band feature fusion mechanism. This mechanism allows them to fuse both shallow and deep features in spectral band, which improves the feature vector sent into the dense layer.The work proposed in [19], introduces adaptive spectral unmixing into a 3D-2D CNN along with a early exit strategy. The early exit strategy reduces computational cost for easy samples.…”