Hyperspectral Image Classification with Capsule Network Using Limited Training Samples

Abstract

Deep learning techniques have boosted the performance of hyperspectral image (HSI) classification. In particular, convolutional neural networks (CNNs) have shown superior performance to that of the conventional machine learning algorithms. Recently, a novel type of neural networks called capsule networks (CapsNets) was presented to improve the most advanced CNNs. In this paper, we present a modified two-layer CapsNet with limited training samples for HSI classification, which is inspired by the comparability and simplicity of the shallower deep learning models. The presented CapsNet is trained using two real HSI datasets, i.e., the PaviaU (PU) and SalinasA datasets, representing complex and simple datasets, respectively, and which are used to investigate the robustness or representation of every model or classifier. In addition, a comparable paradigm of network architecture design has been proposed for the comparison of CNN and CapsNet. Experiments demonstrate that CapsNet shows better accuracy and convergence behavior for the complex data than the state-of-the-art CNN. For CapsNet using the PU dataset, the Kappa coefficient, overall accuracy, and average accuracy are 0.9456, 95.90%, and 96.27%, respectively, compared to the corresponding values yielded by CNN of 0.9345, 95.11%, and 95.63%. Moreover, we observed that CapsNet has much higher confidence for the predicted probabilities. Subsequently, this finding was analyzed and discussed with probability maps and uncertainty analysis. In terms of the existing literature, CapsNet provides promising results and explicit merits in comparison with CNN and two baseline classifiers, i.e., random forests (RFs) and support vector machines (SVMs).

Polyphonic Sound Event Detection by Using Capsule Neural Networks

“…Furthermore, the network trainable parameters are reduced with respect to other deep learning architectures, confirming the architectural advantage given by the introduced features also in the task of polyphonic SED. The results we observed in this work are consistent with many other classification tasks in various domains [44]- [46] and they prove that the CapsNet is an effective approach which enhances the well-established representation capabilities of the CNNs also in the audio field. However, several aspects still remain unexplored and require further studies: the robustness of CapsNets to overlapping signals (i.e., images or sounds) has been demonstrated in this work as well as in [23].…”

Remote Sensing Image Scene Classification Using CNN-CapsNet

“…CapsNet is a completely novel deep learning architecture, which is robust to affine transformation [41]. In CapsNet, a capsule is defined as a vector that consists of a group of neurons, whose parameters can represent various properties of a specific type of entity that is presented in an image, such as position, size, and orientation.…”

“…school dense residential Recently, the advent of the capsule network (CapsNet) [36], which is a novel architecture to encode the properties and spatial relationship of the features in an image and is a more effective image recognition algorithm, shows encouraging results on image classification. Although the CapsNet is still in its infancy [37], it has been successfully applied in many fields [38][39][40][41][42][43][44][45][46][47][48][49] in recent years, such as brain tumor classification, sound event detection, object segmentation, and hyperspectral image classification. The CapsNet uses a group of neurons as a capsule to replace a neuron in the traditional neural network.…”

A Capsule based Approach for Polyphonic Sound Event Detection

“…Recently, capsule networks have been proposed for remote sensing classification tasks, though their applications primarily involve high-resolution satellite imagery. Deng et al (2018) achieved a 20% relative improvement in land use classification accuracy with capsule networks over CNN, additionally finding that capsule networks required far fewer labelled training data points to achieve high confidence in predictions. Paoletti et al (2019) also employ capsule networks to classify land use in high resolution satellite imagery, finding that the architecture greatly outperforms deep that of a deep CNN.…”