Hyperspectral Image Super Resolution Using Anomaly Weighted Gabor Based CNN

Abstract

Hyperspectral images have high spectral resolution. But, due to the tradeoff between spectral and spatial resolution and various hardware constraints, imaging a hyperspectral image with high spatial resolution is not practical. Hyperspectral super resolution is a soft approach to solve this challenge. Recently, deep learning based methods such as convolutional neural network (CNN) show great success in this field. But, the contextual details in object boundaries and anomalies present in the scene are not well addressed. To this end, a new CNN based framework is proposed for hyperspectral image super resolution in this work. To improve ability of the convolutional blocks in simultaneous extraction of spectral and spatial characteristics, the weighted Gabor features are concatenated in output of the defined convolutional blocks. To extract more details containing anomalous targets present in the scene, the anomaly scores of pixels are calculated and used for weighting the Gabor features. The experiments on three real hyperspectral images acquired by AVIRIS and ROSIS sensors show superior performance of the proposed framework compared to several state-of-the-art methods based on CNN and residual networks. In addition to common super resolution metrics such as SAM and ERGAS, the efficiency of different methods are evaluated according to the classification accuracy metrics such as overall accuracy and kappa coefficient. The overall classification accuracy is increased from 70.39 to 88.23 in Indian dataset, from 86.07 to 96.20 in Pavia University dataset, and from 95.82 to 99.12 in Pavia center dataset.