3D Expansion of SRCNN for Spatial Enhancement of Hyperspectral Remote Sensing Images

Abstract

Hyperspectral Imagery (HSI) have high spectral resolution but suffer from low spatial resolution due to sensor tradeoffs. This limitation hinders utilizing the full potential of HSI. Single Image Super Resolution (SISR) techniques can be used to enhance the spatial resolution of HSI. Since these techniques rely on estimating missing information from one Low Resolution (LR) HSI, they are considered ill-posed. Furthermore, most spatial enhancement techniques cause spectral distortions in the estimated High Resolution (HR) HSI. This paper deals with the extension and modification of Convolutional Neural Networks (CNNs) to enhance HSI while preserving their spectral fidelity. The proposed method is tested, evaluated, and compared against other methodologies quantitatively using Peak Signal-tonoise Ratio (PSNR), Structural Similarity Index Measurement (SSIM), and Spectral Angle Mapper (SAM).

SISR of Hyperspectral Remote Sensing Imagery Using 3D Encoder-Decoder RUNet Architecture

“…To prevent spatial reduction, the input image must be padded at the borders. The simplest padding method is adding zeros to the boarders of the image [10].…”

“…Due to the success of Deep Convolutional Neural Networks (DCNNs) in image classification in 2014 [8], researchers studied DCNNs in the context of SISR, which prevailed over traditional methods, such as bicubic interpolation [9]. Currently, the literature is rich with DCNN-based SISR methods for both MSI and HSI [10]- [16]. Researchers argue that 2D DCNNs that are commonly used for MSI cannot be used for HSI enhancement due to the main difference between these two types of images; the spectral resolution [10].…”

“…Currently, the literature is rich with DCNN-based SISR methods for both MSI and HSI [10]- [16]. Researchers argue that 2D DCNNs that are commonly used for MSI cannot be used for HSI enhancement due to the main difference between these two types of images; the spectral resolution [10]. 2D DCNNs fail to preserve the spectral fidelity of HSI.…”

A Comparative Study of Loss Functions for Hyperspectral SISR

“…It is important to enhance HSI while preserving their spectral fidelity, which can be achieved by 3D CNNs due to their ability to successfully capture spectral context. There are many studies in the literature that utilize 3D CNNs for this purpose [9]- [14]. However, the studies do not investigate the effect of loss functions on the performance of the CNN.…”

“…The studied loss functions are Mean Squared Error (MSE), Mean Absolute Error (MAE), Mean Squared Logarithmic Error (MSLE), Log Hyperbolic Cosine (LHC), Huber, Charbonier, Cosine Similarity (CS), and a proposed hybrid loss function. The network chosen to perform this study is 3D-SRCNN, which was previously proposed in [14] and has proven efficiency against other state- is performed based on Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM) [17], and Spectral Angle Mapper (SAM), which are the most frequently used evaluation metrics for HSI. PSNR and SSIM measure the spatial similarity between the Ground Truth (GT) image and the estimated (enhanced) one, while SAM measures the similarity between the spectra of the GT image and that of the estimated one.…”