Spectral Super-Resolution with Optimized Bands

Abstract

Hyperspectral (HS) sensors sample reflectance spectrum in very high resolution, which allows us to examine material properties in very fine details. However, their widespread adoption has been hindered because they are very expensive. Reflectance spectra of real materials are high dimensional but sparse signals. By utilizing prior information about the statistics of real HS spectra, many previous studies have reconstructed HS spectra from multispectral (MS) signals (which can be obtained from cheaper, lower spectral resolution sensors). However, most of these techniques assume that the MS bands are known apriori and do not optimize the MS bands to produce more accurate reconstructions. In this paper, we propose a new end-to-end fully convolutional residual neural network architecture that simultaneously learns both the MS bands and the transformation to reconstruct HS spectra from MS signals by analyzing large quantity of HS data. The learned band can be implemented in hardware to obtain an MS sensor that collects data that is best to reconstruct HS spectra using the learned transformation. Using a diverse set of real-world datasets, we show how the proposed approach of optimizing MS bands along with the transformation can drastically increase the reconstruction accuracy. Additionally, we also investigate the prospects of using reconstructed HS spectra for land cover classification.

Deeply learned broadband encoding stochastic hyperspectral imaging

“…However, their hardware still suffers from the large volume due to the coded mask and prism. For the compact random filter design, some deep learning approaches have been developed for both the target response definition 18,19 and inverse design [20][21][22][23] . These methods work to some extent; however, none of them comprehensively considered both procedures.…”

Brightness Invariant Deep Spectral Super-Resolution

“…Finally, the argument can be made that the RGB normalization layer might not be necessary when a scale invariant loss function is considered. Examples might be the spectral angular error, spectral information divergence [ 24 ] or spectral derivative based loss functions [ 25 ]. It should be noted that from a purely theoretical point of view there is no guarantee that training a network with a loss function that is invariant to changes in brightness will make the fully-trained network invariant to changes in brightness.…”

Deep‐Learned Broadband Encoding Stochastic Filters for Computational Spectroscopic Instruments

“…Emerging deep learning‐based solutions provide an alternative that might be conducive to the BEST‐SI design. Aiming at the procedure of target response definition, several works [ 14,15 ] have added artificial priors, e.g., Gaussian‐shaped or smoothness limit, to avoid converging to the white‐noise‐like spectral responses. The others [ 16–19 ] have enhanced the optical design precision under certain conditions.…”