A survey on computational spectral reconstruction methods from RGB to hyperspectral imaging

Abstract

Hyperspectral imaging enables many versatile applications for its competence in capturing abundant spatial and spectral information, which is crucial for identifying substances. However, the devices for acquiring hyperspectral images are typically expensive and very complicated, hindering the promotion of their application in consumer electronics, such as daily food inspection and point-of-care medical screening, etc. Recently, many computational spectral imaging methods have been proposed by directly reconstructing the hyperspectral information from widely available RGB images. These reconstruction methods can exclude the usage of burdensome spectral camera hardware while keeping a high spectral resolution and imaging performance. We present a thorough investigation of more than 25 state-of-the-art spectral reconstruction methods which are categorized as prior-based and data-driven methods. Simulations on open-source datasets show that prior-based methods are more suitable for rare data situations, while data-driven methods can unleash the full potential of deep learning in big data cases. We have identified current challenges faced by those methods (e.g., loss function, spectral accuracy, data generalization) and summarized a few trends for future work. With the rapid expansion in datasets and the advent of more advanced neural networks, learnable methods with fine feature representation abilities are very promising. This comprehensive review can serve as a fruitful reference source for peer researchers, thus paving the way for the development of computational hyperspectral imaging.

Robust mean shift filter for mixed Gaussian and impulsive noise reduction in color digital images

“…Common noise sources include malfunctioning pixel elements in the camera sensors, thermal noise of the photosensitive matrix, shot noise caused by fluctuations in the photon flux, faulty memory locations or bit errors in hardware, timing errors in analog-to-digital converters, noise generated by electromagnetic interference and atmospheric turbulence as well as errors caused by imperfect optics and transmission. The resulting image degradations are frequently modelled as a mixture of additive white Gaussian noise, mainly responsible for dark and shot currents, and impulsive noise generating pixels with random channel intensity values 3 , 9 – 11 .…”

Irradiance Independent Spectrum Reconstruction from Camera Signals Using the Interpolation Method

“…Under the white balance condition, the spectral sensitivity vector and signal value of a signal channel of the RGBF camera are D Cam U = S Cam U /( S White S D65 ) T S Cam U and U = S Reflection T D Cam U , respectively, for U = R , G , B , and F ; and S White is the spectral reflectance vector of a white card. The same white card in [ 28 , 29 ] was taken, which is the white side of a Kodak gray card. Figure 2 shows the vectors D CamR , D CamG , D CamB , and D CamF .…”

“…Methods for estimating the spectral reflectance from camera signals are critical for improving measurement accuracy and detection speed. The methods can be based on basis spectra [ 17 , 18 , 19 , 20 ], Wiener estimation [ 14 , 21 , 22 ], regression [ 23 , 24 , 25 , 26 , 27 , 28 ], and interpolation [ 29 , 30 , 31 , 32 , 33 , 34 ]. The basis-spectrum methods assume that the target spectrum to be reconstructed is a linear combination of the basis spectra derived from training samples.…”

“…For example, the authors of [ 32 ] showed spectral reflectance images reconstructed using the LUT method. The regression method using a deep-learning neural network can reconstruct spectral reflectance images taking into account the spatial structure of the image [ 28 ]. This approach is attractive, although the reconstructed spectra are shown to be less accurate in color [ 26 ].…”

Dynamic Negative Sampling Autoencoder for Hyperspectral Anomaly Detection

“…The spectral dimension of the HSI usually ranges from the visible to the near-infrared wavelength by a step of less than 10 nm. The rich and detailed spectral information is the key for accurate identification of the subtle difference between different objects [2], [3].…”