Reconstructing Spectral Images from RGB-Images Using a Convolutional Neural Network

Stiebei, Tarek; Koppers, Simon; Seltsam, Philipp; Merhof, Dorit

doi:10.1109/cvprw.2018.00140

Cited by 47 publications

(47 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to evaluate the proposed methodology, distinct network architectures from the current state-of-the-art methods are considered: the HSCNN+R [ 17 ], an adopted UNet [ 18 ], the adaptive weight attention network (AWAN) [ 15 ] and the pixel-aware deep function-mixture network (FMNet) [ 13 ]. The respective code is publicly available for all individual network architectures.…”

Section: Methodsmentioning

confidence: 99%

“…In following such an approach, it is possible to rely on a single, unified workflow for training and evaluation. All methods but the modified UNet [ 18 ] rely on ensemble strategies to further push their performance, mostly self-ensemble and model-ensemble.…”

Section: Methodsmentioning

confidence: 99%

“…The recent NTIRE2020 challenge on spectral reconstruction from RGB [ 11 ] as well as its predecessor in 2018 [ 12 ] provide a concise overview over the state-of-the-art methods. In summary, the best performing methods in terms of spectral reconstruction quality exclusively consist of complex convolutional neural networks [ 13 , 14 , 15 , 16 , 17 , 18 , 19 ]. A major limitation of deep learning based approaches for SSR is their susceptibility to a varying brightness, i.e., signal scale.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Brightness Invariant Deep Spectral Super-Resolution

Stiebel

Merhof

2020

Sensors

Self Cite

View full text Add to dashboard Cite

Spectral reconstruction from RGB or spectral super-resolution (SSR) offers a cheap alternative to otherwise costly and more complex spectral imaging devices. In recent years, deep learning based methods consistently achieved the best reconstruction quality in terms of spectral error metrics. However, there are important properties that are not maintained by deep neural networks. This work is primarily dedicated to scale invariance, also known as brightness invariance or exposure invariance. When RGB signals only differ in their absolute scale, they should lead to identical spectral reconstructions apart from the scaling factor. Scale invariance is an essential property that signal processing must guarantee for a wide range of practical applications. At the moment, scale invariance can only be achieved by relying on a diverse database during network training that covers all possibly occurring signal intensities. In contrast, we propose and evaluate a fundamental approach for deep learning based SSR that holds the property of scale invariance by design and is independent of the training data. The approach is independent of concrete network architectures and instead focuses on reevaluating what neural networks should actually predict. The key insight is that signal magnitudes are irrelevant for acquiring spectral reconstructions from camera signals and are only useful for a potential signal denoising.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Brightness Invariant Deep Spectral Super-Resolution

Stiebel

Merhof

2020

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Indeed, it is well known that faces, chlorophyl (in foliage) and daylights have very characteristic shapes (amongst other scene features). Of the current developments, deep neural networks [3], [24], [36], [15], [35] provide the leading performance in spectral reconstruction.…”

Section: Related Workmentioning

confidence: 99%

“…Spectral reconstruction (SR) is an alternative approach to recording hyperspectral information, where hyperspectral images are recovered from RGB images [30], [3], [26], [24], [36], [15], [7], [26], [22], [4], [1], [15], [35], [5]. The idea is not as naïve as it might first appear.…”

Section: Introductionmentioning

confidence: 99%

Exposure Invariance in Spectral Reconstruction from RGB Images

Lin

Finlayson

2019

cic

View full text Add to dashboard Cite

Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images. Moreover, this spectral reconstruction problem (SR) can often be solved with good (low) error. However, these methods are not physically plausible: that is when the recovered spectra are reintegrated with the underlying camera sensitivities, the resulting predicted RGB is not the same as the actual RGB, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most learningbased SR models train for a fixed exposure setting and we show that this can result in poor performance when exposure varies.In this paper we show how CNN learning can be extended so that physical plausibility is enforced and the problem resulting from changing exposures is mitigated. Our SR solution improves the state-of-the-art spectral recovery performance under varying exposure conditions while simultaneously ensuring physical plausibility (i.e. the recovered spectra reintegrate to the input RGBs exactly).

show abstract

Extending function mixture network for improved spectral super-resolution

Hussain,

Lall

2023

Computer Vision and Image Understanding

View full text Add to dashboard Cite

Reconstructing Spectral Images from RGB-Images Using a Convolutional Neural Network

Cited by 47 publications

References 7 publications

Brightness Invariant Deep Spectral Super-Resolution

Brightness Invariant Deep Spectral Super-Resolution

Exposure Invariance in Spectral Reconstruction from RGB Images

Extending function mixture network for improved spectral super-resolution

Contact Info

Product

Resources

About