Efficient spectral reconstruction using a trichromatic camera via sample optimization

Li, Yuqi; Wang, Chong; Zhao, Jieyu; Yuan, Qingshu

doi:10.1007/s00371-017-1469-3

Cited by 14 publications

(8 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The training set we used is 100,000 spectral image patches selected from each dataset using volume maximization strategy [13]. Applying K-svd algorithm, we learned a multispectral patch dictionary consisting of q=3000 atoms for sparse reconstruction, where each atom is a whl×1 vector.…”

Section: Resultsmentioning

confidence: 99%

“…Here spectral reconstruction is an operation that recovering spectrum at the position of each pixel from incomplete samples output from a MSFA sensor. The most widely applied solution first obtains the full resolution image of multiple channels through multispectral demosaicing [10], and then recovers spectral information of each pixel using the obtained multiple responses [11,12,13]. Recently, some works have demonstrated the great capability of sparse coding either for color demosaicing [14,15] or for spectral recovery [16].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction

Xie

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

Multispectral filter array (MSFA)-based imaging is a compact, practical technique for snapshot spectral image capturing and reconstruction. The imaging and reconstruction quality is highly influenced by the spectral sensitivities and spatial arrangement of channels on MSFAs, and the used reconstruction method. In order to design a MSFA with high imaging capacity, we propose a sparse representation based approach to optimize spectral sensitivities and spatial arrangement of MSFAs. The proposed approach first overall models the various errors associated with spectral reconstruction, and then uses a global heuristic searching method to optimize MSFAs via minimizing the estimated error of MSFAs. Our MSFA optimization method can select filters from off-the-shelf candidate filter sets while assigning the selected filters to the designed MSFA. Experimental results on three datasets show that the proposed method is more efficient, flexible, and can design MSFAs with lower spectral construction errors when compared with existing state-of-the-art methods. The MSFAs designed by our method show better performance than others even using different spectral reconstruction methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction

Xie

et al. 2019

Sensors

Self Cite

View full text Add to dashboard Cite

show abstract

“…Usually this involves both spatial and spectral interpolation, based on a transformation learned from large collections of training samples. The data redundancy problem has been addressed with different sample optimization techniques [17][18][19], with a common requirement of maximizing diversity among the selected samples. In [20], the samples are selected based on local color and texture descriptions of the neighborhood around each pixel, and the results show that including texture improves the quality of the optimized dataset.…”

Section: Related Workmentioning

confidence: 99%

Efficient Training Procedures for Multi-Spectral Demosaicing

Shopovska¹,

Jovanov²,

Philips³

2020

Sensors

View full text Add to dashboard Cite

The simultaneous acquisition of multi-spectral images on a single sensor can be efficiently performed by single shot capture using a mutli-spectral filter array. This paper focused on the demosaicing of color and near-infrared bands and relied on a convolutional neural network (CNN). To train the deep learning model robustly and accurately, it is necessary to provide enough training data, with sufficient variability. We focused on the design of an efficient training procedure by discovering an optimal training dataset. We propose two data selection strategies, motivated by slightly different concepts. The general term that will be used for the proposed models trained using data selection is data selection-based multi-spectral demosaicing (DSMD). The first idea is clustering-based data selection (DSMD-C), with the goal to discover a representative subset with a high variance so as to train a robust model. The second is an adaptive-based data selection (DSMD-A), a self-guided approach that selects new data based on the current model accuracy. We performed a controlled experimental evaluation of the proposed training strategies and the results show that a careful selection of data does benefit the speed and accuracy of training. We are still able to achieve high reconstruction accuracy with a lightweight model.

show abstract

“…This is achieved by adding a regularization term which calculates second derivative vectors, imposing smoothness condition for the sensitivities [8]. Li et al [19] proposed to learn an optimized training set of RGB-hyperspectral pairs and use radial basis function interpolation to infer spectrum of a given image while assuming the spectral power distributions of illumination is known.…”

Section: Related Workmentioning

confidence: 99%

Towards Spectral Estimation from a Single RGB Image in the Wild

Kaya

Can

Timofte

2019

2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)

View full text Add to dashboard Cite

In contrast to the current literature, we address the problem of estimating the spectrum from a single common trichromatic RGB image obtained under unconstrained settings (e.g. unknown camera parameters, unknown scene radiance, unknown scene contents). For this we use a reference spectrum as provided by a hyperspectral image camera, and propose efficient deep learning solutions for sensitivity function estimation and spectral reconstruction from a single RGB image. We further expand the concept of spectral reconstruction such that to work for RGB images taken in the wild and propose a solution based on a convolutional network conditioned on the estimated sensitivity function. Besides the proposed solutions, we study also generic and sensitivity specialized models and discuss their limitations. We achieve state-of-the-art competitive results on the standard example-based spectral reconstruction benchmarks: ICVL, CAVE, NUS and NTIRE. Moreover, our experiments show that, for the first time, accurate spectral estimation from a single RGB image in the wild is within our reach. 1

show abstract

Efficient spectral reconstruction using a trichromatic camera via sample optimization

Cited by 14 publications

References 34 publications

Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction

Optimized Multi-Spectral Filter Arrays for Spectral Reconstruction

Efficient Training Procedures for Multi-Spectral Demosaicing

Towards Spectral Estimation from a Single RGB Image in the Wild

Contact Info

Product

Resources

About