RGB-NIR Demosaicing Using Deep Residual U-Net

Shopovska, Ivana; Jovanov, Ljubomir; Philips, Wilfried

doi:10.1109/telfor.2018.8611819

Cited by 11 publications

(10 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the table, the methods in the first group are classical methods from the literature with publicly available codes. The method denoted as RGB-NIR-Unet is the model proposed in our prior work [8]. As indicated, this model is larger in number of trainable parameters compared to the models proposed in this paper.…”

Section: Demosaicing Performancementioning

confidence: 95%

“…In our prior work [8] we proposed a neural network-based method for demosaicing raw RGB-NIR images using two different sampling patterns. Based on this work, this paper extends the focus towards data sampling and training procedures.…”

Section: Contributionsmentioning

confidence: 99%

“…In this paper we also decreased the number of network coefficients compared to our prior work [8] by 65% to improve training and run-time speed at the cost of a slight performance decrease. We trained a smaller model by reducing the dimensionality of the convolutional filters.…”

Section: Training Dataset Creationmentioning

confidence: 99%

“…Following our previous work [8], we focus on reconstructing an approximationx of the ideal, full resolution color image x using a convolutional neural network. The goal of the neural network is to perform the inverse of S, which is a non-linear operation over the mosaic input y.…”

Section: Full Resolution Color Image Reconstructionmentioning

confidence: 99%

“…In our prior work [8] the demosaicing model was based on the U-Net architecture [31], modified to include additional skip connections forming residual blocks. The U-Net consists of a down-sampling path and an up-sampling path connected with a bridge of layers and skip connections from the down-sampling to the up-sampling layers.…”

Section: Full Resolution Color Image Reconstructionmentioning

confidence: 99%

See 4 more Smart Citations

Efficient Training Procedures for Multi-Spectral Demosaicing

Shopovska¹,

Jovanov²,

Philips³

2020

Sensors

Self Cite

View full text Add to dashboard Cite

The simultaneous acquisition of multi-spectral images on a single sensor can be efficiently performed by single shot capture using a mutli-spectral filter array. This paper focused on the demosaicing of color and near-infrared bands and relied on a convolutional neural network (CNN). To train the deep learning model robustly and accurately, it is necessary to provide enough training data, with sufficient variability. We focused on the design of an efficient training procedure by discovering an optimal training dataset. We propose two data selection strategies, motivated by slightly different concepts. The general term that will be used for the proposed models trained using data selection is data selection-based multi-spectral demosaicing (DSMD). The first idea is clustering-based data selection (DSMD-C), with the goal to discover a representative subset with a high variance so as to train a robust model. The second is an adaptive-based data selection (DSMD-A), a self-guided approach that selects new data based on the current model accuracy. We performed a controlled experimental evaluation of the proposed training strategies and the results show that a careful selection of data does benefit the speed and accuracy of training. We are still able to achieve high reconstruction accuracy with a lightweight model.

show abstract

Section: Demosaicing Performancementioning

confidence: 95%

Section: Contributionsmentioning

confidence: 99%

Section: Training Dataset Creationmentioning

confidence: 99%

Section: Full Resolution Color Image Reconstructionmentioning

confidence: 99%

Section: Full Resolution Color Image Reconstructionmentioning

confidence: 99%

See 3 more Smart Citations