NTIRE 2022 Spectral Demosaicing Challenge and Data Set

Arad, Boaz; Timofte, Radu; Yahel, Rony; Morag, Nimrod; Bernat, Amir; Wu, Yaqi; Wu, Xun; Fan, Zhihao; Xia, Chenjie; Zhang, Feng; Liu, Shuai; Li, Yongqiang; Feng, Chaoyu; Lei, Lei; Zhang, Mingwei; Feng, Kai; Zhang, Xun; Yao, Jiaxin; Zhao, Yue; Ma, Suina; He, Fazhi; Dong, Yangyang; Shi, Yu; Qiu, Difa; Liu, Jinhui; Bi, Mengzhao; Song, Biao; Sun, WenFang; Zheng, Jiesi; Zhao, Bowen; Yang, Jiangxin; Cao, Yanan; Kong, Xiangyu; Yu, Jianhua; Xue, Yuanyang; Xie, Zheng

doi:10.1109/cvprw56347.2022.00103

Cited by 35 publications

(17 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Demosaicing. Demosaicing has been a well-established field with many great works proposed [23,29,35,47,15,6,5]. The general idea is to use the measured signals at sparse locations to fill up the missing values in neighboring locations.…”

Section: Related Workmentioning

confidence: 99%

Jointly Learning Band Selection and Filter Array Design for Hyperspectral Imaging

Dai²,

Gool

2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

A single-shot multispectral camera equipped with an optimized color filter array (CFA) has the potential to deliver a fast and low-cost hyperspectral (HS) imaging system. Previous solutions are largely restricted to designing demosaicing algorithms for fixed CFAs -be it the Bayer color pattern or evenly-spaced spectral multiplexing patterns. Since sampling and reconstruction are tightly-coupled, the ability to search for an optimal solution is severely constrained by using predefined CFAs. In this work, we simultaneously address the problem of spectral band selection, CFA design, image demosaicing, and spectral image recovery in a joint learning framework for single-shot HS imaging. We propose a reinforcement learning (RL) based method for spectral band selection and a novel neural network for CFA generation, image demosaicing, and HS image recovery. The final spectral reconstruction accuracy is used to supervise the training of the main network to maximize the synergies between those tightly-related tasks. The RL method regards the main network as an agent to collect reward. Our final method delivers a simple setup -as simple as an RGB camera -for HS imaging. Experimental results show that our method outperforms competing methods by a large margin.

show abstract

Section: Related Workmentioning

confidence: 99%

Jointly Learning Band Selection and Filter Array Design for Hyperspectral Imaging

Dai²,

Gool

2023

2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

show abstract

“…This data set is further expanded for the NTIRE 2022 Spectral Demosaic Challenge [6], where 16 channel spectral images are provided over a 400-1000nm range -covering a wider range of wavelengths at a reduced spectral resolution.…”

Section: Data Setmentioning

confidence: 99%

“…This challenge is one of the NTIRE 2022 associated challenges: spectral recovery [7], spectral demosaicing [6], perceptual image quality assessment [19], inpainting [40], night photography rendering [17], efficient super-resolution [31], learning the super-resolution space [34], super-resolution and quality enhancement of compressed video [49], high dynamic range [38], stereo super-resolution [45], burst super-resolution [8].…”

Section: Introductionmentioning

confidence: 99%

NTIRE 2022 Spectral Recovery Challenge and Data Set

Arad¹,

Timofte²,

Yahel³

et al. 2022

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)

Self Cite

View full text Add to dashboard Cite

This paper reviews the third biennial challenge on spectral reconstruction from RGB images, i.e., the recovery of whole-scene hyperspectral (HS) information from a 3-channel RGB image. This challenge presents the "ARAD 1K" data set: a new, larger-than-ever natural hyperspectral image data set containing 1,000 images. Challenge participants were required to recover hyperspectral information from synthetically generated JPEGcompressed RGB images simulating capture by a known calibrated camera, operating under partially known parameters, in a setting which includes acquisition noise. The challenge was attended by 241 teams, with 60 teams competing in the final testing phase, 12 of which provided detailed descriptions of their methodology which are included in this report. The performance of these submissions is reviewed and provided here as a gauge for the current stateof-the-art in spectral reconstruction from natural RGB images.

show abstract

“…scanning. To address this issue, snapshot HI systems [3,26], such as computed tomography [18,31] and light-field imaging [4,16], have been introduced recently, which capture both spectral and spatial information rapidly. These snapshot HI systems can be realized by snapshot mosaic HI systems or Multi-Spectral Filter Array (MSFA) cameras [33].…”

Section: Introductionmentioning

confidence: 99%

MSFA-Frequency-Aware Transformer for Hyperspectral Images Demosaicing

Zeng¹,

Feng²,

Shaoguang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Hyperspectral imaging systems that use multispectral filter arrays (MSFA) capture only one spectral component in each pixel. Hyperspectral demosaicing is used to recover the non-measured components. While deep learning methods have shown promise in this area, they still suffer from several challenges, including limited modeling of non-local dependencies, lack of consideration of the periodic MSFA pattern that could be linked to periodic artifacts, and difficulty in recovering high-frequency details. To address these challenges, this paper proposes a novel demosaicing framework, the MSFA-frequency-aware Transformer network (FDM-Net). FDM-Net integrates a novel MSFA-frequency-aware multi-head self-attention mechanism (MaFormer) and a filter-based Fourier zero-padding method to reconstruct high pass components with greater difficulty and low pass components with relative ease, separately. The advantage of Maformer is that it can leverage the MSFA information and non-local dependencies present in the data. Additionally, we introduce a joint spatial and frequency loss to transfer MSFA information and enhance training on frequency components that are hard to recover. Our experimental results demonstrate that FDM-Net outperforms state-of-the-art methods with 6dB PSNR, and reconstructs high-fidelity details successfully.

show abstract

NTIRE 2022 Spectral Demosaicing Challenge and Data Set

Cited by 35 publications

References 38 publications

Jointly Learning Band Selection and Filter Array Design for Hyperspectral Imaging

Jointly Learning Band Selection and Filter Array Design for Hyperspectral Imaging

NTIRE 2022 Spectral Recovery Challenge and Data Set

MSFA-Frequency-Aware Transformer for Hyperspectral Images Demosaicing

Contact Info

Product

Resources

About