Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction

Cai, Yuanhao; Lin, Jing; Hu, Xiaowan; Wang, Haoqian; Yuan, Xin; Zhang, Yulun; Timofte, Radu; Gool, Luc Van

doi:10.1109/cvpr52688.2022.01698

Cited by 131 publications

(89 citation statements)

References 54 publications

(43 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For maize RGB images to HSIs conversion, the HSCNN+ which we chose for maize spectral recovery was compared with several state-of-the-art algorithms ( Zamir et al. (2020) ; Cai et al. (2022) ; Zhao et al.…”

Section: Experiments and Discussionmentioning

confidence: 99%

“…For maize RGB images to HSIs conversion, the HSCNN+ which we chose for maize spectral recovery was compared with several state-of-the-art algorithms (Zamir et al (2020); Cai et al (2022); Zhao et al (2020); Shi et al (2018)). The dataset we used was mentioned in section 2.1, and the test set was strictly never used for training.…”

Section: Evaluation Of Spectral Recovery Qualitymentioning

confidence: 99%

See 1 more Smart Citation

Maize disease detection based on spectral recovery from RGB images

Liu

Zhao

et al. 2022

Front. Plant Sci.

View full text Add to dashboard Cite

Maize is susceptible to infect pest disease, and early disease detection is key to preventing the reduction of maize yields. The raw data used for plant disease detection are commonly RGB images and hyperspectral images (HSI). RGB images can be acquired rapidly and low-costly, but the detection accuracy is not satisfactory. On the contrary, using HSIs tends to obtain higher detection accuracy, but HSIs are difficult and high-cost to obtain in field. To overcome this contradiction, we have proposed the maize spectral recovery disease detection framework which includes two parts: the maize spectral recovery network based on the advanced hyperspectral recovery convolutional neural network (HSCNN+) and the maize disease detection network based on the convolutional neural network (CNN). Taking raw RGB data as input of the framework, the output reconstructed HSIs are used as input of disease detection network to achieve disease detection task. As a result, the detection accuracy obtained by using the low-cost raw RGB data almost as same as that obtained by using HSIs directly. The HSCNN+ is found to be fit to our spectral recovery model and the reconstruction fidelity was satisfactory. Experimental results demonstrate that the reconstructed HSIs efficiently improve detection accuracy compared with raw RGB image in tested scenarios, especially in complex environment scenario, for which the detection accuracy increases by 6.14%. The proposed framework has the advantages of fast, low cost and high detection precision. Moreover, the framework offers the possibility of real-time and precise field disease detection and can be applied in agricultural robots.

show abstract

Section: Experiments and Discussionmentioning

confidence: 99%

Section: Evaluation Of Spectral Recovery Qualitymentioning

confidence: 99%

Maize disease detection based on spectral recovery from RGB images

Liu

Zhao

et al. 2022

Front. Plant Sci.

View full text Add to dashboard Cite

show abstract

“…In 2022, Transformers have been used for spectral SCI reconstruction, from end-to-end network 10,12 to deep unfolding structure, 9,11 in which the nine stage degradation-aware unfolding half-shuffle transformer (DAUHST) has led to best results so far. Meanwhile, a dual-domain learning method has also been proposed and led to good results.…”

Section: A Short History Of Using Deep Learning For Reconstructionmentioning

confidence: 99%

“…Coded aperture snapshot spectral imager (CASSI) 1,2 or spectral snapshot compressive imaging (SCI) 3 has recently coming to its revival due to the advanced deep learning algorithms. [4][5][6][7][8] Most recently, the emerging Transformer [9][10][11][12] has also been used for the spectral SCI reconstruction, which leads to state-of-the-art results on the simulation benchmark and real data captured by our re-built CASSI system. 5 In this manner, the end-to-end compressive spectral imaging system can be built and there is no barrier left to preclude the wide applications of SCI systems in our daily life.…”

Section: Introductionmentioning

confidence: 99%

Recent advances of deep learning for spectral snapshot compressive imaging

Suo

Wu²,

Li³

et al. 2023

Optoelectronic Imaging and Multimedia Technology IX

View full text Add to dashboard Cite

We review the recent deep learning reconstruction algorithms for spectral snapshot compressive imaging (SCI), which used a single shot measurement to capture the three-dimensional (3D, x, y, λ) spectral image. Recent years, deep learning has been the dominant algorithm to conduct reconstruction due to high speed and accuracy. Various frameworks such as end-to-end neural networks, deep unfolding, plug-and-play networks have been developed. Furthermore, the untrained neural networks have also been used. In this paper, we review diverse deep learning methods for spectral SCI. In addition to the aforementioned frameworks, different backbones and network structures including the most recent Transformers are reviewed. Simulation and real data results are presented to compare these methods.

show abstract

“…Significant advances have been made in image denoising with the advent of deep learning. Although deep convolutional neural networks (CNNs) for image enhancement have shown promising results [ 4 , 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 , 13 , 14 , 15 ], several crucial obstacles prohibit their deployment in real-world applications. Because learning-based techniques are typically data-driven, training on a given dataset does not always ensure generalization to real-world scenarios.…”

Section: Introductionmentioning

confidence: 99%

NG-GAN: A Robust Noise-Generation Generative Adversarial Network for Generating Old-Image Noise

Hossain

Lee

2022

Sensors

View full text Add to dashboard Cite

Numerous old images and videos were captured and stored under unfavorable conditions. Hence, old images and videos have uncertain and different noise patterns compared with those of modern ones. Denoising old images is an effective technique for reconstructing a clean image containing crucial information. However, obtaining noisy-clean image pairs for denoising old images is difficult and challenging for supervised learning. Preparing such a pair is expensive and burdensome, as existing denoising approaches require a considerable number of noisy-clean image pairs. To address this issue, we propose a robust noise-generation generative adversarial network (NG-GAN) that utilizes unpaired datasets to replicate the noise distribution of degraded old images inspired by the CycleGAN model. In our proposed method, the perception-based image quality evaluator metric is used to control noise generation effectively. An unpaired dataset is generated by selecting clean images with features that match the old images to train the proposed model. Experimental results demonstrate that the dataset generated by our proposed NG-GAN can better train state-of-the-art denoising models by effectively denoising old videos. The denoising models exhibit significantly improved peak signal-to-noise ratios and structural similarity index measures of 0.37 dB and 0.06 on average, respectively, on the dataset generated by our proposed NG-GAN.

show abstract

Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction

Cited by 131 publications

References 54 publications

Maize disease detection based on spectral recovery from RGB images

Maize disease detection based on spectral recovery from RGB images

Recent advances of deep learning for spectral snapshot compressive imaging

NG-GAN: A Robust Noise-Generation Generative Adversarial Network for Generating Old-Image Noise

Contact Info

Product

Resources

About