Deep Recursive Network for Hyperspectral Image Super-Resolution

Wei, Wei; Nie, Jiangtao; Li, Yong; Zhang, Lei; Zhang, Yanning

doi:10.1109/tci.2020.3014451

Cited by 52 publications

(18 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Fusion of HR-RGB and LR-MSIs is another commonly used approach in spectral image super-resolution. One advantage of fusion strategies over the direct reconstruction ones, e.g., HSCNNR, is that higher quality HR-MSIs can be generated when the super-resolution scale is high (often ≥ 8) [56]. However, the final quality of generated HR-MSIs should not be affected if the super-resolution scale is low, e.g., only 4 in our study.…”

Section: Discussionmentioning

confidence: 83%

Deep-Learning-Based Multispectral Image Reconstruction from Single Natural Color RGB Image—Enhancing UAV-Based Phenotyping

et al. 2022

View full text Add to dashboard Cite

Multispectral images (MSIs) are valuable for precision agriculture due to the extra spectral information acquired compared to natural color RGB (ncRGB) images. In this paper, we thus aim to generate high spatial MSIs through a robust, deep-learning-based reconstruction method using ncRGB images. Using the data from the agronomic research trial for maize and breeding research trial for rice, we first reproduced ncRGB images from MSIs through a rendering model, Model-True to natural color image (Model-TN), which was built using a benchmark hyperspectral image dataset. Subsequently, an MSI reconstruction model, Model-Natural color to Multispectral image (Model-NM), was trained based on prepared ncRGB (ncRGB-Con) images and MSI pairs, ensuring the model can use widely available ncRGB images as input. The integrated loss function of mean relative absolute error (MRAEloss) and spectral information divergence (SIDloss) were most effective during the building of both models, while models using the MRAEloss function were more robust towards variability between growing seasons and species. The reliability of the reconstructed MSIs was demonstrated by high coefficients of determination compared to ground truth values, using the Normalized Difference Vegetation Index (NDVI) as an example. The advantages of using “reconstructed” NDVI over Triangular Greenness Index (TGI), as calculated directly from RGB images, were illustrated by their higher capabilities in differentiating three levels of irrigation treatments on maize plants. This study emphasizes that the performance of MSI reconstruction models could benefit from an optimized loss function and the intermediate step of ncRGB image preparation. The ability of the developed models to reconstruct high-quality MSIs from low-cost ncRGB images will, in particular, promote the application for plant phenotyping in precision agriculture.

show abstract

Section: Discussionmentioning

confidence: 83%

Deep-Learning-Based Multispectral Image Reconstruction from Single Natural Color RGB Image—Enhancing UAV-Based Phenotyping

et al. 2022

View full text Add to dashboard Cite

show abstract

“…To enhance the interpretability of deep learning-based approaches, constructing a structured deep network by deep unfolding of the iterative algorithm has been used [56]- [64], [67], [68], and the iterative algorithms used include ADMM [58], [59], [68], projected gradient descent [60], proximal gradient [61], [67], half quadratic splitting [62], [63] and iterative shrinkage thresholding [64]. Approaches to HS/MS image fusion include a concise fusion model incorporating a linear representation of the target image followed by a projected gradient method to solve the model with a deep network constructed by unfolding the corresponding iterative algorithm [61]; an iterative formula for HS/MS image fusion according to an observation with detailed compensation processes leading to construction of a structured deep network by unfolding the iterative formula [69]; taking the original unified optimization model with two fidelity terms and one regularization term and splitting the model into three suboptimization problems via a half quadratic splitting algorithm then using a recursive residual network for the sub-problem associated with the regularization term and unfolding the two sub-problems associated with the fidelity terms into network representations [63].…”

Section: B Deep Learning-based Approachesmentioning

confidence: 99%

Patch-Aware Deep Hyperspectral and Multispectral Image Fusion by Unfolding Subspace-Based Optimization Model

Liu

Shen

et al. 2022

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Hyperspectral and multispectral image fusion aims to fuse a low-spatial-resolution hyperspectral image (HSI) and a high-spatial-resolution multispectral image to form a highspatial-resolution HSI. Motivated by the success of model-and deep learning-based approaches, we propose a novel patch-aware deep fusion approach for HSI by unfolding a subspace-based optimization model, where moderate-sized patches are used in both training and test phases. The goal of this approach is to make full use of the information of patch under subspace representation, restrict the scale and enhance the interpretability of the deep network, thereby improving the fusion. First, a subspacebased fusion model was built with two regularization terms to localize pixels and extract texture. Then, the subspace-based fusion model was solved by the alternating direction method of multipliers algorithm, and the model was divided into one fidelitybased problem and two regularization-based problems. Finally, a structured deep fusion network was proposed by unfolding all steps of the algorithm as network layers. Specifically, the fidelity-based problem was solved by a gradient descent algorithm and implemented by a network. The two regularization-based problems were described by proximal operators and learnt by two u-shaped architectures. Moreover, an aggregation fusion technique was proposed to improve the performance by averaging the fused images in all iterations and aggregating the overlapping patches in the test phase. Experimental results, conducted on both synthetic and real datasets, demonstrated the effectiveness of the proposed approach.

show abstract

“…The mean square error (MSE) and peak signal-to-noise-ratio (PSNR) are common methods used to test and compare the algorithms used for transforming image into highvisibility image [5]. Authors stated that hyperspectral image (HSI) method produces high-visibility pictures by amalgamation of low-visibility HIS and a high-visibility ordinary image [6]. Methods should be combined with actual requirements.…”

Section: Literature Surveymentioning

confidence: 99%

Analysis and Prediction of COVID-19 with Image Super-Resolution Using CNN and SRCNN-Based Approach

Jain

Kosamkar

2021

ICT With Intelligent Applications

View full text Add to dashboard Cite

The advancement of technology has created a huge scope and requirement for transforming the image into a high-visibility image. Here, transforming an image into a high-visibility image means to convert a low-visibility image into a high-visibility image. Super-resolution has many applications worldwide like in medical industries, surveillance, satellite photography, the study of the galaxy, etc. Also, COVID-19 is a monstrous threat to earth. Doctors are predicting whether the patient is having the coronavirus or not via X-Rays and CT-Scans. These scans sometimes miss little details because of the blurriness/low-visibility of the image. This problem can be overcome by using the Super-resolution convolutional neural network (SRCNN). The purpose of the study is the classification of whether the person has the coronavirus or not becomes very accurate by using the SRCNN model. For the transformation of a low-visibility image into a high-visibility image, SRCNN and for classifying whether the person is having coronavirus or not a convolutional neural network (CNN) is used. Our models are trained and tested on four datasets, which are Set5, Set14, COVID-chest-X-Ray dataset, and chest-X-Ray-pneumonia. Our results depict that after applying super-resolution on the X-Rays or the CT-Scans, the classification of COVID-19 attained an accuracy of 95.31% which is higher if compared to the classification of COVID-19 without image super-resolution that attained an accuracy of 92.19%. These were the results after running the model for 20 epochs. Hence, with the help of the SRCNN model, the classification of COVID-19 is much easier and accurate as compared to without the image super-resolution technique.

show abstract

Deep Recursive Network for Hyperspectral Image Super-Resolution

Cited by 52 publications

References 43 publications

Deep-Learning-Based Multispectral Image Reconstruction from Single Natural Color RGB Image—Enhancing UAV-Based Phenotyping

Deep-Learning-Based Multispectral Image Reconstruction from Single Natural Color RGB Image—Enhancing UAV-Based Phenotyping

Patch-Aware Deep Hyperspectral and Multispectral Image Fusion by Unfolding Subspace-Based Optimization Model

Analysis and Prediction of COVID-19 with Image Super-Resolution Using CNN and SRCNN-Based Approach

Contact Info

Product

Resources

About