Unsupervised Deep Hyperspectral Super-Resolution With Unregistered Images

Nie, Jiangtao; Zhang, Lei; Wei, Wei; Zhang, Yanning

doi:10.1109/icme46284.2020.9102881

Cited by 10 publications

(10 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Using both quantitative metrics and perceptive consistency, their results proved that their proposed methodology outperforms the state-of-the-art. Nie et al [102] argued that the success of the existing HSI super-resolution methods based on fusion depends on the premise that the images used for fusion (i.e., the HSI lowspatial-resolution input and the multispectral image with low-spectral resolution) are exactly registered. While such a premise is too idealistic for the real world, few efforts have taken this issue into account.…”

Section: Hyperspectral Image Super Resolutionmentioning

confidence: 99%

“…While such a premise is too idealistic for the real world, few efforts have taken this issue into account. As a solution to this, Nie et al [102] proposed the integration of image registration into HSI super-resolution for joint unsupervised learning. To learn the parameters of the affine transformation between the two input images, they used a UNNP-based spatial transformer network (STN) that avoids overfitting by constraining the STN.…”

Section: Hyperspectral Image Super Resolutionmentioning

confidence: 99%

See 1 more Smart Citation

Untrained Neural Network Priors for Inverse Imaging Problems: A Survey

Qayyum

Ilahi

Shamshad

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

In recent years, advancements in machine learning (ML) techniques, in particular, deep learning (DL) methods have gained a lot of momentum in solving inverse imaging problems, often surpassing the performance provided by hand-crafted approaches. Traditionally, analytical methods have been used to solve inverse imaging problems such as image restoration, inpainting, and superresolution. Unlike analytical methods for which the problem is explicitly defined and the domain knowledge is carefully engineered into the solution, DL models do not benefit from such prior knowledge and instead make use of large datasets to predict an unknown solution to the inverse problem. Recently, a new paradigm of training deep models using a single image, named untrained neural network prior (UNNP) has been proposed to solve a variety of inverse tasks, e.g., restoration and inpainting. Since then, many researchers have proposed various applications and variants of UNNP. In this paper, we present a comprehensive review of such studies and various UNNP applications for different tasks and highlight various open research problems which require further research.

show abstract

Section: Hyperspectral Image Super Resolutionmentioning

confidence: 99%

Section: Hyperspectral Image Super Resolutionmentioning

confidence: 99%

Untrained Neural Network Priors for Inverse Imaging Problems: A Survey

Qayyum

Ilahi

Shamshad

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

show abstract

“…Nie et al [67] argued that the success of the existing HSI super-resolution methods based on fusion depends on the premise that the images used for fusion (i.e., the HSI lowspatial-resolution input and the multispectral image with low-spectral resolution) are exactly registered. While such a premise is too idealistic for real-world, few efforts have taken this issue into account.…”

Section: Hyperspectral Image Super Resolutionmentioning

confidence: 99%

“…While such a premise is too idealistic for real-world, few efforts have taken this issue into account. As a solution to this, Nie et al [67] proposed the integration of image registration into HSI super-resolution for joint unsupervised learning. To learn the parameters of the affine transformation between the two input images, they implement a UNNP-based spatial transformer network (STN) that avoids overfitting by constraining the STN.…”

Section: Hyperspectral Image Super Resolutionmentioning

confidence: 99%

Untrained Neural Network Priors for Inverse Imaging Problems: A Survey

Qayyum¹,

Ilahi²,

Shamshad³

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…These methods usually perform better than SISR approaches, but heavily rely on the complex imaging system and careful calibration to ensure precise alignment, which weakens its effectiveness for practical applications. To alleviate the strong assumption of existing fusion-based approaches, some recent works [17,18,19,20] begin to take into account the misalignment of RGB reference images, e.g., Fu et al [17] propose an alternating direction method of multipliers (ADMM)-based method for solving HSI super-resolution with rigid geometric misaligned RGB reference, Qu et al [19] implicitly learn to correlate the spatial-spectral information from unregistered multimodality images through an unsupervised framework, and applies to the geometric misaligned images and reference images collected from a different time and sources, Zheng et al [20] propose a NonRegSRNet that considers more complex misalignment by randomly shifting some pixels of the aligned reference. Nevertheless, most of these methods are still limited at deal with complex misalignments and they are often restricted to unsupervised approaches due to the lack of real-world unaligned datasets.…”

Section: Introductionmentioning

confidence: 99%

Hyperspectral Image Super-Resolution with RGB Image Super-Resolution as an Auxiliary Task

Dai²,

Gool

2022

2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)

View full text Add to dashboard Cite

This work studies Hyperspectral image (HSI) superresolution (SR). HSI SR is characterized by highdimensional data and a limited amount of training examples. This raises challenges for training deep neural networks that are known to be data hungry. This work addresses this issue with two contributions. First, we observe that HSI SR and RGB image SR are correlated and develop a novel multi-tasking network to train them jointly so that the auxiliary task RGB image SR can provide additional supervision and regulate the network training. Second, we extend the network to a semi-supervised setting so that it can learn from datasets containing only low-resolution HSIs. With these contributions, our method is able to learn hyperspectral image super-resolution from heterogeneous datasets and lifts the requirement for having a large amount of high resolution (HR) HSI training samples. Extensive experiments on three standard datasets show that our method outperforms existing methods significantly and underpin the relevance of our contributions. Our code can be found at https://github.com/kli8996/HSISR.git.

show abstract

Unsupervised Deep Hyperspectral Super-Resolution With Unregistered Images

Cited by 10 publications

References 18 publications

Untrained Neural Network Priors for Inverse Imaging Problems: A Survey

Untrained Neural Network Priors for Inverse Imaging Problems: A Survey

Untrained Neural Network Priors for Inverse Imaging Problems: A Survey

Hyperspectral Image Super-Resolution with RGB Image Super-Resolution as an Auxiliary Task

Contact Info

Product

Resources

About