Deep Hyperspectral Image Fusion Network With Iterative Spatio-Spectral Regularization

Huang, Tao; Dong, Weisheng; Wu, Jinjian; Li, Leida; Li, Xin; Shi, Guang

doi:10.1109/tci.2022.3152700

Cited by 37 publications

(16 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…b) Supervised methods: supervised methods [39]- [41] do not reconstruct high-resolution HSI with a known degradation model, instead, they construct the training set of HSIs and RGB images using hyperspectral and RGB cameras, and train neural networks to learn the fusion for the specific devices. Recently some supervised methods are proposed to jointly learn the degradation model and the fusion from training set, and iteratively improve the reconstruction results by the multi-blocks neural networks [33], [35]- [37]. However, these supervised methods rely on large-scale high-quality training sets, otherwise, the training sets are synthesized using the ground truth of degradation parameters.…”

Section: Related Workmentioning

confidence: 99%

BUSIFusion: Blind Unsupervised Single Image Fusion of Hyperspectral and RGB Images

Wang

et al. 2023

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

Hyperspectral images (HSIs) provide rich spectral information that has been widely used in numerous computer vision tasks. However, their low spatial resolution often prevents their use in applications such as image segmentation and recognition. Fusing low-resolution HSIs with high-resolution RGB images to reconstruct high-resolution HSIs has attracted great research attention recently. In this paper, we propose an unsupervised blind fusion network that operates on a single HSI and RGB image pair and requires neither known degradation models nor any training data. Our method takes full advantage of an unrolling network and coordinate encoding to provide a state-ofthe-art HSI reconstruction. It can also estimate the degradation parameters relatively accurately through the neural representation and implicit regularization of the degradation model. The experimental results demonstrate the effectiveness of our method both in simulations and in our real experiments. The proposed method outperforms other state-of-the-art nonblind and blind fusion methods on two popular HSI datasets. Our related code and data is available at https://github.com/CPREgroup/Real-Spec-RGB-Fusion.

show abstract

Section: Related Workmentioning

confidence: 99%

BUSIFusion: Blind Unsupervised Single Image Fusion of Hyperspectral and RGB Images

Wang

et al. 2023

IEEE Trans. Comput. Imaging

View full text Add to dashboard Cite

show abstract

“…For example, depth data such as steroid camera images, LiDAR point clouds, and radar signals can be fused to enhance the 3D perception [25]. In hyperspectral imaging, by fusion of low-resolution hyperspectral images (LR-HSI) with high-resolution multispectral images (HR-MSI) [26], or RGB images with the coded aperture snapshot spectral imaging (CASSI) system [27,28], the spectral sensing capability can be significantly improved. The event flows provided by a dynamic vision sensors (DVS) can help to ease the motion blur in the images captured by ordinary cameras [29].…”

Section: Without Optical Codingmentioning

confidence: 99%

Key frames assisted hybrid encoding for photorealistic compressive video sensing

Huang¹,

Teng²,

Yu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Snapshot compressive imaging (SCI) encodes high-speed scene video into a snapshot measurement and then computationally makes reconstructions, allowing for efficient high-dimensional data acquisition. Numerous algorithms, ranging from regularization-based optimization and deep learning, are being investigated to improve reconstruction quality, but they are still limited by the ill-posed and information-deficient nature of the standard SCI paradigm. To overcome these drawbacks, we propose a new key frames assisted hybrid encoding paradigm for compressive video sensing, termed KH-CVS, that alternatively captures short-exposure key frames without coding and long-exposure encoded compressive frames to jointly reconstruct photorealistic video. With the use of optical flow and spatial warping, a deep convolutional neural network framework is constructed to integrate the benefits of these two types of frames. Extensive experiments on both simulations and real data from the prototype we developed verify the superiority of the proposed method.

show abstract

“…Image Super-Resolution (SR) reconstruction gathers a series of techniques whose purpose is to enhance the resolution of a single image or a video sequence. When a single image is the source of information for the enhancement, a number of characteristics of the image can be used to further improve the result of the process, such as features of the main object/s [ 1 ], or the used sensor [ 2 , 3 ], or machine learning techniques [ 4 , 5 , 6 ]. Instead, when the source is a video sequence or a series of non-identical low-resolution (LR) images of the same object, the non-redundant information between those sources can be used to enhance the process of escalation [ 7 ].…”

Section: Introductionmentioning

confidence: 99%

Towards Real-Time Hyperspectral Multi-Image Super-Resolution Reconstruction Applied to Histological Samples

Ortega

Quevedo

Quintana

et al. 2023

Sensors

View full text Add to dashboard Cite

Hyperspectral Imaging (HSI) is increasingly adopted in medical applications for the usefulness of understanding the spectral signature of specific organic and non-organic elements. The acquisition of such images is a complex task, and the commercial sensors that can measure such images is scarce down to the point that some of them have limited spatial resolution in the bands of interest. This work proposes an approach to enhance the spatial resolution of hyperspectral histology samples using super-resolution. As the data volume associated to HSI has always been an inconvenience for the image processing in practical terms, this work proposes a relatively low computationally intensive algorithm. Using multiple images of the same scene taken in a controlled environment (hyperspectral microscopic system) with sub-pixel shifts between them, the proposed algorithm can effectively enhance the spatial resolution of the sensor while maintaining the spectral signature of the pixels, competing in performance with other state-of-the-art super-resolution techniques, and paving the way towards its use in real-time applications.

show abstract

Deep Hyperspectral Image Fusion Network With Iterative Spatio-Spectral Regularization

Cited by 37 publications

References 54 publications

BUSIFusion: Blind Unsupervised Single Image Fusion of Hyperspectral and RGB Images

BUSIFusion: Blind Unsupervised Single Image Fusion of Hyperspectral and RGB Images

Key frames assisted hybrid encoding for photorealistic compressive video sensing

Towards Real-Time Hyperspectral Multi-Image Super-Resolution Reconstruction Applied to Histological Samples

Contact Info

Product

Resources

About