Spectral Image Fusion From Compressive Measurements Using Spectral Unmixing and a Sparse Representation of Abundance Maps

Abstract

In the past years, one common way of enhancing the spatial resolution of a hyperspectral (HS) image has been to fuse it with complementary information coming from multispectral (MS) or panchromatic images. This paper proposes a new method for reconstructing a high-spatial, high-spectral image from measurements acquired after compressed sensing by multiple sensors of different spectral ranges and spatial resolutions, with specific attention to HS and MS compressed images. To solve this problem, we introduce a fusion model based on the linear spectral unmixing model classically used for HS images and investigate an optimization algorithm based on a block coordinate descent strategy. The nonnegative and sum-to-one constraints resulting from the intrinsic physical properties of abundances as well as a total variation penalization are used to regularize this ill-posed inverse problem. Simulation results conducted on realistic compressed HS and MS images show that the proposed algorithm can provide fusion results that are very close to those obtained with uncompressed images, with the advantage of using a significantly reduced number of measurements.

A review of various image fusion types and transforms

“…PAN and MS modes are usually used simultaneously in order for some information of the object not to be lost. Multi-sensor image fusion is the fusion of different satellite images [8], [9]. There is another fusion procedure called HSF in which multispectral images with high spatial resolutions enhance spatially a hyperspectral image.…”

Deep-Fusion: An End-To-End Approach for Compressive Spectral Image Fusion

“…CSI fusion methods solve this issue by using two measurements from different CSI architectures, one allowing high-spatial-resolution reconstruction and the other with a high-spectral resolution of the estimated data cube [16] to recover a high-spectral and high-spatial resolution data cube. CSI fusion methods use prior information of the spectral images as a regularization term in the optimization problem such as low-rank [17], total variation, and sparsity [16] or using a low-dimensional representation relying on the linear mixture model [18]. On the other hand, deep learning (DL) approaches have been proposed to fuse MSI and HSI data, without considering CSI projections; for instance, [19,20,21] propose two neural networks (NN) to independently extract spatial features from MSI and spectral features from HIS, which are then fused or concatenated into a single branch.…”

Adaptive Multisensor Acquisition via Spatial Contextual Information for Compressive Spectral Image Classification

“…Furthermore, different variants of the CASSI has been proposed such as the dual disperser CASSI (DD-CASSI) [18], 3D-CASSI [16], and colored-CASSI [19]. In addition, a multi-sensor CSI architecture was recently used to recover high-resolution spectral images from hyperspectral and multispectral compressive samples [20].…”