Nonlocal Coupled Tensor CP Decomposition for Hyperspectral and Multispectral Image Fusion

Abstract

Hyperspectral super-resolution, which aims at enhancing the spatial resolution of hyperspectral images (HSIs), has recently attracted considerable attention. A common way of hyperspectral super-resolution is to fuse the HSI with a higher spatial resolution multispectral image (MSI). Various approaches have been proposed to solve this problem by establishing the degradation model of low spatial resolution HSIs and MSIs based on matrix factorization methods, e.g., unmixing and sparse representation. However, this category of approaches cannot well construct the relationship between the high spatial resolution HSI and MSI. In fact, since the HSI and the MSI capture the same scene, these two image sources must have common factors. In this paper, we propose a nonlocal tensor decomposition model for HSI-MSI fusion. Firstly, the nonlocal similar patch tensors of the HSI are constructed according to the MSI, for the purpose of calculating the smooth order of all the patches for clustering. Then, the relationship between the high spatial resolution HSI and the MSI is explored through a coupled tensor canonical polyadic (CP) decomposition. The fundamental idea of the proposed model is that the factor matrices in CP decomposition of the high spatial resolution HSI's nonlocal tensor can be shared with the matrices factorized by the MSI's nonlocal tensor. Alternating direction method of multipliers is used to solve the proposed model. Through this method, the spatial structure of the MSI can be successfully transferred to the HSI. Experimental results on three synthetic datasets and one real dataset suggest that the proposed method substantially outperforms existing state-of-the-art HSI-MSI fusion methods. Index Terms-e Hyperspectral images, multispectral images, data fusion, nonlocal tensor, coupled CP decomposition.

Blind Hyperspectral and Multispectral Image Fusion Using Coupled Non-Negative Tucker Tensor Decomposition

“…There are several attempts aiming to increase the spatial resolution of HSIs in recent years [1][2][3][4][5][6][7][8][9][10][11][12][13]. Basically, all super-resolution HSI approaches can be described in three categories; fusing low resolution hyperspectral image (LR-HSI) and high resolution multispectral image (HR-MSI) as a Bayesian framework [3,11,[14][15][16][17][18][19], non-negative matrix factorization (NMF) based methods [4, [20][21][22][23][24][25] and tensor factorization based methods [1,2,8,[26][27][28][29]. Bayesian frameworks build the posterior distribution based on observed LR-HSI and HR-MSI and some prior information or regularization term and utilize the alternating direction method of multipliers (ADMM) [11] optimization method to estimate super-resolution HSI.…”

“…Noteworthy, stacking a 3D data into matrix form in NMFbased approaches loses the neighborhood structures, smoothness, and continuity characteristics. In this regard, tensors or multiway arrays have been frequently used in multidimensional data analysis [26,28,[34][35][36]. Additionally, exploiting the power of multilinear algebra of the tensor representation shows more flexibility in the choice of constraints that match data properties.…”

Low-Rank Tensor Decomposition With Smooth and Sparse Regularization for Hyperspectral and Multispectral Data Fusion

“…For example, Kanatsoulis et al [27] proposed a coupled tensor factorization framework (called STEREO) to overcome the aforementioned problems, which needed little information of the degradation operators. Xu et al [28] brought forward a nonlocal tensor decomposition model for HSI-MSI fusion by exploring the relationship between the LR-HSI and the HR-MSI via a coupled CP decomposition. Li et al [29] redefined the fusion problem as an estimation of the core tensor and dictionaries of the three modes, and proposed a coupled sparse tensor factorization (CSTF) method.…”

Hyperspectral Image Super-Resolution Based on Tensor Spatial-Spectral Joint Correlation Regularization

“…Although the matrix-factorization based HSR methods were effective to some extent, they broke the cubic data structure of HSIs and MSIs, and inevitably lost some information. Since the multi-band HSIs and MSIs can be substantially represented by third-order tensors [20], many tensor-based methods have been proposed for HSR [21]- [27]. Dian et al [21] proposed a non-local sparse tensor factorization method that decomposed each cube of HSI as a sparse core tensor and three dictionaries, and grouped the similar cubes to exploit the non-local spatial self-similarities of the HSI.…”

“…Dian et al [21] proposed a non-local sparse tensor factorization method that decomposed each cube of HSI as a sparse core tensor and three dictionaries, and grouped the similar cubes to exploit the non-local spatial self-similarities of the HSI. The works in [24] and [27] solved the HSR problem from the the tensor canonical polyadic (CP) decomposition point of view, and the method in [27] also considered the non-local similarities exist in the HSI. Except for the tensor CP decomposition, tensor Tucker decomposition was also used for HSR methods.…”