A Variational Pan-Sharpening Method Based on Spatial Fractional-Order Geometry and Spectral–Spatial Low-Rank Priors

Liu, Pengfei; Xiao, Liang; Li, Tao

doi:10.1109/tgrs.2017.2768386

Cited by 78 publications

(35 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, there is a need to complete the variational model including the data fitting terms relating the spectral and the spatial data in input with the desired and unknown HRMS data. In that model, the PDI plays the role of a regularization term avoiding the use of other additional prior information often used in the related literature, see, e.g., sparse priors [33] and low-rank priors [34]. Having a look at most of the existing VO methods, the design of the spectral fidelity term is almost the same for all the techniques.…”

Section: B Motivationsmentioning

confidence: 99%

A New Variational Approach Based on Proximal Deep Injection and Gradient Intensity Similarity for Spatio-Spectral Image Fusion

Huang

Deng

et al. 2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Pansharpening is a very debated spatio-spectral fusion problem. It refers to the fusion of a high spatial resolution panchromatic (PAN) image with a lower spatial but higher spectral resolution multispectral (LRMS) image in order to obtain an image with high resolution in both the domains. In this paper, we propose a novel variational optimizationbased (VO) approach to address this issue incorporating the outcome of a deep convolutional neural network (DCNN). This solution can take advantages of both the paradigms. On one hand, higher performance can be expected introducing machine learning methods based on the training by examples philosophy into VO approaches. On other hand, the combination of VO techniques with DCNNs can aid the generalization ability of these latter. In particular, we formulate a 2-based proximal deep injection term to evaluate the distance between the DCNN outcome and the desired high spatial resolution multispectral image. This represents the regularization term for our VO model. Furthermore, a new data fitting term measuring the spatial fidelity is proposed. Finally, the proposed convex VO problem is efficiently solved by exploiting the framework of the alternating direction method of multipliers, thus guaranteeing the convergence of the algorithm. Extensive experiments both on simulated and real datasets demonstrate that the proposed approach can outperform state-of-the-art spatio-spectral fusion methods, even showing a significant generalization ability. Please find the project page: https://liangjiandeng.github.io/Projects Res/DMPIF 2020jstars.html.

show abstract

Section: B Motivationsmentioning

confidence: 99%

A New Variational Approach Based on Proximal Deep Injection and Gradient Intensity Similarity for Spatio-Spectral Image Fusion

Huang

Deng

et al. 2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

show abstract

“…Traditionally, spatial-spectral image fusion methods fuse an LR HSI with an HR panchromatic image (single band), such as pansharpening [9]. As we know, the famous pansharpening methods include intensity-hue-saturation (IHS) [10,11], high-frequency information injection [12,13], and model-based methods [14][15][16]. Owing to the limited spectral resolution of the panchromatic image, these methods often produce some spectral distortions.…”

Section: Related Workmentioning

confidence: 99%

“…However, it is not effective to enhance spatial resolution by improving the imaging quality of the hyperspectral sensors, and a breakthrough in hardware will be difficult and costly. Alternatively, HSI super-resolution (SR) has been proposed to generate a high spatial resolution (HR) HSI by fusing a high spectral resolution image, such as HSI, with an image, such as panchromatic image [9][10][11][12][13][14][15][16] or a multispectral image (MSI) [17][18][19][20][21][22][23][24][25][26][27][28][29][30].…”

Section: Introductionmentioning

confidence: 99%

Hyperspectral Image Super-Resolution via Adaptive Dictionary Learning and Double ℓ 1 Constraint

Tang

Huang

et al. 2019

Remote Sensing

View full text Add to dashboard Cite

Hyperspectral image (HSI) super-resolution (SR) is an important technique for improving the spatial resolution of HSI. Recently, a method based on sparse representation improved the performance of HSI SR significantly. However, the spectral dictionary was learned under a fixed size, empirically, without considering the training data. Moreover, most of the existing methods fail to explore the relationship among the sparse coefficients. To address these crucial issues, an effective method for HSI SR is proposed in this paper. First, a spectral dictionary is learned, which can adaptively estimate a suitable size according to the input HSI without any prior information. Then, the proposed method exploits the nonlocal correlation of the sparse coefficients. Doubleregularized sparse representation is then introduced to achieve better reconstructions for HSI SR. Finally, a high spatial resolution HSI is generated by the obtained coefficients matrix and the learned adaptive size spectral dictionary. To evaluate the performance of the proposed method, we conduct experiments on two famous datasets. The experimental results demonstrate that it can outperform some relatively state-of-the-art methods in terms of the popular universal quality evaluation indexes.

show abstract

“…P AN-SHARPENING [1], [2], [3] refers to the fusion of a low spatial resolution multispectral image (LR-MSI) and a high spatial resolution panchromatic (PAN) image to obtain a high spatial resolution MSI (HR-MSI). The demand for pan-sharpening is increasing since most remote sensing applications [4], [5], [6], [7] require images to be high in both spatial and spectral resolutions.…”

Section: Introductionmentioning

confidence: 99%

A Two-Stream Multiscale Deep Learning Architecture for Pan-Sharpening

Jie

Cai

et al. 2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

Pan-sharpening, which fuses the high resolution panchromatic (PAN) image and the low resolution multispectral image (MSI), is a hot topic in remote sensing. Recently, deep learning technology has been successfully applied in pansharpening. However, the existing methods ignore that the MSI and PAN image are at different resolution and use the same networks to extract features of the two images. To address this problem, we propose a two-stream deep learning architecture, called coupled multi-scale convolutional neural network (CMC), for pan-sharpening. The proposed network has 3 components, feature extraction subnetworks, fusion layer, and super-resolution subnetwork. In the feature extraction subnetworks, two subnetworks are used to extract the features of the MSI and PAN image, separately. Different sizes of convolutional kernels are used in the first layers due to the different spatial resolutions. Thus, the source images are mapped to the similar scale. Then a multi-scale asymmetric convolution factorization is used to extract features at different scales. In the fusion layer, the two feature extraction subnetworks are coupled. Features at the same scale are firstly summed and then the features of all scales are concatenated as one feature map. At last, a super-resolution subnetwork is used to generate the high resolution MSI. Experimental results on both synthetic and real data sets demonstrate that the proposed method outperforms the other state-of-the-art pan-sharpening methods.

show abstract

A Variational Pan-Sharpening Method Based on Spatial Fractional-Order Geometry and Spectral–Spatial Low-Rank Priors

Cited by 78 publications

References 42 publications

A New Variational Approach Based on Proximal Deep Injection and Gradient Intensity Similarity for Spatio-Spectral Image Fusion

A New Variational Approach Based on Proximal Deep Injection and Gradient Intensity Similarity for Spatio-Spectral Image Fusion

Hyperspectral Image Super-Resolution via Adaptive Dictionary Learning and Double ℓ 1 Constraint

A Two-Stream Multiscale Deep Learning Architecture for Pan-Sharpening

Contact Info

Product

Resources

About