The PAN and MS Image Pansharpening Algorithm Based on Adaptive Neural Network and Sparse Representation in the NSST Domain

Wang, Xianghai; Bai, Shifu; Li, Zhi; Song, Ruoxi; Tao, Jingzhe

doi:10.1109/access.2019.2910656

Cited by 27 publications

(13 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In general, the spatial details are firstly extracted from the PAN image by MRA, and then injected into the up-sampled multispectral (UPMS) images. The widely used MRA methods include the Laplacian pyramid [8], wavelet transform [9][10][11], curvelet transform [12], non-subsampled contourlet transform [13,14], sheartlet transform [15], and non-subsampled sheartlet transform (NSST) [42]. Compared with CS-based methods, MRAbased methods present better spectra.…”

Section: Introductionmentioning

confidence: 99%

Two-Stage Pansharpening Based on Multi-Level Detail Injection Network

2020

IEEE Access

View full text Add to dashboard Cite

Pansharpening is an effective technology to obtain high resolution multispectral (HRMS) images by fusing low resolution multispectral (LRMS) images and high resolution panchromatic (PAN) images. With the rapid development of deep learning, some pansharpening methods based on deep learning have been proposed. Although fused images are greatly improved, there are still some areas for improvement. For example, the spectral preservation is not good enough and the details of fused images are not rich enough. To address the above problems, a two-stage pansharpening method based on convolutional neural network (CNN) is proposed. In the first stage, image super-resolution technology with residual block is used to enhance LRMS. In order to preserve spectra, inspired by the SAM (spectral angle mapper) index, a new spectral loss function is proposed. The second stage is the fusion stage. Detail injection block is proposed by combining detail injection and CNN in this stage. Experiments on WorldView2 and GeoEye1 images demonstrate that our fused images present more spatial details and better spectra by comparing with existing methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Two-Stage Pansharpening Based on Multi-Level Detail Injection Network

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The methods proposed by Vivone et al [16], Fei et al [17], and Yin [18] are the ones that combined the component substitution method and the MRA. The combination of the MRA, convolutional neural network (CNN), and sparse modeling was proposed by Wang et al [19], and the combination of the MRA and CNN was proposed by He et al [20].…”

Section: Introductionmentioning

confidence: 99%

Pansharpening by Complementing Compressed Sensing with Spectral Correction

2020

View full text Add to dashboard Cite

Pansharpening (PS) is a process used to generate high-resolution multispectral (MS) images from high-spatial-resolution panchromatic (PAN) and high-spectral-resolution multispectral images. In this paper, we propose a method for pansharpening by focusing on a compressed sensing (CS) technique. The spectral reproducibility of the CS technique is high due to its image reproducibility, but the reproduced image is blurry. Although methods of complementing this incomplete reproduction have been proposed, it is known that the existing method may cause ringing artifacts. On the other hand, component substitution is another technique used for pansharpening. It is expected that the spatial resolution of the images generated by this technique will be as high as that of the high-resolution PAN image, because the technique uses the corrected intensity calculated from the PAN image. Based on these facts, the proposed method fuses the intensity obtained by the component substitution method and the intensity obtained by the CS technique to move the spatial resolution of the reproduced image close to that of the PAN image while reducing the spectral distortion. Experimental results showed that the proposed method can reduce spectral distortion and maintain spatial resolution better than the existing methods.

show abstract

“…Laplacian pyramid (LP) [24] as a powerful tool for pan-sharpening used in [25], and generalized Laplacian pyramid with a modulation transfer function (GLP-MTF) introduced in [26]. Authors in [49] proposed fusion method using an adaptive neural network and sparse representation in the MRA domain. The MRAbased methods mostly keep spectral information of LRM bands better than CS-based methods.…”

Section: Introductionmentioning

confidence: 99%

A Variational Approach for Fusion of Panchromatic and Multispectral Images Using a New Spatial–Spectral Consistency Term

Khateri

Shabanzade

Mirzapour

et al. 2020

IEEE J. Sel. Top. Appl. Earth Observations Remote Sensing

View full text Add to dashboard Cite

In this paper, we propose a variational approach for fusion of two co-registered high-resolution panchromatic (HRP) and low-resolution multi-spectral (LRM) images to reach the high-resolution multi-spectral (HRM) one, i.e. pan-sharpening. In this fusion technique, there is a trade-off between structural information of HRP image and spectral information of LRM one. To reconstruct the HRM image, which benefits from the best characteristics of both images, we consider several fidelity terms. The structural fidelity term is used to transfer structural information of HRP image to HRM one, and spectral fidelity term is utilized to preserve spectral consistency between HRM and LRM images throughout the fusion process. To reduce the spectral distortion occurred due to the discrepancy between intensity values of HRP and LRM images, a novel spatial-spectral fidelity term is designed to keep the intensity ratio between multispectral and panchromatic pixels in the high-resolution space as the same as the low-resolution space. Moreover, the total variation (TV) regularization term is employed as a prior to promote the sparseness of gradient in HRM bands. These fidelity terms were formulated in a convex optimization problem. However, the structural and TV terms made this optimization problem nondifferentiable. Therefore, we developed an efficient majorizationminimization (MM) algorithm for solving the optimization problem. The proposed method applied to three datasets, acquired by WorldView-3, Deimos-2, and QuickBird satellites. To assess the effectiveness of the proposed method, visual analysis, as well as quantitative comparison to various pan-sharpening methods, were carried out. The experimental results suggested that the proposed method outperformed the competitors visually and quantitatively.

show abstract

The PAN and MS Image Pansharpening Algorithm Based on Adaptive Neural Network and Sparse Representation in the NSST Domain

Cited by 27 publications

References 36 publications

Two-Stage Pansharpening Based on Multi-Level Detail Injection Network

Two-Stage Pansharpening Based on Multi-Level Detail Injection Network

Pansharpening by Complementing Compressed Sensing with Spectral Correction

A Variational Approach for Fusion of Panchromatic and Multispectral Images Using a New Spatial–Spectral Consistency Term

Contact Info

Product

Resources

About