Two-Stage Pansharpening Based on Multi-Level Detail Injection Network

Hu, Jianwen; Du, Congcong

doi:10.1109/access.2020.3019201

Cited by 11 publications

(3 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Recently, with the rapid development of deep learning and accessibility of highperformance computing hardware equipment, convolutional neural networks (CNNs) have shown outstanding performance in image processing fields, e.g., image resolution reconstruction [45][46][47][48][49], image segmentation [50][51][52], image fusion [53][54][55][56][57], image classification [58], image denoising [59], etc. Therefore, many methods [34][35][36][37][38]41,42,[58][59][60][61][62][63][64][65][66][67][68][69][70][71][72][73][74][75] based on deep learning have also been applied to solve the pansharpening problem. Benefiting from the powerful nonlinear fitting and feature extraction capabilities of CNNs and the availability of big data, these DL-based methods could perform better than the above three methods to a certain degree, i.e., CS-, MRA-, and VO-based methods.…”

Section: Introductionmentioning

confidence: 99%

Remote Sensing Pansharpening by Full-Depth Feature Fusion

et al. 2022

View full text Add to dashboard Cite

Pansharpening is an important yet challenging remote sensing image processing task, which aims to reconstruct a high-resolution (HR) multispectral (MS) image by fusing a HR panchromatic (PAN) image and a low-resolution (LR) MS image. Though deep learning (DL)-based pansharpening methods have achieved encouraging performance, they are infeasible to fully utilize the deep semantic features and shallow contextual features in the process of feature fusion for a HR-PAN image and LR-MS image. In this paper, we propose an efficient full-depth feature fusion network (FDFNet) for remote sensing pansharpening. Specifically, we design three distinctive branches called PAN-branch, MS-branch, and fusion-branch, respectively. The features extracted from the PAN and MS branches will be progressively injected into the fusion branch at every different depth to make the information fusion more broad and comprehensive. With this structure, the low-level contextual features and high-level semantic features can be characterized and integrated adequately. Extensive experiments on reduced- and full-resolution datasets acquired from WorldView-3, QuickBird, and GaoFen-2 sensors demonstrate that the proposed FDFNet only with less than 100,000 parameters performs better than other detail injection-based proposals and several state-of-the-art approaches, both visually and quantitatively.

show abstract

Section: Introductionmentioning

confidence: 99%

Remote Sensing Pansharpening by Full-Depth Feature Fusion

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Hu et. al [27], incorporates super resolution method in a deep learning framework to generate HR-MS image from LR-MS image. PAN sharpened image is then obtained from HR-MS image and details (extracted using super resolution and total loss methods).…”

Section: Introductionmentioning

confidence: 99%

Siamese Networks Based Deep Fusion Framework for Multi-Source Satellite Imagery

et al. 2022

View full text Add to dashboard Cite

A critical aim of pansharpening is to fuse coherent spatial and spectral features from panchromatic and multispectral images respectively. This study proposes deep siamese network based pansharpening model as a two-stage framework in a multiscale setting. In the first stage, a siamese network learns a common feature space between panchromatic and multispectral bands. The second stage follows by fusing the output feature maps of the siamese network. The parameters of these two stages are shared across scales in order to add spatial information consistently (across scales). The spectral information is preserved by adding appropriate skip connections from input multispectral image. Multi-level network parameters sharing mechanism in pyramidal reconstruction of pansharpened image, better preserves spatial and spectral details simultaneously. Experimental work carried out using deep siamese network in multi-scale setting (to obtain inter-band similarity among different sensor data) outperforms several latest pansharpening methods.INDEX TERMS Pansharpening, image fusion, deep-learning, siamese networks, remote sensing, depth-of-field.

show abstract

“…PS with spatial and spectral gradient difference-induced nonconvex sparsity priors (PSSGDNSP) [43] uses the eigenband correlation of MS images to process MS images as third-order tensors. In addition, there are some fusion methods based on DL [44][45][46][47][48][49][50][51][52][53][54]. The DL method's main disadvantages are the lack of ideal PS samples for training, it relies on generating reference samples from unlabeled real data (such as MS images).…”

Section: Introductionmentioning

confidence: 99%

A Two-Stage Pansharpening Method for the Fusion of Remote-Sensing Images

et al. 2022

View full text Add to dashboard Cite

The pansharpening (PS) of remote-sensing images aims to fuse a high-resolution panchromatic image with several low-resolution multispectral images for obtaining a high-resolution multispectral image. In this work, a two-stage PS model is proposed by integrating the ideas of component replacement and the variational method. The global sparse gradient of the panchromatic image is extracted by variational method, and the weight function is constructed by combining the gradient of multispectral image in which the global sparse gradient can provide more robust gradient information. Furthermore, we refine the results in order to reduce spatial and spectral distortions. Experimental results show that our method had high generalization ability for QuickBird, Gaofen-1, and WorldView-4 satellite data. Experimental results evaluated by seven metrics demonstrate that the proposed two-stage method enhanced spatial details subjective visual effects better than other state-of-the-art methods do. At the same time, in the process of quantitative evaluation, the method in this paper had high improvement compared with that other methods, and some of them can reach a maximal improvement of 60%.

show abstract

Two-Stage Pansharpening Based on Multi-Level Detail Injection Network

Cited by 11 publications

References 55 publications

Remote Sensing Pansharpening by Full-Depth Feature Fusion

Remote Sensing Pansharpening by Full-Depth Feature Fusion

Siamese Networks Based Deep Fusion Framework for Multi-Source Satellite Imagery

A Two-Stage Pansharpening Method for the Fusion of Remote-Sensing Images

Contact Info

Product

Resources

About