Going Deeper with Densely Connected Convolutional Neural Networks for Multispectral Pansharpening

Abstract

In recent years, convolutional neural networks (CNNs) have shown promising performance in the field of multispectral (MS) and panchromatic (PAN) image fusion (MS pansharpening). However, the small-scale data and the gradient vanishing problem have been preventing the existing CNN-based fusion approaches from leveraging deeper networks that potentially have better representation ability to characterize the complex nonlinear mapping relationship between the input (source) and the targeting (fused) images. In this paper, we introduce a very deep network with dense blocks and residual learning to tackle these problems. The proposed network takes advantage of dense connections in dense blocks that have connections for arbitrarily two convolution layers to facilitate gradient flow and implicit deep supervision during training. In addition, reusing feature maps can reduce the number of parameters, which is helpful for reducing overfitting that resulted from small-scale data. Residual learning is explored to reduce the difficulty for the model to generate the MS image with high spatial resolution. The proposed network is evaluated via experiments on three datasets, achieving competitive or superior performance, e.g. the spectral angle mapper (SAM) is decreased over 10% on GaoFen-2, when compared with other state-of-the-art methods.

MESSFN : a Multi-level and Enhanced Spectral-Spatial Fusion Network for Pan-sharpening

“…The spectral discriminators and the spatial discriminators are established to approximate the generated HRMS image consistent with the source images distribution. Besides, there are some other fusion models, such as the dense connection in DenseNet [32,33] the progressive up-sampling method [28], the feedback mechanism [34], etc. In conclusion, CNNs bring a new exploring direction and a superior fusion performance for pan-sharpening.…”

Fusion of Panchromatic and Multispectral Images Using Multiscale Convolution Sparse Decomposition

“…Wei et al [43]- [44] proposed a deep residual pan-sharpening neural network (DRPNN) to boost the accuracy of the fusion results. Besides, Wang et al [45] introduced dense blocks into CNN and residual learning is considered for spatial resolution enhancement.…”

Heterogeneous two-Stream Network with Hierarchical Feature Prefusion for Multispectral Pan-Sharpening

“…However, with hardware limitations, many satellites can only collect single band HR PAN images and MS images with low spatial resolution. MS pansharpening refers to the technology that obtains the ideal HR-MS images by combining the PAN images and MS images [4].…”