Remote sensing image fusion based on two-stream fusion network

et al. 2019

Remote Sensing

In recent years, convolutional neural networks (CNNs) have shown promising performance in the field of multispectral (MS) and panchromatic (PAN) image fusion (MS pansharpening). However, the small-scale data and the gradient vanishing problem have been preventing the existing CNN-based fusion approaches from leveraging deeper networks that potentially have better representation ability to characterize the complex nonlinear mapping relationship between the input (source) and the targeting (fused) images. In this paper, we introduce a very deep network with dense blocks and residual learning to tackle these problems. The proposed network takes advantage of dense connections in dense blocks that have connections for arbitrarily two convolution layers to facilitate gradient flow and implicit deep supervision during training. In addition, reusing feature maps can reduce the number of parameters, which is helpful for reducing overfitting that resulted from small-scale data. Residual learning is explored to reduce the difficulty for the model to generate the MS image with high spatial resolution. The proposed network is evaluated via experiments on three datasets, achieving competitive or superior performance, e.g. the spectral angle mapper (SAM) is decreased over 10% on GaoFen-2, when compared with other state-of-the-art methods.

Section: Cnn Based Pansharpening Methodsmentioning

confidence: 99%

Section: Comparison With Other Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Going Deeper with Densely Connected Convolutional Neural Networks for Multispectral Pansharpening

et al. 2019

Remote Sensing

“…Deep learning techniques have gained wide success in many vision tasks in recent years [27], [28], [29]. CNNs are among the most popular deep architectures, and often used as backbone or feature extractors to solve vision tasks for their good generalizability, such as AlexNet [30], VGGNet [31], and GoogleNet [32].…”

Section: B Deep Learning Based Change Detectionmentioning

confidence: 99%

From W-Net to CDGAN: Bitemporal Change Detection via Deep Learning Techniques

Hou

Liu

IEEE Trans. Geosci. Remote Sensing

et al. 2020

Self Cite

127

Traditional change detection methods usually follow the image differencing, change feature extraction and classification framework, and their performance is limited by such simple image domain differencing and also the hand-crafted features. Recently, the success of deep convolutional neural networks (CNNs) has widely spread across the whole field of computer vision for their powerful representation abilities. In this paper, we therefore address the remote sensing image change detection problem with deep learning techniques. We firstly propose an end-to-end dual-branch architecture, termed as the W-Net, with each branch taking as input one of the two bi-temporal images as in the traditional change detection models. In this way, CNN features with more powerful representative abilities can be obtained to boost the final detection performance. Also, W-Net performs differencing in the feature domain rather than in the traditional image domain, which greatly alleviates loss of useful information for determining the changes. Furthermore, by reformulating change detection as an image translation problem, we apply the recently popular Generative Adversarial Network (GAN) in which our W-Net serves as the Generator, leading to a new GAN architecture for change detection which we call CDGAN. To train our networks and also facilitate future research, we construct a large scale dataset by collecting images from Google Earth and provide carefully manually annotated ground truths. Experiments show that our proposed methods can provide fine-grained change detection results superior to the existing state-of-the-art baselines.

“…However, spectral distortion may occur when using [6]. In this paper, we develop a novel two-stream network as generator to generate pan-sharpened images [14]. The architecture of it is shown in the left part of Fig.…”

Section: Two-stream Generatormentioning

confidence: 99%

Psgan: A Generative Adversarial Network for Remote Sensing Image Pan-Sharpening

Liu

2018 25th IEEE International Conference on Image Processing (ICIP)

Liu

2018

Self Cite

Remote sensing image fusion (also known as pan-sharpening) aims to generate a high resolution multi-spectral image from inputs of a high spatial resolution single band panchromatic (PAN) image and a low spatial resolution multi-spectral (MS) image. In this paper, we propose PSGAN, a generative adversarial network (GAN) for remote sensing image pansharpening. To the best of our knowledge, this is the first attempt at producing high quality pan-sharpened images with GANs. The PSGAN consists of two parts. Firstly, a two-stream fusion architecture is designed to generate the desired high resolution multi-spectral images, then a fully convolutional network serving as a discriminator is applied to distinct "real" or "pan-sharpened" MS images. Experiments on images acquired by Quickbird and GaoFen-1 satellites demonstrate that the proposed PSGAN can fuse PAN and MS images effectively and significantly improve the results over the state of the art traditional and CNN based pan-sharpening methods.