Remote Sensing Image Fusion With Deep Convolutional Neural Network

Shao, Zhenfeng; Cai, Jiajun

doi:10.1109/jstars.2018.2805923

Cited by 286 publications

(132 citation statements)

References 51 publications

Supporting

Mentioning

122

Contrasting

Unclassified

Order By: Relevance

“…Recently, a number of DL-based pansharpening methods were proposed by exploiting different network structures [15,22,42,43,29,30,32]. These methods can be easily adapted to MS/HS fusion problem.…”

Section: Deep Learning Based Methodsmentioning

confidence: 99%

Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net

Xie

Zhou

Zhao

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

240

138

View full text Add to dashboard Cite

Hyperspectral imaging can help better understand the characteristics of different materials, compared with traditional image systems. However, only high-resolution multispectral (HrMS) and low-resolution hyperspectral (LrHS) images can generally be captured at video rate in practice. In this paper, we propose a model-based deep learning approach for merging an HrMS and LrHS images to generate a high-resolution hyperspectral (HrHS) image. In specific, we construct a novel MS/HS fusion model which takes the observation models of low-resolution images and the lowrankness knowledge along the spectral mode of HrHS image into consideration. Then we design an iterative algorithm to solve the model by exploiting the proximal gradient method. And then, by unfolding the designed algorithm, we construct a deep network, called MS/HS Fusion Net, with learning the proximal operators and model parameters by convolutional neural networks. Experimental results on simulated and real data substantiate the superiority of our method both visually and quantitatively as compared with state-of-the-art methods along this line of research.

show abstract

Section: Deep Learning Based Methodsmentioning

confidence: 99%

Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net

Xie

Zhou

Zhao

et al. 2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

240

138

View full text Add to dashboard Cite

show abstract

“…Because deep learning can automatically learn additional features from various types of data, it has gained considerable attention in recent years [17][18][19][20][21]. Different from the conventional pansharpening methods, deep learning-based methods present more ideal solutions for improving the performance of pansharpening.…”

Section: Introductionmentioning

confidence: 99%

“…To obtain the high quality fused image, Wei et al [19] presented a deep residual network (ResNet) for pansharpening. Shao et al [20] developed a two-branch network that can separately obtain salient features from MS and PAN images. A multiscale and multidepth convolutional neural network (CNN) for pansharpening was introduced by Yuan et al [12].…”

Section: Introductionmentioning

confidence: 99%

PCDRN: Progressive Cascade Deep Residual Network for Pansharpening

Yang

Huang

et al. 2020

Remote Sensing

View full text Add to dashboard Cite

Pansharpening is the process of fusing a low-resolution multispectral (LRMS) image with a high-resolution panchromatic (PAN) image. In the process of pansharpening, the LRMS image is often directly upsampled by a scale of 4, which may result in the loss of high-frequency details in the fused high-resolution multispectral (HRMS) image. To solve this problem, we put forward a novel progressive cascade deep residual network (PCDRN) with two residual subnetworks for pansharpening. The network adjusts the size of an MS image to the size of a PAN image twice and gradually fuses the LRMS image with the PAN image in a coarse-to-fine manner. To prevent an overly-smooth phenomenon and achieve high-quality fusion results, a multitask loss function is defined to train our network. Furthermore, to eliminate checkerboard artifacts in the fusion results, we employ a resize-convolution approach instead of transposed convolution for upsampling LRMS images. Experimental results on the Pléiades and WorldView-3 datasets prove that PCDRN exhibits superior performance compared to other popular pansharpening methods in terms of quantitative and visual assessments.

show abstract

“…One of the key benefits of DL is shaping the hierarchical representations by getting abnormal, that is, high‐level state, features from low‐level feature 19 . At long last, the fused image is acquired by utilizing both the choice guide and the source images 20 . The inverse transform is performed on fused sub bands to yield the fusion information at the last stage 21 …”

Section: Introductionmentioning

confidence: 99%

A novel approach in multimodality medical image fusion using optimal shearlet and deep learning

Velmurugan

Sivakumar

Sampson

2020

Int J Imaging Syst Tech

View full text Add to dashboard Cite

Multi‐modality medical image fusion (MMIF) procedures have been generally utilized in different clinical applications. MMIF can furnish an image with anatomical as well as physiological data for specialists that could advance the diagnostic procedures. Various models were proposed earlier related to MMIF though there is a need still exists to enhance the efficiency of the previous techniques. In this research, the authors proposed a novel fusion model based on optimal thresholding with deep learning concepts. An enhanced monarch butterfly optimization (EMBO) is utilized to decide the optimal threshold of fusion rules in shearlet transform. Then, low and high‐frequency sub‐bands were fused on the basis of feature maps and were given by the extraction part of the deep learning method. Here, restricted Boltzmann machine (RBM) was utilized to conduct the MMIF procedure. A benchmark dataset was utilized for training and testing purposes. The investigations were conducted utilizing a set of generally‐utilized pre‐enrolled CT and MR images that are publicly accessible. From the usage of fused low and high level frequency groups, the fused image can be attained. The simulation performance results were attained and the proposed model was proved to offer effective performance in terms of SD, edge quality (EQ), mutual information (MI), fusion factor (FF), entropy, correlation factor (CF), and spatial frequency (SF) with respective values being 97.78, 0.96, 5.71, 6.53, 7.43, 0.97, and 25.78 over the compared methods.

show abstract

Remote Sensing Image Fusion With Deep Convolutional Neural Network

Cited by 286 publications

References 51 publications

Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net

Multispectral and Hyperspectral Image Fusion by MS/HS Fusion Net

PCDRN: Progressive Cascade Deep Residual Network for Pansharpening

A novel approach in multimodality medical image fusion using optimal shearlet and deep learning

Contact Info

Product

Resources

About