DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion

Ma, Jiayi; Xu, Han; Jiang, Junjun; Mei, Xiaoguang; Zhang, Xiaoping

doi:10.1109/tip.2020.2977573

Cited by 717 publications

(276 citation statements)

References 51 publications

Supporting

Mentioning

274

Contrasting

Order By: Relevance

“…Cheng proposed a new adaptive dual-channel PCNN for fusing the infrared and visual images [ 34 ], which exhibits good fusion performance. Ma et al also tried to address the fusion issues for infrared and visible images by using neural networks, such as the DDcGAN method [ 35 ], the FusionGAN method [ 36 ], and the detail-preserving adversarial learning method [ 37 ]. These models have attained very satisfactory results.…”

Section: Introductionmentioning

confidence: 99%

An Improved Pulse-Coupled Neural Network Model for Pansharpening

Yan

Xie

et al. 2020

Sensors

View full text Add to dashboard Cite

Pulse-coupled neural network (PCNN) and its modified models are suitable for dealing with multi-focus and medical image fusion tasks. Unfortunately, PCNNs are difficult to directly apply to multispectral image fusion, especially when the spectral fidelity is considered. A key problem is that most fusion methods using PCNNs usually focus on the selection mechanism either in the space domain or in the transform domain, rather than a details injection mechanism, which is of utmost importance in multispectral image fusion. Thus, a novel pansharpening PCNN model for multispectral image fusion is proposed. The new model is designed to acquire the spectral fidelity in terms of human visual perception for the fusion tasks. The experimental results, examined by different kinds of datasets, show the suitability of the proposed model for pansharpening.

show abstract

Section: Introductionmentioning

confidence: 99%

An Improved Pulse-Coupled Neural Network Model for Pansharpening

Yan

Xie

et al. 2020

Sensors

View full text Add to dashboard Cite

show abstract

“…They utilize a novel CNN architecture and designed a no-reference quality metric as the loss function. In FusionGAN and its variants [16,22,23], a generative adversarial network is applied to fuse infrared and visible images. The fused image generated by the generator is forced to have more details existing in the visible image by applying the discriminator to distinguish differences between them.…”

Section: Fusion Methods Based On Deep Learningmentioning

confidence: 99%

GANFuse: a novel multi-exposure image fusion method based on generative adversarial networks

Yang

Chen

et al. 2020

Neural Comput & Applic

View full text Add to dashboard Cite

In this paper, a novel multi-exposure image fusion method based on generative adversarial networks (termed as GANFuse) is presented. Conventional multi-exposure image fusion methods improve their fusion performance by designing sophisticated activity-level measurement and fusion rules. However, these methods have a limited success in complex fusion tasks. Inspired by the recent FusionGAN which firstly utilizes generative adversarial networks (GAN) to fuse infrared and visible images and achieves promising performance, we improve its architecture and customize it in the task of extreme exposure image fusion. To be specific, in order to keep content of extreme exposure image pairs in the fused image, we increase the number of discriminators differentiating between fused image and extreme exposure image pairs. While, a generator network is trained to generate fused images. Through the adversarial relationship between generator and discriminators, the fused image will contain more information from extreme exposure image pairs. Thus, this relationship can realize better performance of fusion. In addition, the method we proposed is an end-to-end and unsupervised learning model, which can avoid designing hand-crafted features and does not require a number of ground truth images for training. We conduct qualitative and quantitative experiments on a public dataset, and the experimental result shows that the proposed model demonstrates better fusion ability than existing multi-exposure image fusion methods in both visual effect and evaluation metrics.

show abstract

“…Xia et al [29] proposed a novel fusion scheme for multi-modal medical images, which utilizes both the features of the multi-scale transformation and deep convolutional neural network. As for the fusion methods based on GAN, Xu et alandMaet al [12], [30] proposed DDcGAN, which employs a generator with two discriminators to acquire the functional information in I F and texture details in I S . Besides, Yang et al [31] are committed to applying wasserstein GAN to the fusion of medical images.…”

Section: B Deep Learning-based Medical Image Fusionmentioning

confidence: 99%

MGMDcGAN: Medical Image Fusion Using Multi-Generator Multi-Discriminator Conditional Generative Adversarial Network

Huang

et al. 2020

IEEE Access

View full text Add to dashboard Cite

In this paper, we propose a novel end-to-end model for fusing medical images characterizing structural information, i.e., I S , and images characterizing functional information, i.e., I F , of different resolutions, by using a multi-generator multi-discriminator conditional generative adversarial network (MGMDcGAN). In the first cGAN, the generator aims to generate a real-like fused image based on a specifically designed content loss to fool two discriminators, while the discriminators aim to distinguish the structure differences between the fused image and source images. On this basis, we employ the second cGAN with a mask to enhance the information of dense structure in the final fused image, while preventing the functional information from being weakened. Consequently, the final fused image is forced to concurrently keep the structural information in I S and the functional information in I F. In addition, as a unified method, MGMDcGAN can be applied to different kinds of medical image fusion, i.e., MRI-PET, MRI-SPECT, and CT-SPECT, where MRI and CT are two kinds of I S of high resolution, PET and SPECT are typical kinds of I F of low resolution. Qualitative and quantitative experiments on publicly available datasets demonstrate the superiority of our MGMDcGAN over the state-of-the-art. INDEX TERMS Medical image fusion, generative adversarial network, different resolutions, end-to-end, unified method.

show abstract

DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion

Cited by 717 publications

References 51 publications

An Improved Pulse-Coupled Neural Network Model for Pansharpening

An Improved Pulse-Coupled Neural Network Model for Pansharpening

GANFuse: a novel multi-exposure image fusion method based on generative adversarial networks

MGMDcGAN: Medical Image Fusion Using Multi-Generator Multi-Discriminator Conditional Generative Adversarial Network

Contact Info

Product

Resources

About