Coupled GAN With Relativistic Discriminators for Infrared and Visible Images Fusion

Li, Qilei; Lu, Lu; Li, Zhen; Wu, Wei; Liu, Zheng; Jeon, Gwanggil; Yang, Xiaobo

doi:10.1109/jsen.2019.2921803

Cited by 75 publications

(36 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, GAN is also favored by researchers for image fusion. For example, Ma proposed Pan-GAN in remote sensing image fusion, which can obtain multi-spectral images of high-resolution by fusing panchromatic images and multispectral images of low-resolution [ 25 ], Li proposed coupled GAN with relativistic discriminators for infrared and visible image fusion, where the simple yet efficient relativistic discriminator is applied [ 26 ], and so on.…”

Section: Ganmentioning

confidence: 99%

A Noisy SAR Image Fusion Method Based on NLM and GAN

Fang

Wang

et al. 2021

Entropy

View full text Add to dashboard Cite

The unavoidable noise often present in synthetic aperture radar (SAR) images, such as speckle noise, negatively impacts the subsequent processing of SAR images. Further, it is not easy to find an appropriate application for SAR images, given that the human visual system is sensitive to color and SAR images are gray. As a result, a noisy SAR image fusion method based on nonlocal matching and generative adversarial networks is presented in this paper. A nonlocal matching method is applied to processing source images into similar block groups in the pre-processing step. Then, adversarial networks are employed to generate a final noise-free fused SAR image block, where the generator aims to generate a noise-free SAR image block with color information, and the discriminator tries to increase the spatial resolution of the generated image block. This step ensures that the fused image block contains high resolution and color information at the same time. Finally, a fused image can be obtained by aggregating all the image blocks. By extensive comparative experiments on the SEN1–2 datasets and source images, it can be found that the proposed method not only has better fusion results but is also robust to image noise, indicating the superiority of the proposed noisy SAR image fusion method over the state-of-the-art methods.

show abstract

Section: Ganmentioning

confidence: 99%

A Noisy SAR Image Fusion Method Based on NLM and GAN

Fang

Wang

et al. 2021

Entropy

View full text Add to dashboard Cite

show abstract

“…To solve these problems and further improve the quality of the fused image, Li et al [17] proposed RCGAN. In RCGAN, two groups of GAN are used to make the fused image close to the infrared image and the visible image respectively.…”

Section: A Fusion With Gansmentioning

confidence: 99%

“…Some state-of-the-art fusion methods have been proposed based on the powerful representation ability of deep networks and achieved satisfactory results [7]- [9], [12]- [17]. Various network architectures have been exploited in fusion methods, including convolutional neural network(CNN)-based [7]- [9], [12], [13], encoderdecoder-based [14], [15], and GAN-based [16], [17] models. Liu et al [7] fused infrared and visible images by introducing Laplacian pyramid decomposition into shallow convolutional neural network.…”

Section: Introductionmentioning

confidence: 99%

“…Ma et al [16] proposed a fusionGAN, the first fusion network involving GAN, and applied it to the field of infrared and visible image fusion. Li et al [17] proposed a couple GAN termed RCGAN to improve the performance of fusionGAN. A symmetrical network structure made it easier to acquire features of both visible and infrared images, ensuring that the fused image contained more information from the source images.…”

Section: Introductionmentioning

confidence: 99%

“…The first row from left to right are infrared and visible images. The second row from left to right are images fused by fusionGAN [16], RCGAN [17], ResNet50-based method [12], VGG19-based method [8], and our DECA network. In the first fused image, the target (the highlighted area in the infrared image) is obvious but the details from the visible image are missed.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Infrared and Visible Image Fusion Using Detail Enhanced Channel Attention Network

Cui

Mei

2019

IEEE Access

View full text Add to dashboard Cite

Fusion of infrared and visible images aims to maintain thermal radiation information and detailed texture information on a single image. Previous deep learning based methods require complex architecture of networks to extract features of both sources. In these methods, convolution filters act equally on each channel so the feature maps containing different brightness and gradient information are treated equally across channels, which reduces the representational ability of networks. In this paper, we innovatively introduce channel-attention mechanism to the fusion network. We propose a detail enhanced channel attention(DECA) block and apply it to the fusion network. DECA block takes into account the average of brightness and gradient information to rescale the feature maps. It allows network selectively emphasize useful features and suppress less useful ones. The proposed fusion network has relatively simple architecture and low operational complexity. This end-to-end network can generate fused images directly from source images. The experimental results on fusion of infrared and visible images demonstrate that our proposed method outperforms the state-of-the-art fusion methods.INDEX TERMS Channel-wise attention, DECA, image fusion.

show abstract

Multiscale channel attention network for infrared and visible image fusion

Zhu

Dou

Jian

et al. 2020

Concurrency and Computation

Self Cite

View full text Add to dashboard Cite

Imaging systems with different imaging sensors are widely applied to surveillance field, military field, and medicine field. Particularly, infrared imaging sensors can acquire thermal radiations emitted by different objects but lack textural details, and visible imaging sensors can capture abundant textural information but suffer from loss of scene information under poor weather conditions. The fusion of infrared and visible images can synthesize a new image with complementary information of the source images. In this paper, we present a deep learning method with encoder–decoder architecture for infrared and visible image fusion. Firstly, multiscale channel attention blocks are introduced to extract features at different scales, which can preserve more meaningful information and enhance the important information. Secondly, we utilize the improved fusion strategy based on visual saliency to fuse feature maps. Lastly, the fusion result is restored via reconstruction network. In comparison with other state‐of‐the‐art approaches, our experimental results achieve appealing performance on visual effects and objective assessments.

show abstract

Coupled GAN With Relativistic Discriminators for Infrared and Visible Images Fusion

Cited by 75 publications

References 23 publications

A Noisy SAR Image Fusion Method Based on NLM and GAN

A Noisy SAR Image Fusion Method Based on NLM and GAN

Infrared and Visible Image Fusion Using Detail Enhanced Channel Attention Network

Multiscale channel attention network for infrared and visible image fusion

Contact Info

Product

Resources

About