Multigrained Attention Network for Infrared and Visible Image Fusion

Li, Jing; Huo, Hongtao; Li, Chang; Wang, Ren-Hua; Sui, Chenhong; Zhao, Lei

doi:10.1109/tim.2020.3029360

Cited by 37 publications

(25 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, some researchers had exploited the generative adversarial network (GAN) [32][33][34][35][36] for image fusion, and achieved satisfactory results to some extent. Typically, Ma et al firstly presented FusionGAN [13] where the adversarial learning network including a generator and a discriminator was proposed.…”

Section: B Deep Learning-based Fusion Methodsmentioning

confidence: 99%

“…Ulteriorly, Yang et al [34] constructed a texture conditional generative adversarial network to capture texture map, and further proposed the squeeze-and-excitation module to highlight texture information. Li et al presented a multi-grained attentional network, namely MgAN-Fuse [35], which integrated attention modules into the encoder-decoder network to capture the context information in the generator. Meanwhile, they also introduced AttentionFGAN [36] where a multi-scale attention module was integrated into both generator and discriminator.…”

Section: B Deep Learning-based Fusion Methodsmentioning

confidence: 99%

See 1 more Smart Citation

SwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images

Wang¹,

Chen²,

Shao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Section: B Deep Learning-based Fusion Methodsmentioning

confidence: 99%

Section: B Deep Learning-based Fusion Methodsmentioning

confidence: 99%

SwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images

Wang¹,

Chen²,

Shao³

et al. 2022

Preprint

View full text Add to dashboard Cite

“…In order to settle these issues, Li et al [35] employed a multi-grained attention network with two independent encoders, namely MgAN-Fuse, which integrated a channel attention model into multi-scale layers of the encoder, and then multi-grained attention maps were reconstructed a fused image by the decoder. Subsequently, they extended the attention mechanism into generator and discriminator, termed as At-tentionFGAN [36], which designed two multi-scale attention networks to generate the respective attention maps of infrared and visible images, and were directly concatenated with source images for the fusion network to produce a fused result.…”

Section: B Gan-based Fusion Methodsmentioning

confidence: 99%

Infrared and Visible Image Fusion via Interactive Compensatory Attention Adversarial Learning

Wang¹,

Shao²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

The existing generative adversarial fusion methods generally concatenate source images and extract local features through convolution operation, without considering their global characteristics, which tends to produce an unbalanced result and is biased towards the infrared image or visible image. Toward this end, we propose a novel end-to-end mode based on generative adversarial training to achieve better fusion balance, termed as interactive compensatory attention fusion network (ICAFusion). In particular, in the generator, we construct a multi-level encoderdecoder network with a triple path, and adopt infrared and visible paths to provide additional intensity and gradient information. Moreover, we develop interactive and compensatory attention modules to communicate their pathwise information, and model their long-range dependencies to generate attention maps, which can more focus on infrared target perception and visible detail characterization, and further increase the representation power for feature extraction and feature reconstruction. In addition, dual discriminators are designed to identify the similar distribution between fused result and source images, and the generator is optimized to produce a more balanced result. Extensive experiments illustrate that our ICAFusion obtains superior fusion performance and better generalization ability, which precedes other advanced methods in the subjective visual description and objective metric evaluation. Our codes will be public at https://github.com/Zhishe-Wang/ICAFusion.

show abstract

“…The second type as Fig. 14 (b) has one generator and two discriminators such as [116] [117] [118]. Their inputs are the concatenated infrared and visible Images, while one discriminator compares fusion image with visible image and the other compares fusion image with infrared image.…”

Section: Infrared-visible Image Fusionmentioning

confidence: 99%

“…(a) is the architecture with one generator and one discriminator as[114][111][115]. (b) is the architecture with one generator and two discriminators as[116][117][118]. (c) is the coupled GAN architecture as[119][120].…”

mentioning

confidence: 99%

A Review on Remote Sensing Data Fusion with Generative Adversarial Networks (GAN)

Liu¹

2021

Preprint

View full text Add to dashboard Cite

In the past decades, remote sensing (RS) data fusion has always been an active research community. A large number of algorithms and models have been developed. Generative Adversarial Networks (GAN), as an important branch of deep learning, show promising performances in variety of RS image fusions. This review provides an introduction to GAN for remote sensing data fusion. We briefly review the frequently-used architecture and characteristics of GAN in data fusion and comprehensively discuss how to use GAN to realize fusion for homogeneous RS data, heterogeneous RS data, and RS and ground observation data. We also analyzed some typical applications with GAN-based RS image fusion. This review takes insight into how to make GAN adapt to different types of fusion tasks and summarizes the advantages and disadvantages of GAN-based RS data fusion. Finally, we discuss the promising future research directions and make a prediction on its trends.

show abstract

Multigrained Attention Network for Infrared and Visible Image Fusion

Cited by 37 publications

References 39 publications

SwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images

SwinFuse: A Residual Swin Transformer Fusion Network for Infrared and Visible Images

Infrared and Visible Image Fusion via Interactive Compensatory Attention Adversarial Learning

A Review on Remote Sensing Data Fusion with Generative Adversarial Networks (GAN)

Contact Info

Product

Resources

About