Infrared and Visible Image Fusion Using Bimodal Transformers

Park, Seonghyun; Vien, An Gia; Lee, Chul

doi:10.1109/icip46576.2022.9897993

Cited by 7 publications

(2 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As late fusion algorithms fuse the features extracted using independently trained networks, they can preserve the intrinsic features of each image. Most researches have focused on the design of elaborate network architectures for end-to-end fusion capable of both better feature extraction and feature fusion [21], [23], [24], [32], [39]- [41], [43]. In addition, in [31], an algorithm was developed to generate weight maps for effective fusion using pretrained networks.…”

Section: B Learning-based Fusionmentioning

confidence: 99%

See 1 more Smart Citation

Multiscale Progressive Fusion of Infrared and Visible Images

Park

Lee

2022

IEEE Access

Self Cite

View full text Add to dashboard Cite

Infrared and visible image fusion aims to generate more informative images of a given scene by combining multimodal images with complementary information. Although recent learning-based approaches have shown significant fusion performance, developing an effective fusion algorithm that can preserve complementary information while preventing bias toward either of the source images remains a significant challenge. In this work, we propose a multiscale progressive fusion (MPFusion) algorithm that extracts and progressively fuses multiscale features of infrared and visible images. The proposed algorithm consists of two networks, IRNet and FusionNet, which extract the intrinsic features of infrared and visible images, respectively. We transfer the multiscale information of the infrared image from IRNet to FusionNet to generate an informative fusion result. To this end, we develop the multi-dilated residual block (MDRB) and the progressive fusion block (PFB), which progressively combines the multiscale features from IRNet with those from FusionNet to fuse complementary features effectively and adaptively. Furthermore, we exploit edge-guided attention maps to preserve complementary edge information in the source images during fusion. Experimental results on several datasets demonstrate that the proposed algorithm outperforms state-of-the-art infrared and visible image fusion algorithms on both quantitative and qualitative comparisons.

show abstract

Section: B Learning-based Fusionmentioning

confidence: 99%

“…More recently, several transformer-based fusion algorithms [39]- [43] that can capture the inter-domain longrange dependencies with self-attention mechanism have been developed. For example, in [39], [40], the local features of CNNs and global features of transformers were integrated to take advantages of both CNN and transformer.…”

Section: Introductionmentioning

confidence: 99%