Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity

Tang, Linfeng; Zhang, Hao; Xu, Han; Ma, Jiayi

doi:10.1016/j.inffus.2023.101870

Cited by 46 publications

(9 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare our method with related methods in the last three years, including DenseFuse 17 , DIDFuse 35 , NestFuse 36 , Dual Branch 37 , UNFusion 38 , RFN-Nest 12 , GANMcC 11 , Res2Fusion 39 , CUFD 40 , LRRNet 41 , Cddfuse 42 , PSFusion 43 , and AUIF 15 . The fusion images are shown in Fig.…”

Section: Resultsmentioning

confidence: 99%

Interactive residual coordinate attention and contrastive learning for infrared and visible image fusion in triple frequency bands

Xie,

Zong,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

The auto-encoder (AE) based image fusion models have achieved encouraging performance on infrared and visible image fusion. However, the meaningful information loss in the encoding stage and simple unlearnable fusion strategy are two significant challenges for such models. To address these issues, this paper proposes an infrared and visible image fusion model based on interactive residual attention fusion strategy and contrastive learning in the frequency domain. Firstly, the source image is transformed into three sub-bands of the high-frequency, low-frequency, and mid-frequency for powerful multiscale representation from the prospective of the frequency spectrum analysis. To further cope with the limitations of the straightforward fusion strategy, a learnable coordinate attention module in the fusion layer is incorporated to adaptively fuse representative information based on the characteristics of the corresponding feature maps. Moreover, the contrastive learning is leveraged to train the multiscale decomposition network for enhancing the complementarity of information at different frequency spectra. Finally, the detail-preserving loss, feature enhancing loss and contrastive loss are incorporated to jointly train the entire fusion model for good detail maintainability. Qualitative and quantitative comparisons demonstrate the feasibility and validity of our model, which can consistently generate fusion images containing both highlight targets and legible details, outperforming the state-of-the-art fusion methods.

show abstract

Section: Resultsmentioning

confidence: 99%

Interactive residual coordinate attention and contrastive learning for infrared and visible image fusion in triple frequency bands

Xie,

Zong,

et al. 2024

Sci Rep

View full text Add to dashboard Cite

show abstract

“…RFNNest [43] proposes a residual fusion network and can better retain detailed features. PSFusion [44] presents a practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity constraints and the fusion images have good visual appeal. CDDFuse [45] is inspired by multi-scale decomposition and uses neural networks to decompose images into basic parts and detailed parts.…”

Section: Autoencoder-based Methodsmentioning

confidence: 99%

“…Optimization-based generation methods include DDcGAN [30]. Methods based on deep neural network feature representation learning include DenseFuse [40], RFNNest [43], FlFuse [51], PIAFusion [37], and PSFusion [44]. Model-based methods include GDFusion [28], and DDFM [52].…”

Section: Methods For Comparisonmentioning

confidence: 99%

SharDif: Sharing and Differential Learning for Image Fusion

Liang,

Gao

2024

Entropy

View full text Add to dashboard Cite

Image fusion is the generation of an informative image that contains complementary information from the original sensor images, such as texture details and attentional targets. Existing methods have designed a variety of feature extraction algorithms and fusion strategies to achieve image fusion. However, these methods ignore the extraction of common features in the original multi-source images. The point of view proposed in this paper is that image fusion is to retain, as much as possible, the useful shared features and complementary differential features of the original multi-source images. Shared and differential learning methods for infrared and visible light image fusion are proposed. An encoder with shared weights is used to extract shared common features contained in infrared and visible light images, and the other two encoder blocks are used to extract differential features of infrared images and visible light images, respectively. Effective learning of shared and differential features is achieved through weight sharing and loss functions. Then, the fusion of shared features and differential features is achieved via a weighted fusion strategy based on an entropy-weighted attention mechanism. The experimental results demonstrate the effectiveness of the proposed model with its algorithm. Compared with the-state-of-the-art methods, the significant advantage of the proposed method is that it retains the structural information of the original image and has better fusion accuracy and visual perception effect.

show abstract

“…17 To eliminate the arbitrariness of handcrafted feature extractors and fusion strategies, several end-to-end fusion frameworks have been proposed. Ma et al 18 introduced a fusion framework using a generative adversarial network (FusionGAN) into the domain of visible and infrared image fusion. However, there are still some issues in current unsupervised deep learning-based visible and infrared image fusion methods, hindering the development of this field.…”

Section: Introductionmentioning

confidence: 99%

“…To eliminate the arbitrariness of handcrafted feature extractors and fusion strategies, several end-to-end fusion frameworks have been proposed. Ma et al 18 . introduced a fusion framework using a generative adversarial network (FusionGAN) into the domain of visible and infrared image fusion.…”

Section: Introductionmentioning

confidence: 99%

Infrared and visible image fusion method based on hierarchical attention mechanism

Li,

Yan,

Luo

2024

J. Electron. Imag.

View full text Add to dashboard Cite

Currently, most feature extraction methods lack generality and require specific feature extraction methods for images from different sensors, particularly in the detection of faults in power equipment. However, the challenge lies in how to restore realistic texture details while correcting the color distortion. To address this issue, we propose an innovative infrared-visible light image fusion technique that combines hierarchical attention modules and collaborative refinement modules to facilitate feature fusion by jointly preserving intricate details and correcting lighting conditions. The hierarchical attention module aims to provide two different attention weight maps, which help select the most salient information from the source images in the infrared and visible light domains. This ultimately produces intermediate fusion results with comprehensive and complementary data. The collaborative refinement module consists of an edge enhancement network and a lighting correction network, which works together to enhance edge details and correct color differences. Experimental results validate the effectiveness of this method in successfully fusing the two types of images. The results demonstrate that, compared with several mainstream fusion methods, this method exhibits significant advantages in publicly available datasets and scenarios, such as power equipment detection.

show abstract

Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity

Cited by 46 publications

References 68 publications

Interactive residual coordinate attention and contrastive learning for infrared and visible image fusion in triple frequency bands

Interactive residual coordinate attention and contrastive learning for infrared and visible image fusion in triple frequency bands

SharDif: Sharing and Differential Learning for Image Fusion

Infrared and visible image fusion method based on hierarchical attention mechanism

Contact Info

Product

Resources

About