DSA-Net: Infrared and Visible Image Fusion via Dual-Stream Asymmetric Network

Abstract

Infrared and visible image fusion technologies are used to characterize the same scene using diverse modalities. However, most existing deep learning-based fusion methods are designed as symmetric networks, which ignore the differences between modal images and lead to source image information loss during feature extraction. In this paper, we propose a new fusion framework for the different characteristics of infrared and visible images. Specifically, we design a dual-stream asymmetric network with two different feature extraction networks to extract infrared and visible feature maps, respectively. The transformer architecture is introduced in the infrared feature extraction branch, which can force the network to focus on the local features of infrared images while still obtaining their contextual information. The visible feature extraction branch uses residual dense blocks to fully extract the rich background and texture detail information of visible images. In this way, it can provide better infrared targets and visible details for the fused image. Experimental results on multiple datasets indicate that DSA-Net outperforms state-of-the-art methods in both qualitative and quantitative evaluations. In addition, we also apply the fusion results to the target detection task, which indirectly demonstrates the fusion performances of our method.

TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion

“…With the substantial value of infrared and visible light image fusion across various applications, there has been significant scholarly interest. The rise of deep learning has spurred numerous fusion methods, including those based on Convolutional Neural Networks [ 9 , 10 , 11 , 12 ], Autoencoders [ 13 , 14 , 15 , 16 ], and Generative Adversarial Networks [ 17 , 18 , 19 , 20 ]. While these methods have achieved commendable results, several challenges remain.…”