PIAFusion: A progressive infrared and visible image fusion network based on illumination aware

Tang, Linfeng; Yuan, Jiteng; Zhang, Hao; Jiang, X. Y.; Ma, Jiayi

doi:10.1016/j.inffus.2022.03.007

Cited by 270 publications

(98 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subjective and objective evaluations of our method were carried out on two different datasets. The datasets are derived from TNO [ 35 ] and MSRS [ 36 ], and the selected images are aligned. Among them, the TNO dataset contains infrared and visible images of different military scenes, and MSRS dataset contains multiple infrared and visible images of multi-spectral road scenarios.…”

Section: Experimental Results and Comparisonsmentioning

confidence: 99%

Infrared and Visible Image Fusion with Significant Target Enhancement

Huo

Deng

Shao

2022

Entropy

View full text Add to dashboard Cite

Existing fusion rules focus on retaining detailed information in the source image, but as the thermal radiation information in infrared images is mainly characterized by pixel intensity, these fusion rules are likely to result in reduced saliency of the target in the fused image. To address this problem, we propose an infrared and visible image fusion model based on significant target enhancement, aiming to inject thermal targets from infrared images into visible images to enhance target saliency while retaining important details in visible images. First, the source image is decomposed with multi-level Gaussian curvature filtering to obtain background information with high spatial resolution. Second, the large-scale layers are fused using ResNet50 and maximizing weights based on the average operator to improve detail retention. Finally, the base layers are fused by incorporating a new salient target detection method. The subjective and objective experimental results on TNO and MSRS datasets demonstrate that our method achieves better results compared to other traditional and deep learning-based methods.

show abstract

Section: Experimental Results and Comparisonsmentioning

confidence: 99%

Infrared and Visible Image Fusion with Significant Target Enhancement

Huo

Deng

Shao

2022

Entropy

View full text Add to dashboard Cite

show abstract

“…The MSRS dataset [ 14 ] was chosen to train the network model. This dataset was a mixture of daytime and nighttime road-related scenes, containing various elements such as pedestrians, vehicles, buildings, etc.…”

Section: Methodsmentioning

confidence: 99%

“…Xu et al [ 13 ] adopted a dense connection structure to extract image features and added multiple jump connections between the dense blocks to increase the information flow. Tang et al [ 14 ] proposed an image fusion network based on illumination perception, which could adaptively maintain the intensity distribution of salient targets according to light distribution.…”

Section: Related Workmentioning

confidence: 99%

Infrared and Visible Image Fusion for Highlighting Salient Targets in the Night Scene

Zhan

Wang

Jiang

et al. 2022

Entropy

View full text Add to dashboard Cite

The goal of infrared and visible image fusion in the night scene is to generate a fused image containing salient targets and rich textural details. However, the existing image fusion methods fail to take the unevenness of nighttime luminance into account. To address the above issue, an infrared and visible image fusion method for highlighting salient targets in the night scene is proposed. First of all, a global attention module is designed, which rescales the weights of different channels after capturing global contextual information. Second, the loss function is divided into the foreground loss and the background loss, forcing the fused image to retain rich texture details while highlighting the salient targets. Finally, a luminance estimation function is introduced to obtain the trade-off control parameters of the foreground loss function based on the nighttime luminance. It can effectively highlight salient targets by retaining the foreground information from the source images. Compared with other advanced methods, the experimental results adequately demonstrate the excellent fusion performance and generalization of the proposed method.

show abstract

“…To combine fusion tasks with a high-level visual task, a fusion method assisted by a high-level semantic task is proposed [55]. In addition, there are a few methods to investigate how lighting condition affects image fusion [19], [56].…”

Section: A Infrared and Visible Image Fusionmentioning

confidence: 99%

“…In recent years, with the rapid development of deep learning technology, researchers have explored fusion algorithms based on deep neural networks. Generally, the current mainstream deep fusion methods can be divided into three categories: methods based on autoencoder (AE) [18], methods based on convolutional neural network (CNN) [4], [19] and methods based on generative adversarial network (GAN) [9], [20]. As an image generation task, the existing infrared and visible image fusion methods lack in-depth exploration of the generation model.…”

mentioning

confidence: 99%

Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Models

Yue¹,

Fang²,

Xia³

et al. 2023

Preprint

View full text Add to dashboard Cite

Color plays an important role in human visual perception, reflecting the spectrum of objects. However, the existing infrared and visible image fusion methods rarely explore how to handle multi-spectral/channel data directly and achieve high color fidelity. This paper addresses the above issue by proposing a novel method with diffusion models, termed as Dif-Fusion, to generate the distribution of the multi-channel input data, which increases the ability of multi-source information aggregation and the fidelity of colors. In specific, instead of converting multi-channel images into single-channel data in existing fusion methods, we create the multi-channel data distribution with a denoising network in a latent space with forward and reverse diffusion process. Then, we use the denoising network to extract the multi-channel diffusion features with both visible and infrared information. Finally, we feed the multi-channel diffusion features to the multi-channel fusion module to directly generate the three-channel fused image. To retain the texture and intensity information, we propose multi-channel gradient loss and intensity loss. Along with the current evaluation metrics for measuring texture and intensity fidelity, we introduce a new evaluation metric to quantify color fidelity. Extensive experiments indicate that our method is more effective than other state-of-the-art image fusion methods, especially in color fidelity. The source code will be publicly available at https://github.com/GeoVectorMatrix/Dif-Fusion.

show abstract

PIAFusion: A progressive infrared and visible image fusion network based on illumination aware

Cited by 270 publications

References 50 publications

Infrared and Visible Image Fusion with Significant Target Enhancement

Infrared and Visible Image Fusion with Significant Target Enhancement

Infrared and Visible Image Fusion for Highlighting Salient Targets in the Night Scene

Dif-Fusion: Towards High Color Fidelity in Infrared and Visible Image Fusion with Diffusion Models

Contact Info

Product

Resources

About