TPFusion: Texture Preserving Fusion of Infrared and Visible Images via Dense Networks

Abstract

In this paper, we design an infrared (IR) and visible (VIS) image fusion via unsupervised dense networks, termed as TPFusion. Activity level measurements and fusion rules are indispensable parts of conventional image fusion methods. However, designing an appropriate fusion process is time-consuming and complicated. In recent years, deep learning-based methods are proposed to handle this problem. However, for multi-modality image fusion, using the same network cannot extract effective feature maps from source images that are obtained by different image sensors. In TPFusion, we can avoid this issue. At first, we extract the textural information of the source images. Then two densely connected networks are trained to fuse textural information and source image, respectively. By this way, we can preserve more textural details in the fused image. Moreover, loss functions we designed to constrain two densely connected convolutional networks are according to the characteristics of textural information and source images. Through our method, the fused image will obtain more textural information of source images. For proving the validity of our method, we implement comparison and ablation experiments from the qualitative and quantitative assessments. The ablation experiments prove the effectiveness of TPFusion. Being compared to existing advanced IR and VIS image fusion methods, our fusion results possess better fusion results in both objective and subjective aspects. To be specific, in qualitative comparisons, our fusion results have better contrast ratio and abundant textural details. In quantitative comparisons, TPFusion outperforms existing representative fusion methods.

Fusion of Infrared and Visible Images Based on Three-Scale Decomposition and ResNet Feature Transfer

“…Image fusion plays an important role in many fields, including the medical field, agricultural field, military field, etc. The main purpose of image fusion is to combine the feature information of images captured by multiple sensors into a single image to obtain a rich and comprehensive image, which plays a key role in subsequent image processing tasks [ 1 ]. Since infrared and visible image fusion has a wide range of applications, it is the most common type of fusion method.…”

Infrared and Visible Image Fusion for Highlighting Salient Targets in the Night Scene

“…Image fusion fuses multimodal images of the same scene to produce a fused image with richer information and clearer details [ 1 , 2 ]. Infrared and visible image fusion, as an indispensable part in image fusion direction, has been applied in all walks of life [ 3 , 4 , 5 ]. The fused image can be utilized as a preprocessing unit for subsequent high-level visual tasks such as object detection, object tracking and semantic segmentation.…”