TPFNet: a tri-band polarimetric image fusion neural network

Abstract

In current image fusion techniques, image fusion is usually performed on a dual-band image to obtain a fused image with significant target information, or on an intensity image and a polarization image to obtain an image with stronger visual perception. If more information is to be obtained in a single image, tri-band fusion and intensity/polarization image fusion techniques can be combined. In order to solve the above problems, in this paper, we have acquired some tri-band polarization images through a common aperture multispectral polarization photoelectric device, which contains intensity and polarization visible (VIS) images, as well as the intensity images of near-infrared (NIR) and long-wave infrared (LIR). Besides, in order to obtain good image fusion results, we built an end-to-end self-supervised image fusion network and designed an efficient loss function to train the network. We conducted experiments on TPFNet on the acquired dataset and compared it with other image fusion algorithms. The results show that TPFNet achieves excellent results in both subjective and objective evaluations.