Exploring efficient and effective generative adversarial network for thermal infrared image colorization

Abstract

Thermal infrared image colorization is very difficult, and colorized images suffer from poor texture detail recovery and low color matching. To solve the above problems, this paper proposes an Efficient and Effective Generative Adversarial Network (E2GAN). This paper proposes multi-level dense module, feature fusion module, and color-aware attention module in the improved generator. Adding multi-level dense module can enhance the feature extraction capability and the improve detail recovery capability Using the feature fusion module in the middle of the encoder–decoder reduces the information loss caused by encoder down-sampling and improves the prediction of fine color of the image. Using the color-aware attention module during up-sampling allows for capturing more semantic details, focusing on more key objects, and generating high-quality colorized images. And the proposed discriminator is the PatchGAN with color-aware attention module, which enhances its ability to discriminate between true and false colorized images. Meanwhile, this paper proposes a novel composite loss function that can improve the quality of colorized images, generate fine local details, and recover semantic and texture information. Extensive experiments demonstrate that the proposed E2GAN has significantly improved SSIM, PSNR, LPIPS, and NIQE on the KAIST dataset and the FLIR dataset compared to existing methods.