Infrared and visible image fusion with edge detail implantation

Liu, Junyu; Zhang, Yafei; Li, Fan

doi:10.3389/fphy.2023.1180100

Cited by 2 publications

(2 citation statements)

References 52 publications

(53 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DL-based methods can automatically extract meaningful features and fuse the images without the support of default fusion strategies. However, deep neural networks are time consuming and not suitable for real-time computer applications [18,19]. In addition, these methods can only be applied to images with a similar scene content as the training dataset, and their generalization capabilities are limited by the size of the dataset.…”

Section: Thermal Infrared and Visible Image Fusionmentioning

confidence: 99%

Nighttime Thermal Infrared Image Translation Integrating Visible Images

Yang,

Sun,

Lou

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

Nighttime Thermal InfraRed (NTIR) image colorization, also known as the translation of NTIR images into Daytime Color Visible (DCV) images, can facilitate human and intelligent system perception of nighttime scenes under weak lighting conditions. End-to-end neural networks have been used to learn the mapping relationship between temperature and color domains, and translate NTIR images with one channel into DCV images with three channels. However, this mapping relationship is an ill-posed problem with multiple solutions without constraints, resulting in blurred edges, color disorder, and semantic errors. To solve this problem, an NTIR2DCV method that includes two steps is proposed: firstly, fuse Nighttime Color Visible (NCV) images with NTIR images based on an Illumination-Aware, Multilevel Decomposition Latent Low-Rank Representation (IA-MDLatLRR) method, which considers the differences in illumination conditions during image fusion and adjusts the fusion strategy of MDLatLRR accordingly to suppress the adverse effects of nighttime lights; secondly, translate the Nighttime Fused (NF) image to DCV image based on HyperDimensional Computing Generative Adversarial Network (HDC-GAN), which ensures feature-level semantic consistency between the source image (NF image) and the translated image (DCV image) without creating semantic label maps. Extensive comparative experiments and the evaluation metrics values show that the proposed algorithms perform better than other State-Of-The-Art (SOTA) image fusion and translation methods, such as FID and KID, which decreased by 14.1 and 18.9, respectively.

show abstract

Section: Thermal Infrared and Visible Image Fusionmentioning

confidence: 99%

Nighttime Thermal Infrared Image Translation Integrating Visible Images

Yang,

Sun,

Lou

et al. 2024

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…How to effectively integrate multi-sensor, high-resolution, multispectral, and multi-temporal remote sensing data for fusion processing has become a hot and key research topic in the field of remote sensing at present. Multi-source image matching [2,3], especially the matching between optical and SAR images [4,5], is one of the core problems that urgently needs to be solved. However, due to the completely different imaging mechanisms, there are radiation anomalies, geometric differences, and scale differences between optical and SAR images.…”

Section: Introductionmentioning

confidence: 99%

Structure similarity virtual map generation network for optical and SAR image matching

Chen,

Mei

2024

Front. Phys.

View full text Add to dashboard Cite

Introduction: Optical and SAR image matching is one of the fields within multi-sensor imaging and fusion. It is crucial for various applications such as disaster response, environmental monitoring, and urban planning, as it enables comprehensive and accurate analysis by combining the visual information of optical images with the penetrating capability of SAR images. However, the differences in imaging mechanisms between optical and SAR images result in significant nonlinear radiation distortion. Especially for SAR images, which are affected by speckle noises, resulting in low resolution and blurry edge structures, making optical and SAR image matching difficult and challenging. The key to successful matching lies in reducing modal differences and extracting similarity information from the images.Method: In light of this, we propose a structure similarity virtual map generation network (SVGNet) to address the task of optical and SAR image matching. The core innovation of this paper is that we take inspiration from the concept of image generation, to handle the predicament of image matching between different modalities. Firstly, we introduce the Attention U-Net as a generator to decouple and characterize optical images. And then, SAR images are consistently converted into optical images with similar textures and structures. At the same time, using the structural similarity (SSIM) to constrain structural spatial information to improve the quality of generated images. Secondly, a conditional generative adversarial network is employed to further guide the image generation process. By combining synthesized SAR images and their corresponding optical images in a dual channel, we can enhance prior information. This combined data is then fed into the discriminator to determine whether the images are true or false, guiding the generator to optimize feature learning. Finally, we employ least squares loss (LSGAN) to stabilize the training of the generative adversarial network.Results and Discussion: Experiments have demonstrated that the SVGNet proposed in this paper is capable of effectively reducing modal differences, and it increases the matching success rate. Compared to direct image matching, using image generation ideas results in a matching accuracy improvement of more than twice.

show abstract

Infrared and visible image fusion with edge detail implantation

Cited by 2 publications

References 52 publications

Nighttime Thermal Infrared Image Translation Integrating Visible Images

Nighttime Thermal Infrared Image Translation Integrating Visible Images

Structure similarity virtual map generation network for optical and SAR image matching

Contact Info

Product

Resources

About