LatLRR-CNN: an infrared and visible image fusion method combining latent low-rank representation and CNN

Yang, Yong; Gao, Chengrui; Ming, Zhangqiang; Guo, Jixiang; Leopold, Edou; Cheng, Junlong; Zuo, Jing; Zhu, Min

doi:10.1007/s11042-023-14967-0

Cited by 4 publications

(2 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Tao et al [ 61 ] proposed LatLRR-VGG19 which uses VGG19 to fuse the base parts, while the salient parts are fused using a summation strategy. In 2023, Yang and his team [ 55 ] presented LatLRR-CNN, where both the base parts and the salient parts were initially fused using CNN, and the final fused image was obtained by summing the two. These studies aim to enhance image fusion performance by leveraging LatLRR and various other feature extraction techniques.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Using Sparse Parts in Fused Information to Enhance Performance in Latent Low-Rank Representation-Based Fusion of Visible and Infrared Images

Hao,

Chen,

Ning

et al. 2024

Sensors

View full text Add to dashboard Cite

Latent Low-Rank Representation (LatLRR) has emerged as a prominent approach for fusing visible and infrared images. In this approach, images are decomposed into three fundamental components: the base part, salient part, and sparse part. The aim is to blend the base and salient features to reconstruct images accurately. However, existing methods often focus more on combining the base and salient parts, neglecting the importance of the sparse component, whereas we advocate for the comprehensive inclusion of all three parts generated from LatLRR image decomposition into the image fusion process, a novel proposition introduced in this study. Moreover, the effective integration of Convolutional Neural Network (CNN) technology with LatLRR remains challenging, particularly after the inclusion of sparse parts. This study utilizes fusion strategies involving weighted average, summation, VGG19, and ResNet50 in various combinations to analyze the fusion performance following the introduction of sparse parts. The research findings show a significant enhancement in fusion performance achieved through the inclusion of sparse parts in the fusion process. The suggested fusion strategy involves employing deep learning techniques for fusing both base parts and sparse parts while utilizing a summation strategy for the fusion of salient parts. The findings improve the performance of LatLRR-based methods and offer valuable insights for enhancement, leading to advancements in the field of image fusion.

show abstract

Section: Related Workmentioning

confidence: 99%

“…Latent Low-Rank Representation (LatLRR) has emerged as a recently employed method for image fusion [ 54 , 55 , 56 , 57 , 58 ]. LatLRR decomposes images into three components: base, salient, and sparse parts [ 59 ].…”

Section: Introductionmentioning

confidence: 99%

Using Sparse Parts in Fused Information to Enhance Performance in Latent Low-Rank Representation-Based Fusion of Visible and Infrared Images

Hao,

Chen,

Ning

et al. 2024

Sensors

View full text Add to dashboard Cite

show abstract

Integrating Parallel Attention Mechanisms and Multi-Scale Features for Infrared and Visible Image Fusion

Xu,

Zheng

2024

IEEE Access

View full text Add to dashboard Cite

Infrared and visible image fusion (IVIF) is used to synthesize a single image that contain rich information by integrating the advantages of these two modalities. However, current methods struggle to preserve the thermal radiation information from infrared images and textural information from visible images. To overcome this shortcoming, we present a fusion framework called, AMFusionNet, designed for IVIF. AMFusionNet contains three modules: multi-kernel convolution block (MKCBlock) combined with parallel spatial attention and channel attention modules (PSCNet), and a decoder. Specifically, multi-kernel convolution can not only extract a variety of feature information from source images but also expand the network's receptive field; To better extract salient information from infrared images, we introduced a parallel attention mechanism. This mechanism integrates the channel attention module and the spatial attention module in a parallel fashion, utilizing Gaussian Error Linear Units (GELU) as its nonlinear activation function. To improve the detailed preservation capabilities of the network, a multi-scale structural similarity (MS-SSIM) loss function is incorporated into the comprehensive loss function. Experiments on the TNO and FLIR datasets demonstrate that AMFusionNet outperforms other methods in object and subject evaluations. INDEX TERMS Infrared and visible image fusion, Parallel attention mechanism, Multi-kernel convolution, MS-SSIM.

show abstract

Infrared and Harsh Light Visible Image Fusion Using an Environmental Light Perception Network

Yan,

Gao,

et al. 2024

Entropy

View full text Add to dashboard Cite

The complementary combination of emphasizing target objects in infrared images and rich texture details in visible images can effectively enhance the information entropy of fused images, thereby providing substantial assistance for downstream composite high-level vision tasks, such as nighttime vehicle intelligent driving. However, mainstream fusion algorithms lack specific research on the contradiction between the low information entropy and high pixel intensity of visible images under harsh light nighttime road environments. As a result, fusion algorithms that perform well in normal conditions can only produce low information entropy fusion images similar to the information distribution of visible images under harsh light interference. In response to these problems, we designed an image fusion network resilient to harsh light environment interference, incorporating entropy and information theory principles to enhance robustness and information retention. Specifically, an edge feature extraction module was designed to extract key edge features of salient targets to optimize fusion information entropy. Additionally, a harsh light environment aware (HLEA) module was proposed to avoid the decrease in fusion image quality caused by the contradiction between low information entropy and high pixel intensity based on the information distribution characteristics of harsh light visible images. Finally, an edge-guided hierarchical fusion (EGHF) module was designed to achieve robust feature fusion, minimizing irrelevant noise entropy and maximizing useful information entropy. Extensive experiments demonstrate that, compared to other advanced algorithms, the method proposed fusion results contain more useful information and have significant advantages in high-level vision tasks under harsh nighttime lighting conditions.

show abstract

LatLRR-CNN: an infrared and visible image fusion method combining latent low-rank representation and CNN

Cited by 4 publications

References 46 publications

Using Sparse Parts in Fused Information to Enhance Performance in Latent Low-Rank Representation-Based Fusion of Visible and Infrared Images

Using Sparse Parts in Fused Information to Enhance Performance in Latent Low-Rank Representation-Based Fusion of Visible and Infrared Images

Integrating Parallel Attention Mechanisms and Multi-Scale Features for Infrared and Visible Image Fusion

Infrared and Harsh Light Visible Image Fusion Using an Environmental Light Perception Network

Contact Info

Product

Resources

About