Infrared-visible image fusion method based on sparse and prior joint saliency detection and LatLRR-FPDE

Li, Yonghua; Liu, Gang; Bavirisetti, Durga Prasad; Gu, Xinjie; Zhou, Xihong

doi:10.1016/j.dsp.2023.103910

Cited by 17 publications

(4 citation statements)

References 68 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Saliency-based approaches are used to capture and preserve critical features from both infrared and visible images within the fused image. These techniques not only improve the visibility of objects and scenes under low-contrast or low-light conditions but also strengthen the detection and recognition functions of computer vision systems [42], [43]. Saliency-based methods follow three main steps.…”

Section: A Pixel-level Image Fusion Algorithmsmentioning

confidence: 99%

Integrating Parallel Attention Mechanisms and Multi-Scale Features for Infrared and Visible Image Fusion

Xu,

Zheng

2024

IEEE Access

View full text Add to dashboard Cite

Infrared and visible image fusion (IVIF) is used to synthesize a single image that contain rich information by integrating the advantages of these two modalities. However, current methods struggle to preserve the thermal radiation information from infrared images and textural information from visible images. To overcome this shortcoming, we present a fusion framework called, AMFusionNet, designed for IVIF. AMFusionNet contains three modules: multi-kernel convolution block (MKCBlock) combined with parallel spatial attention and channel attention modules (PSCNet), and a decoder. Specifically, multi-kernel convolution can not only extract a variety of feature information from source images but also expand the network's receptive field; To better extract salient information from infrared images, we introduced a parallel attention mechanism. This mechanism integrates the channel attention module and the spatial attention module in a parallel fashion, utilizing Gaussian Error Linear Units (GELU) as its nonlinear activation function. To improve the detailed preservation capabilities of the network, a multi-scale structural similarity (MS-SSIM) loss function is incorporated into the comprehensive loss function. Experiments on the TNO and FLIR datasets demonstrate that AMFusionNet outperforms other methods in object and subject evaluations. INDEX TERMS Infrared and visible image fusion, Parallel attention mechanism, Multi-kernel convolution, MS-SSIM.

show abstract

Section: A Pixel-level Image Fusion Algorithmsmentioning

confidence: 99%

Integrating Parallel Attention Mechanisms and Multi-Scale Features for Infrared and Visible Image Fusion

Xu,

Zheng

2024

IEEE Access

View full text Add to dashboard Cite

show abstract

“…The above operations of E‐step and M‐step are repeated until convergence. The EM algorithm is commonly used in conventional methods, such as the infrared and visible light image fusion method based on LatLRR and FPED proposed by Li et al [ 28 ] In this method, EM algorithm is applied to the fusion of high‐frequency details to capture small differences in the grayscale, so that the fused results retain more details. Although this method has excellent fusion performance, it still cannot avoid the adverse effects caused by manual fusion rules in conventional methods.…”

Section: Related Workmentioning

confidence: 99%

EMA‐GAN: A Generative Adversarial Network for Infrared and Visible Image Fusion with Multiscale Attention Network and Expectation Maximization Algorithm

Xi,

Jin,

Jiang

et al. 2023

Advanced Intelligent Systems

View full text Add to dashboard Cite

The purpose of the infrared and visible image fusion is to generate a fused image with rich information. Although most fusion methods can achieve good performance, there are still shortcomings in extracting feature information from source images, which make it difficult to balance the thermal radiation region information and texture detail information in the fused image. To address the above issues, an expectation maximization (EM) learning framework based on adversarial generative networks (GAN) for infrared and visible image fusion is proposed. The EM algorithm (EMA) can obtain maximum likelihood estimation for problems with potential variables, which is helpful in solving the problem of lack of labels in infrared and visible image fusion. The axial‐corner attention mechanism is designed to capture long‐range semantic information and texture information of the visible image. The multifrequency attention mechanism digs the relationships between features at different scales to highlight target information of infrared images in the fused result. Meanwhile, two discriminators are used to balance two different features, and a new loss function is designed to maximize the likelihood estimate of the data with soft class label assignments, which is obtained from the expectation network. Extensive experiments demonstrate the superiority of EMA‐GAN over the state‐of‐the‐art.

show abstract

“…With the rapid development of image stitching and image fusion technologies, methods for obtaining multi-view or even global perspectives through multiple single viewpoints have been widely applied in human production and daily life [1][2][3][4][5]. For instance, the extensive use of technologies such as panoramic images, autonomous driving, and virtual reality (VR) enables the precise remote observation of scenes by individuals [6][7][8].…”

Section: Introductionmentioning

confidence: 99%

Rectification for Stitched Images with Deformable Meshes and Residual Networks

Fan,

Mao,

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Image stitching is an important method for digital image processing, which is often prone to the problem of the irregularity of stitched images after stitching. And the traditional image cropping or complementation methods usually lead to a large number of information loss. Therefore, this paper proposes an image rectification method based on deformable mesh and residual network. The method aims to minimize the information loss at the edges of the spliced image and the information loss inside the image. Specifically, the method can select the most suitable mesh shape for residual network regression according to different images. Its loss function includes global loss and local loss, aiming to minimize the loss of image information within the grid and global target. The method in this paper not only greatly reduces the information loss caused by irregular shapes after image stitching, but also adapts to different images with various rigid structures. Meanwhile, its validation on the DIR-D dataset shows that the method outperforms the state-of-the-art methods in image rectification.

show abstract

Infrared-visible image fusion method based on sparse and prior joint saliency detection and LatLRR-FPDE

Cited by 17 publications

References 68 publications

Integrating Parallel Attention Mechanisms and Multi-Scale Features for Infrared and Visible Image Fusion

Integrating Parallel Attention Mechanisms and Multi-Scale Features for Infrared and Visible Image Fusion

EMA‐GAN: A Generative Adversarial Network for Infrared and Visible Image Fusion with Multiscale Attention Network and Expectation Maximization Algorithm

Rectification for Stitched Images with Deformable Meshes and Residual Networks

Contact Info

Product

Resources

About