Infrared and Visible Image Fusion Using Visual Saliency Sparse Representation and Detail Injection Model

Yang, Yong; Zhang, Yingmei; Huang, Shuying; Zuo, Yifan; Sun, Jiancheng

doi:10.1109/tim.2020.3011766

Cited by 60 publications

(34 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is known to us that just one evaluation index could not well demonstrate the quality of fused images in quantitative assessment. Thus, for the sake of making a comprehensive evaluation for the fusion images, six popular fusion evaluation metrics are introduced in this section, namely visual information fidelity for fusion (VIFF) [ 29 , 30 , 31 , 32 , 33 ], Q S [ 34 ], average gradient (AG) [ 20 , 35 , 36 ], correlation coefficient (CC) [ 20 , 37 , 38 ], spatial frequency (SF) [ 20 , 39 , 40 , 41 ], and Q W [ 34 , 42 ]. In terms of all the six metrics, the higher the value data of the evaluation index, the better the fusion performance will be.…”

Section: Resultsmentioning

confidence: 99%

Saliency-Guided Nonsubsampled Shearlet Transform for Multisource Remote Sensing Image Fusion

2021

Sensors

View full text Add to dashboard Cite

The rapid development of remote sensing and space technology provides multisource remote sensing image data for earth observation in the same area. Information provided by these images, however, is often complementary and cooperative, and multisource image fusion is still challenging. This paper proposes a novel multisource remote sensing image fusion algorithm. It integrates the contrast saliency map (CSM) and the sum-modified-Laplacian (SML) in the nonsubsampled shearlet transform (NSST) domain. The NSST is utilized to decompose the source images into low-frequency sub-bands and high-frequency sub-bands. Low-frequency sub-bands reflect the contrast and brightness of the source images, while high-frequency sub-bands reflect the texture and details of the source images. Using this information, the contrast saliency map and SML fusion rules are introduced into the corresponding sub-bands. Finally, the inverse NSST reconstructs the fusion image. Experimental results demonstrate that the proposed multisource remote image fusion technique performs well in terms of contrast enhancement and detail preservation.

show abstract

Section: Resultsmentioning

confidence: 99%

Saliency-Guided Nonsubsampled Shearlet Transform for Multisource Remote Sensing Image Fusion

2021

Sensors

View full text Add to dashboard Cite

show abstract

“…Li and Wu [24] employed the encode/decode network architecture and introduced the densely connected convolution layer in the encoder to extract the features of the source image to avoid losing information in the convolution process. Yang et al [25] proposed a fusion model based on visual saliency sparse representation and detail injection to avoid the loss of significant thermal radiation targets of infrared images. Zhang et al [26] propose an image fusion network based on proportional maintenance of gradient and intensity, named PMGI, which can preserve source image information through the gradient and intensity path.…”

Section: Deep Learning-based Methodsmentioning

confidence: 99%

A Generative Adversarial Network with Dual Discriminators for Infrared and Visible Image Fusion Based on Saliency Detection

Zhang¹,

Hou

et al. 2021

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Infrared and visible image fusion needs to preserve both the salient target of the infrared image and the texture details of the visible image. Therefore, an infrared and visible image fusion method based on saliency detection is proposed. Firstly, the saliency map of the infrared image is obtained by saliency detection. Then, the specific loss function and network architecture are designed based on the saliency map to improve the performance of the fusion algorithm. Specifically, the saliency map is normalized to [0, 1], used as a weight map to constrain the loss function. At the same time, the saliency map is binarized to extract salient regions and nonsalient regions. And, a generative adversarial network with dual discriminators is obtained. The two discriminators are used to distinguish the salient regions and the nonsalient regions, respectively, to promote the generator to generate better fusion results. The experimental results show that the fusion results of our method are better than those of the existing methods in both subjective and objective aspects.

show abstract

“…Saliency-based methods [ 23 , 24 ] imitate the feature of the human visual system, which is easily attracted to prominent objects, and improve the visual effect of fused images by preserving the integrity of salient objects. Zhang et al [ 25 ] utilized salient target areas of an infrared image to determine the fusion weights, which can ensure that the fusion results have more significant thermal radiation targets.…”

Section: Related Workmentioning

confidence: 99%

A Generative Adversarial Network for Infrared and Visible Image Fusion Based on Semantic Segmentation

Hou

Zhang

et al. 2021

Entropy

View full text Add to dashboard Cite

This paper proposes a new generative adversarial network for infrared and visible image fusion based on semantic segmentation (SSGAN), which can consider not only the low-level features of infrared and visible images, but also the high-level semantic information. Source images can be divided into foregrounds and backgrounds by semantic masks. The generator with a dual-encoder-single-decoder framework is used to extract the feature of foregrounds and backgrounds by different encoder paths. Moreover, the discriminator’s input image is designed based on semantic segmentation, which is obtained by combining the foregrounds of the infrared images with the backgrounds of the visible images. Consequently, the prominence of thermal targets in the infrared images and texture details in the visible images can be preserved in the fused images simultaneously. Qualitative and quantitative experiments on publicly available datasets demonstrate that the proposed approach can significantly outperform the state-of-the-art methods.

show abstract

Infrared and Visible Image Fusion Using Visual Saliency Sparse Representation and Detail Injection Model

Cited by 60 publications

References 41 publications

Saliency-Guided Nonsubsampled Shearlet Transform for Multisource Remote Sensing Image Fusion

Saliency-Guided Nonsubsampled Shearlet Transform for Multisource Remote Sensing Image Fusion

A Generative Adversarial Network with Dual Discriminators for Infrared and Visible Image Fusion Based on Saliency Detection

A Generative Adversarial Network for Infrared and Visible Image Fusion Based on Semantic Segmentation

Contact Info

Product

Resources

About