Fusion of visible and infrared images using global entropy and gradient constrained regularization

Zhao, Jincheng; Cui, Guangmang; Gong, Xiaoli; Zang, Yue; Tao, Shuyin; Wang, Daodang

doi:10.1016/j.infrared.2017.01.012

Cited by 83 publications

(20 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Prior to introducing RGB and NIR image fusion, we first review RGB and IR image fusion. The research on RGB and IR image fusion has a long history, which can be divided into seven categories: multiscale transform [10], [18], [35], sparse representation [11], [28], neural networks [8], [29], subspace methods [1], [7], and saliency-based methods [33], [36], hybrid models [12], [15], and other methods [14], [37]. NIR images provide higher resolution and better details than IR ones in low light condition.…”

Section: A Related Workmentioning

confidence: 99%

Scale-Aware Multispectral Fusion of RGB and NIR Images Based on Alternating Guidance

Zhou

Jung

2020

IEEE Access

View full text Add to dashboard Cite

In low light condition, color (RGB) images captured by increasing the camera ISO contain much noise and detail loss. However, near infrared (NIR) images are robust to noise and have clear textures without color. In this paper, we propose scale-aware multispectral fusion of RGB and NIR images based on alternating guidance. Low light RGB images provide large-scale image structure and color information, while NIR images have fine details lost in RGB images. Since they are complementary, we adopt alternating guidance for the fusion of them using weighted least squares (WLS). First, we perform the first guidance to denoise the RGB image and obtain base layer. Then, we conduct the second guidance for scale-aware detail transfer of the NIR image and yield detail layer. Finally, we combine the base and detail layers to generate a fusion image. We maximize the multispectral advantage of RGB and NIR images for fusion based on alternating guidance. Experimental results show that the proposed method achieves good performance in noise reduction, detail transfer and color reproduction, and is superior to the state-of-the-art ones in terms of quantitative measurement and computational efficiency. INDEX TERMS Image fusion, alternating guidance, detail transfer, weighted least squares, scale-aware.

show abstract

Section: A Related Workmentioning

confidence: 99%

Scale-Aware Multispectral Fusion of RGB and NIR Images Based on Alternating Guidance

Zhou

Jung

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Image fusion based on the transformation domain involves transforming multisource images, combining coefficients from the transformation to obtain transformation coefficients of the fused images, and conducting inverse transformation to obtain the fused images. Common fusion algorithms based on the transform domain include those based on the discrete cosine transform (DCT), the fast Fourier transform (FFT), the multiscale transform [3][4][5], image subspace technology [6,7], the saliency method [8,9], the sparse representation method [10,11], and others [12][13][14][15].…”

Section: Introductionmentioning

confidence: 99%

The Fusion of Unmatched Infrared and Visible Images Based on Generative Adversarial Networks

Zhao

Wang

et al. 2020

Mathematical Problems in Engineering

View full text Add to dashboard Cite

Visible images contain clear texture information and high spatial resolution but are unreliable under nighttime or ambient occlusion conditions. Infrared images can display target thermal radiation information under day, night, alternative weather, and ambient occlusion conditions. However, infrared images often lack good contour and texture information. Therefore, an increasing number of researchers are fusing visible and infrared images to obtain more information from them, which requires two completely matched images. However, it is difficult to obtain perfectly matched visible and infrared images in practice. In view of the above issues, we propose a new network model based on generative adversarial networks (GANs) to fuse unmatched infrared and visible images. Our method generates the corresponding infrared image from a visible image and fuses the two images together to obtain more information. The effectiveness of the proposed method is verified qualitatively and quantitatively through experimentation on public datasets. In addition, the generated fused images of the proposed method contain more abundant texture and thermal radiation information than other methods.

show abstract

“…According to the fusion strategies and theories adopted [6], several representative infrared and visible image fusion algorithms have been proposed, including multi-scale transform- [7][8][9], sparse representation- [10,11], neural network- [12,13], subspace- [14,15], and saliency-based [16,17] methods, hybrid models [18,19], and other methods [20,21]. These methods are widely used and still studied by many researchers.…”

Section: Introductionmentioning

confidence: 99%

Infrared and Visible Image Fusion with a Generative Adversarial Network and a Residual Network

Wang

et al. 2020

Applied Sciences

View full text Add to dashboard Cite

Infrared and visible image fusion can obtain combined images with salient hidden objectives and abundant visible details simultaneously. In this paper, we propose a novel method for infrared and visible image fusion with a deep learning framework based on a generative adversarial network (GAN) and a residual network (ResNet). The fusion is accomplished with an adversarial game and directed by the unique loss functions. The generator with residual blocks and skip connections can extract deep features of source image pairs and generate an elementary fused image with infrared thermal radiation information and visible texture information, and more details in visible images are added to the final images through the discriminator. It is unnecessary to design the activity level measurements and fusion rules manually, which are now implemented automatically. Also, there are no complicated multi-scale transforms in this method, so the computational cost and complexity can be reduced. Experiment results demonstrate that the proposed method eventually gets desirable images, achieving better performance in objective assessment and visual quality compared with nine representative infrared and visible image fusion methods.

show abstract

Fusion of visible and infrared images using global entropy and gradient constrained regularization

Cited by 83 publications

References 26 publications

Scale-Aware Multispectral Fusion of RGB and NIR Images Based on Alternating Guidance

Scale-Aware Multispectral Fusion of RGB and NIR Images Based on Alternating Guidance

The Fusion of Unmatched Infrared and Visible Images Based on Generative Adversarial Networks

Infrared and Visible Image Fusion with a Generative Adversarial Network and a Residual Network

Contact Info

Product

Resources

About