Visible and Near-Infrared Image Synthesis Using PCA Fusion of Multiscale Layers

Abstract

This study proposes a method of blending visible and near-infrared (NIR) images to enhance their edge details and local contrast based on the Laplacian pyramid and principal component analysis (PCA). In the proposed method, both the Laplacian pyramid and PCA are implemented to generate a radiance map. Using the PCA algorithm, the soft-mixing method and the mask-skipping filter were applied when the images were fused. The color compensation method uses the ratio between the radiance map fused by the Laplacian pyramid and the PCA algorithm and the luminance channel of the visible image to preserve the chrominance of the visible image. The results show that the proposed method improves edge details and local contrast effectively.

Tone Image Classification and Weighted Learning for Visible and NIR Image Fusion

“…Laplacian–Gaussian pyramids and the local entropy-fusion algorithm first make a weight map by measuring local contrast, local entropy, and visibility, then perform multiple resolution decomposition on visible and NIR images through the Laplacian–Gaussian pyramid, finally reconstruct image through the weight map. The Laplacian pyramid is a multi-scale processing algorithm that decomposes input images into multi-scale images [ 4 ]. To generate Laplacian pyramid, up-sample the Gaussian pyramid and subtract the previous level image of the Gaussian pyramid to obtain a differential image.…”

“…However, since the infrared image contains detailed information that is not expressed in the visible light image, the visibility of the image can be improved by the information fusion method. For example, in the case of fog and haze, since infrared rays have a stronger penetrability to particles than visible rays, more detailed information can be obtained from infrared images [ 3 , 4 ].…”

AFSFusion: An Adjacent Feature Shuffle Combination Network for Infrared and Visible Image Fusion

“…On the other hand, visible images can capture high-resolution texture details. Therefore, researchers have become interested in the optimal integration of complementary information from IR and visible images through image fusion techniques to compensate for sensor limitations and enhance human and machine perception [1][2][3][4][5]. The fusion of IR and visible images has a wide range of applications, including intelligent surveillance, target monitoring, and video analysis.…”

“…Traditional methods for image fusion can generally be categorized into three stages, namely feature extraction, feature fusion, and image reconstruction [1][2][3]. Typically, the fusion method employed in this case involves converting the image from the spatial domain into a domain that enables the extraction of features, which are then integrated using specific principles.…”

“…The enhancement of deep learning techniques [6] over the last decade and their successful application to the field of low-level vision processing have resulted in the development of various types of network models for IR and visible image fusion by researchers in this field [7][8][9][10][11][12][13][14][15][16][17][18][19]. Based on the stage of IR and visible image feature fusion, this process can generally be classified into the following three types [20]: (1) early fusion [7][8][9], which involves merging the original images at a low-level feature extraction stage. For instance, an end-to-end fusion model using detail-preserving adversarial learning (DPAL) techniques was proposed by Ma et al [7].…”