Visible-Image-Assisted Nonuniformity Correction of Infrared Images Using the GAN with SEBlock

Abstract

Aiming at reducing image detail loss and edge blur in the existing nonuniformity correction (NUC) methods, a new visible-image-assisted NUC algorithm based on a dual-discriminator generative adversarial network (GAN) with SEBlock (VIA-NUC) is proposed. The algorithm uses the visible image as a reference for better uniformity. The generative model downsamples the infrared and visible images separately for multiscale feature extraction. Then, image reconstruction is achieved by decoding the infrared feature maps with the assistance of the visible features at the same scale. During decoding, SEBlock, a channel attention mechanism, and skip connection are used to ensure that more distinctive channel and spatial features are extracted from the visible features. Two discriminators based on vision transformer (Vit) and discrete wavelet transform (DWT) were designed, which perform global and local judgments on the generated image from the texture features and frequency domain features of the model, respectively. The results are then fed back to the generator for adversarial learning. This approach can effectively remove nonuniform noise while preserving the texture. The performance of the proposed method was validated using public datasets. The average structural similarity (SSIM) and average peak signal-to-noise ratio (PSNR) of the corrected images exceeded 0.97 and 37.11 dB, respectively. The experimental results show that the proposed method improves the metric evaluation by more than 3%.

Scene-Based Nonuniformity Correction Method Using Principal Component Analysis for Infrared Focal Plane Arrays

“…Moreover, the unidirectional total variation model [15], weighted least squares [16], temporal filter [17] as well as spatial correlation [18] have been taken into consideration in order to gain a balance between complexity and real-time performance. Furthermore, with the development of various kinds of neural networks, methods based on deep learning have been gradually introduced into stripe removal and nonuniformity correction [19][20][21]. All the NUC methods face the same problems of convergence speed and "ghosting artifacts".…”