Single-image-based nonuniformity correction of uncooled long-wave infrared detectors: a deep-learning approach

Abstract

Fixed-pattern noise (FPN), which is caused by the nonuniform opto-electronic responses of microbolometer focal-plane-array (FPA) optoelectronics, imposes a challenging problem in infrared imaging systems. In this paper, we successfully demonstrate that a better single-image-based non-uniformity correction (NUC) operator can be directly learned from a large number of simulated training images instead of being handcrafted as before. Our proposed training scheme, which is based on convolutional neural networks (CNNs) and a column FPN simulation module, gives rise to a powerful technique to reconstruct the noise-free infrared image from its corresponding noisy observation. Specifically, a comprehensive column FPN model is utilized to depict the nonlinear characteristics of column amplifiers in the readout circuit of FPA. A large number of high-fidelity training images are simulated based on this model and the end-to-end residual deep network is capable of learning the intrinsic difference between undesirable FPN and original image details. Therefore, column FPN can be accurately estimated and further subtracted from the raw infrared images to obtain NUC results. Comparative results with state-of-the-art single-image-based NUC methods, using real-captured noisy infrared images, demonstrate that our proposed deep-learning-based approach delivers better performances of FPN removal, detail preservation, and artifact suppression.

Temporal-Spatial Nonlinear Filtering for Infrared Focal Plane Array Stripe Nonuniformity Correction

“…In experiment 2, four representative NUC algorithms were selected to compare with our method in terms of performance. Figure 12 shows the results of TVRNN-NUC [15], BF-THPF NUC [12], MIRE NUC [21] and CNN NUC [23].…”

“…In experiment 2, four representative NUC algorithms were selected to compare with our method in terms of performance. Figure 12 shows the results of TVRNN-NUC [15], BF-THPF NUC [12], MIRE NUC [21] and CNN NUC [23]. Furthermore, in order to see more clearly the difference between our method and BF-THPF NUC method in scene detail preservation, Figure 13 shows 3D views of the images in Figure 12c2,f2, which were corrected via BF-THPF and the proposed NUC method.…”

“…More recently, Kuang et al [22] utilized deep convolutional neural networks to directly predict the correction result. He et al [23] used deep residual networks to end-to-end estimate the nonuniformity. Zeng et al [24] proposed a two-stage filtering NUC method which was suitable for stripe FPN correction.…”

Dual-Branch Structured De-Striping Convolution Network Using Parametric Noise Model

“…Among the aforementioned non-uniformities, due to the inherent property of infrared detector, FPN can appear in the raw infrared image even though uniform energy is provided to an infrared detector. FPN has vertical directionality and periodicity because the detector readout circuit (ROIC) process the signal of entire FPA in row by row, whereas the pixels in the same column share a column-wise multiplexer [3], [4]. As shown in Fig.…”

“…Kuang et al [11] introduced a simple three-layer convolutional network for the stripe FPN removal from single infrared image. He et al [4] implemented a residual deep network-based FPN de-striping method to compute residual information through learning. On the one hand, Xiao et al [8] presented a deep convolutional network utilizing a local-global combination model for infrared cloud images with FPN.…”