Fixed pattern noise reduction for infrared images based on cascade residual attention CNN

Abstract

Existing fixed pattern noise reduction (FPNR) methods are easily affected by the motion state of the scene and working condition of the image sensor, which leads to over smooth effects, ghosting artifacts as well as slow convergence rate. To address these issues, we design an innovative cascade convolution neural network (CNN) model with residual skip connections to realize single frame blind FPNR operation without any parameter tuning. Moreover, a coarse-fine convolution (CF-Conv) unit is introduced to extract complementary features in various scales and fuse them to pick more spatial information. Inspired by the success of the visual attention mechanism, we further propose a particular spatial-channel noise attention unit (SCNAU) to separate the scene details from fixed pattern noise more thoroughly and recover the real scene more accurately.Experimental results on test data demonstrate that the proposed cascade CNN-FPNR method outperforms the existing FPNR methods in both of visual effect and quantitative assessment. affected by the fixed pattern noise (FPN), which is mainly caused by the spatial non-uniform response of individual detectors in the sensor [6][7]. More seriously, spatial FPN generally drifts with time, which makes the problem be more challenging [8][9][10][11]. As a result, the FPN causes a significant decline in imaging quality and decreases the precision for object detection and recognition. To meet this challenge, the cost-effective fixed pattern noise reduction (FPNR) techniques based on signal processing are continually investigated and applied in nearly all the infrared imaging systems.Existing FPNR algorithms are mainly divided into two primary categories: reference-based FPNR (RB-FPNR) and scene-based FPNR (SB-FPNR) [12][13][14]. The RB-FPNR methods remove the FPN according to fixed calibration parameters calculated from the response of blackbody radiation at different temperatures [15]. Unfortunately, such a calibration requires the camera to halt the normal operation and update the calibration parameters due to the inherent temporal drift of detector characteristics [16]. Given this fact, most of the recent researches have focused on developing SB-FPNR methods, such as neural networks (NN) [17], temporal high-pass filter (THPF) [18,19] and constant-statistics (CS) method [20][21]. As for SB-FPNR algorithms, the calibration parameters are iteratively updated by utilizing the information extracted from inter-frame motion, therefore, ghosting artifacts and over smooth effects resulted from the sudden deceleration of scene motion often seriously degrade the noise reduction performance, moreover, the relatively slow convergence process occurred in scene switching is unacceptable for most of the practical applications.In recent years, convolution neural network (CNN) [22] models were explored deeply and applied in various image processing tasks [23], such as image super resolution [24,25], image denoising [26], and sketch synthesis [27][28][29]. To the best of our knowledge, CNN based FPNR m...

Fourier Domain Anomaly Detection and Spectral Fusion for Stripe Noise Removal of TIR Imagery

“…Spatial methods for destriping can be roughly divided into 1D-filtering-based methods [11,16,17], statistics-based methods [12][13][14][18][19][20], optimization-based methods [21][22][23][24][25][26][27][28][29][30], and deep-learning-based methods [31][32][33][34]. Currently, 1D-filtering-based methods are mainly performed on conventional TIR images, and their basic idea is to utilize 1D edge-preserving filters to progressively separate stripe noise from the contaminated image in horizontal and vertical directions.…”

Remote Sensing Image Fusion via Boundary Measured Dual-Channel PCNN in Multi-Scale Morphological Gradient Domain

“…The expressions of a simplified dual-channel PCNN model can be defined as As is shown in Fig. 5, S 1 ij and S 2 ij denote the pixel value of two input images at the point (i, j) in this neural network; L ij represents the linking parameter; β 1 ij and β 2 ij denote the linking strength; F 1 ij and F 2 ij represent the feedback of inputs. U ij is the ouput of the dual-channel.…”

“…Nowadays, with the rapid development of remote sensing technology, there are more types of remote sensing images are available when imaging a view [1], [2]. As a widely used remote sensing image, multispectral (MS) images containing plentiful spectral information are applied in the fields of disaster monitoring [3], marine research [4], object recognition [5], etc.…”

Dual-Branch Structured De-Striping Convolution Network Using Parametric Noise Model

“…From results of experimental analysis, they exploited a non-linear cubic FPN model to be used their training data [4]. Other approaches ( [7], [11], [12], and [35]) define the stripe pattern noise as a random distribution model with a mean of zero and a small standard deviation.…”

“…In addition, they applied the structure called local-global combination to recover the detail information of image. On the one hand, in [35], Guan et al introduced a mixed convolutional layer consisting of dilated convolution, sub-pixel convolution, and standard convolution to extract the multi-grained features in various scales.…”