Joint Removal of Random and Fixed-Pattern Noise Through Spatiotemporal Video Filtering

Abstract

We propose a framework for the denoising of videos jointly corrupted by spatially correlated (i.e., nonwhite) random noise and spatially correlated fixed-pattern noise. Our approach is based on motion-compensated 3D spatiotemporal volumes, i.e., a sequence of 2D square patches extracted along the motion trajectories of the noisy video. First, the spatial and temporal correlations within each volume are leveraged to sparsify the data in 3D spatiotemporal transform domain, and then the coefficients of the 3D volume spectrum are shrunk using an adaptive 3D threshold array. Such array depends on the particular motion trajectory of the volume, the individual power spectral densities of the random and fixed-pattern noise, and also the noise variances which are adaptively estimated in transform domain. Experimental results on both synthetically corrupted data and real infrared videos demonstrate a superior suppression of the random and fixed-pattern noise from both an objective and a subjective point of view.

Fixed-pattern noise suppression in low-sensing environment of Time-of-Flight devices

“…Every light-capturing device with digital sensor (CCD or CMOS) is influenced by spatially-related offset mask called Fixed-pattern noise (FPN) [11,12]. In the case of ToF sensors, FPN appears as stripes of brighter or darker intensity across the image pixels (c.f.…”

“…Those are defined as different operation limits of sensing elements to discharge data of low or oversaturated value charges [5,11,12,13]. Consequently, the noise behavior will be observed in the direction of discharging flow process and it will appear as periodic "column-" or "row-wise" stripes.…”

“…In our earlier works, we have observed that the performance of the denoising methods is quite much influenced by the presence of noise component, characterized as fixed-pattern noise (FPN), similarly to phenomena in other 2D imaging devices [5,11,12,13]. Such noise component has to be suppressed by proper linear filtering before a more dedicated denoising algorithm can be efficiently applied.…”

Denoising Algorithm for CFA Image Sensors Considering Inter-Channel Correlation

“…To restore degraded image sequence data, denoising methods extended to the temporal domain have also been presented in recent years [4,5,6,7,8,9,10,11]. In the denoising process for monochromatic data, explicit motion representation refers to the detection of the true motion trajectories of a physical point via motion estimation and compensation.…”

“…Many temporal filters, such as the locally adaptive linear minimum-mean-squared-error spatio-temporal filter (3D-LLMMSE) [4] and the multi-hypothesis motion-compensated filter (MHMCF) [5,6], adopt this approach. The motion-compensated denoising algorithm based on 3D spatiotemporal volume for random and fixed-pattern noises [10] and using regularized optical flow [11] were presented recently. This motion-compensated filtering approach can be effective in preventing motion blurring during video denoising processes.…”