Spatial-Temporal Color Video Reconstruction From Noisy CFA Sequence

Abstract

Abstract-Single-sensor digital video cameras use a color filter array (CFA) to capture video and a color demosaicking (CDM) procedure to reproduce the full color sequence. The reproduced video frames suffer from the inevitable sensor noise introduced in the video acquisition process. This paper presents a spatial-temporal denoising and demosaicking scheme that works without explicit motion estimation. We first perform patch based denoising on the mosaic CFA video. For each CFA patch to be denoised, similar patches are selected within a local spatialtemporal neighborhood. The principal component analysis is performed on the selected patches to remove noise. We then apply an initial single-frame CDM to the denoised CFA data, and subsequently post-process the demosaicked frames by exploiting the spatial-temporal redundancy to reduce the color artifacts. The experimental results on simulated and real noisy CFA sequences demonstrate that the proposed spatial-temporal CFA video denoising and demosaicking scheme can significantly reduce the noise-caused color artifacts and effectively preserve the image edge structures.Index Terms-Color demosaicking (CDM), color filter array (CFA), denoising, video enhancement.

Denoising Algorithm for CFA Image Sensors Considering Inter-Channel Correlation

“…Because demosaicing process before denoising changes the statistical characteristics of the noise and spreads spatially. The method described in [32] has severe motion artifacts around the edges and details in the region of motion, as shown in Figure 11e. Because method [32] did not consider the inter-channel correlation in video denoising process using motion information.…”

“…The method described in [32] has severe motion artifacts around the edges and details in the region of motion, as shown in Figure 11e. Because method [32] did not consider the inter-channel correlation in video denoising process using motion information. The high-frequency regions, which are difficult to denoise, including small details and textures, are preserved significantly better and the color artifacts are reduced in the proposed method, as shown in Figure 10f.…”

“…The methods discussed in [31] demonstrate ringing artifacts around the edges and over-smoothed results in the detail regions, as shown in Figure 12c. Although the denoised results are clear and definite in [9] and [32], over-shooting and color artifacts appear around the strong edges, as shown in Figure 12d,e. In the proposed method in Figure 12f, the denoised results are more explicit and better than those in the other methods.…”

“…To address these disadvantages, some denoising methods have been directly used with the CFA [30,31,32], as shown in Figure 1c. In the proposed approach, a denoising algorithm is also performed in the CFA pattern before the demosaicing process, as shown in Figure 1c.…”

Efficient Background Modeling Based on Sparse Representation and Outlier Iterative Removal

“…Another type of modeling techniques is to set up the model at the region level. Some works have focused on the local region, and different local features have been proposed [11], [24], [25], [26]. In addition, global-region based algorithms have also been proposed.…”