Video denoising using separable 4D nonlocal spatiotemporal transforms

Abstract

We propose a powerful video denoising algorithm that exploits temporal and spatial redundancy characterizing natural video sequences. The algorithm implements the paradigm of nonlocal grouping and collaborative filtering, where a higher-dimensional transform-domain representation is leveraged to enforce sparsity and thus regularize the data. The proposed algorithm exploits the mutual similarity between 3-D spatiotemporal volumes constructed by tracking blocks along trajectories defined by the motion vectors. Mutually similar volumes are grouped together by stacking them along an additional fourth dimension, thus producing a 4-D structure, termed group, where different types of data correlation exist along the different dimensions: local correlation along the two dimensions of the blocks, temporal correlation along the motion trajectories, and nonlocal spatial correlation (i.e. self-similarity) along the fourth dimension. Collaborative filtering is realized by transforming each group through a decorrelating 4-D separable transform and then by shrinkage and inverse transformation. In this way, collaborative filtering provides estimates for each volume stacked in the group, which are then returned and adaptively aggregated to their original position in the video. Experimental results demonstrate the effectiveness of the proposed procedure which outperforms the state of the art.

Video Denoising with Optical Flow Estimation

“…In this section we compare the proposed approach with state-of-the-art algorithms VBM3D [6] and VBM4D [14]. The Matlab implementations of VBM3D and VBM4D were obtained from the author's web site and the default parameters were used in the tests.…”

“…Arias et al extended NL-Bayes [12] to video [1,2]. The BM3D extension, VBM4D [14], exploits the mutual similarity between 3-D spatio-temporal volumes constructed by tracking blocks along trajectories defined by the motion vectors. Methods based on sparse decompositions are extended to image sequences [15,17,20,13], as well as approaches based on low rank approximation [9,10].…”

Video Denoising via Empirical Bayesian Estimation of Space-Time Patches

“…In [38,39] the authors introduced V-BM4D, an extension of BM3D to video by collaborative filtering of similar motion compensated 3D patches. Trajectories are computed with a block matching strategy based on the sum of squared differences (SSD) and a temporal regularization term, which favours trajectories with small velocity and low acceleration.…”

Spatiotemporal Video Denoising Based on Adaptive Thresholding and Clustering

“…Most algorithms in this field have been proposed for signal processing especially for video denoising [6][7][8][9][10][11].…”

“…As a result, the significant amount of the true signals in the less noisy areas will be removed, which will lead to deterioration of the visual quality of the output video. And then the blockmatching in the VBM3D occasionally searches out of the region that contains the reference block, which will result in poor matching in the areas that heavily contaminated by noise and this would lead to blurred edges [9].…”