Denoising image sequences does not require motion estimation

Abstract

State of the art movie restoration methods either estimate motion and filter out the trajectories, or compensate the motion by an optical flow estimate and then filter out the compensated movie. Now, the motion estimation problem is ill-posed. This fact is known as the aperture problem: trajectories are ambiguous since they could coincide with any promenade in the space-time isophote surface. In this paper, we try to show that, for denoising, the aperture problem can be taken advantage of. Indeed, by the aperture problem, many pixels in the neighboring frames are similar to the current pixel one wishes to denoise. Thus, denoising by an averaging process can use many more pixels than just the ones on a single trajectory. This observation leads to use for movies a recently introduced image denoising method, the NL-means algorithm. This static 3D algorithm outperforms motion compensated algorithms, as it does not lose movie details. It involves the whole movie isophote and not just a trajectory.

Video denoising using higher order optimal space-time adaptation

“…It is important to point out that the application of the OSA algorithm in 3-D for the purpose of video denoising does not require explicit motion estimation, as also indicated in [5,6]. The same comment applies to the extensions of OSA which we propose in this paper.…”

Space-Time Adaptation for Patch-Based Image Sequence Restoration

“…The reported results demonstrate that our method can cope with the presence of motion while preserving temporal discontinuities and reaches a P SN R of 23.59 using 7 × 7 patches and 6 iterations. By applying our own implementation of the non-local mean algorithm [5] on the "Flower garden" sequence using a 7 × 7 patch in a 21 × 21 × 3 neighborhood and choosing a bandwidth equal to h = 12 × τ 2 , we get a P SN R equal to 21.14dB only.…”

“…However, dense motion estimation is known to be a difficult task in noisy contexts [5]. Then, we propose to robustly estimate a global parametric motion model only, which is able to capture the dominant image motion due to the camera movement.…”

“…Recently an extension of the non-local mean filter [6] also related to the universal denoising (DUDE) algorithm [21], has been proposed to process image sequences and relies on the principle that the image sequence contains repeated patterns [5] by averaging. The detection of repeated patterns can be used to reduce the noise in images.…”

Video Denoising based on Sparse Transformation and Low Rank Matrix Completion

“…[1] These methods typically utilize both the sparsity and the statistical properties of a multiresolution representation as well as the inherent correlations between frames in temporal dimension. A recent denoising strategy, the non-local spatial estimation [2], has also been adapted to video denoising [3]. In this approach, similarity between 2D patches is used to determine the weights in a weighted averaging between the central pixels of these patches.…”