Space-Time Adaptation for Patch-Based Image Sequence Restoration

Abstract

We present a novel space-time patch-based method for image sequence restoration. We propose an adaptive statistical estimation framework based on the local analysis of the bias-variance trade-off. At each pixel, the space-time neighborhood is adapted to improve the performance of the proposed patch-based estimator. The proposed method is unsupervised and requires no motion estimation. Nevertheless, it can also be combined with motion estimation to cope with very large displacements due to camera motion. Experiments show that this method is able to drastically improve the quality of highly corrupted image sequences. Quantitative evaluations on standard artificially noise-corrupted image sequences demonstrate that our method outperforms other recent competitive methods. We also report convincing results on real noisy image sequences.Key-words: Image sequence restoration, denoising, non parametric estimation, non linear filtering, bias-variance trade-offThe authors would thank the project ACI IMPBio for its support. Nous présentons une nouvelle méthode pour la restauration de séquences d'images exploitant des "patches". Nous proposons une approche d'estimation adaptative s'appuyant sur l'analyse locale du compromis biais-variance. Pour chaque pixel, le voisinage spatio-temporel considéré est adapté afin d'améliorer les performances de l'estimateur proposé. Cette méthode est non-supervisée et ne requiert pas d'estimation du mouvement. Cependant, elle peut aussiêtre combinée avec une estimation du mouvement afin de faire face, par exemple, aux grands déplacements dus aux mouvements de la caméra. Les expériences montrent que cette méthode est capable d'améliorer significativement la qualité de séquences d'image fortement bruitées. Desévaluations quantitatives sur des séquences artificiellement bruitées selon un protocole standard démontrent que notre méthode surpasse d'autres méthodes récentes. Nous apportonségalement des résultats convaincants sur des séquences d'images réelles.

Video denoising using higher order optimal space-time adaptation

“…This method, in addition to extending the Non-Local Means(NLM) method of [2], employs an iteratively growing window scheme, and a local estimate of the mean square error to very effectively remove noise from images. By adopting an iteratively growing space-time window, the method was recently extended to 3-D for video denoising in [3]. In the present paper, we demonstrate a simple, but effective improvement on the OSA method in both 2-and 3-D. We demonstrate that the OSA implicitly relies on a locally constant model of the underlying signal.…”

“…The peak signal-to-noise ratio defined as PSNR = 10log 10 (255 2 /MSE) was used to measure the quality of the denoised resultẑ versus the original video z. For a fair evaluation, we set the parameters in the same way as [3]. In Tables 1 and 2, we present the PSNR(dB) comparisons of the proposed higher order OSA and the standard OSA for a few sequences in cases Algorithm 1 OSA algorithms in 3D Let { J : Patch size, α : Percentile, : Parameter for stopping rule, L Δ : Maximum Iteration } be the parameters.…”

“…An iterative version of this, as elaborated below, with the adaptive window size, is the basis of the superior OSA formulation of Boulanger et al in [1,3]. It is important to point out that the application of the OSA algorithm in 3-D for the purpose of video denoising does not require explicit motion estimation, as also indicated in [5,6].…”

“…The so-called optimal spatial adaptation (OSA) method proposed by Kervrann et al [1] is one such example, which improves upon the earlier NLM by iteratively growing the local analysis window size along with a pixel-wise local stopping rule based on a local estimate of the mean square error [1]. OSA was further extended to 3-D [3] for video denoising by applying iteratively growing space-time window, which at the time of its publication, apparently achieved state-of-art…”

Video Denoising with Optical Flow Estimation

“…Besides the obvious use of these techniques on a frame by frame basis, local average methods, such as the bilateral filter [19], or patch based methods such as NL-means [4] or BM3D [7] and NLBayes [12] can be easily adapted to video just by extending the neighboring area to the adjacent frames. Kervrann and Boulanger [3] extended NL-means to video by growing adaptively the spatio-temporal neighborhood. Arias et al extended NL-Bayes [12] to video [1,2].…”