A Variational Framework for Exemplar-Based Image Inpainting

Abstract

Non-local methods for image denoising and inpainting have gained considerable attention in recent years. This is in part due to their superior performance in textured images, a known weakness of purely local methods. Local methods on the other hand have demonstrated to be very appropriate for the recovering of geometric structures such as image edges. The synthesis of both types of methods is a trend in current research. Variational analysis in particular is an appropriate tool for a unified treatment of local and non-local methods. In this work we propose a general variational framework non-local image inpainting, from which important and representative previous inpainting schemes can be derived, in addition to leading to novel ones. We explicitly study some of these, relating them to previous work and showing results on synthetic and real images.

Image Zoom Completion

“…As the setting for IZC is an LR/HR image pair, one can consider training both LR and visible HR patches in order to learn an adapted dictionary. When the super-resolution factor r is important and/or the blurring matrix attenuates too much the high frequencies of the HR image, one can consider an iterative minimization process alternating between weight computation and nonlocal regularization as was done in [41] for image inpainting.…”

Semantic Image Inpainting through Improved Wasserstein Generative Adversarial Networks

“…These methods show good performance in propagating smooth level lines or gradients, but fail in the presence of texture or for large missing regions. Non-local methods (also called exemplaror patch-based) exploit the self-similarity prior by directly sampling the desired texture to perform the synthesis (Efros and Leung, 1999;Demanet et al, 2003;Criminisi et al, 2004;Wang, 2008;Kawai et al, 2009;Aujol et al, 2010;Arias et al, 2011;Huang et al, 2014;Fedorov et al, 2016). They provide impressive results in inpainting textures and repetitive structures even in the case of large holes.…”

Frame Interpolation With Occlusion Detection Using a Time Coherent Segmentation

“…To fill-in the remaining holes we use an inpainting strategy. Each pixel in a hole of I δ is filled in by an exemplar based interpolation as in (Arias et al, 2011) (see also (Criminisi et al, 2004)). To fill a pixel in frame t we search for patches in the previous and next frames.…”

“…Forward and backward motion fields are then used to interpolate the intermediate image taking into account the latter points. After motion-based interpolation, there may still be some holes which are filled using a suitable hole filling algorithm (Criminisi et al, 2004;Arias et al, 2011).…”