Exemplar-Based Inpainting from a Variational Point of View

Abstract

Among all methods for reconstructing missing regions in a digital image, the so-called exemplar-based algorithms are very efficient and often produce striking results. They are based on the simple idea-initially used for texture synthesis-that the unknown part of an image can be reconstructed by simply pasting samples extracted from the known part. Beyond heuristic considerations, there have been very few contributions in the literature to explain from a mathematical point of view the performances of these purely algorithmic and discrete methods. More precisely, a recent paper by Levina and Bickel [64] provides a theoretical explanation of their ability to recover very well the texture, but nothing equivalent has been done so far for the recovery of geometry. Our purpose in this paper is twofold: 1. To propose well-posed variational models in the continuous domain that can be naturally associated to exemplar-based algorithms; 2. To investigate their ability to reconstruct either local or long-range geometric features like edges. In particular, we propose several optimization models in R N , we discuss their relation with the original algorithms, and show the existence of minimizers in the framework of functions of bounded variation. Focusing on a simple 2D situation, we provide experimental evidences that basic exemplar-based algorithms are able to reconstruct a local geometric information whereas the minimization of the proposed variational models allows a global reconstruction of geometry and in particular of smooth edges. The derivation of globally minimizing algorithms associated to these models is still an open problem. Yet the results presented in this paper are a first step towards new inpainting algorithms with an improved quality of geometry reconstruction and no loss of quality for texture reconstruction.

Geometrically Guided Exemplar-Based Inpainting

“…However, we may wonder whether a theoretical link can be found between the replication of patches and the geometry. This is the purpose of the next proposition (see also [3] for a more general variational formulation), where we investigate a very simplified model of replication. More precisely, we prove that the difference between a smooth set and the concatenation of "straight" patches (i.e.…”

Affine Invariant Self-similarity for Exemplar-based Inpainting

“…The exemplar-based approach to inpainting has been intensively studied (Demanet et al, 2003;Criminisi et al, 2004;Wexler et al, 2007;Kawai et al, 2009;Aujol et al, 2010;Arias et al, 2011). However, many such methods are based on the assumption that the information necessary to complete the image is available elsewhere and can be copied without any modification but a translation.…”

An Examplar-Based Approach for Texture Compaction Synthesis and Retrieval

“…Some researchers have modified the approach with the goal of making it work as well for near-homogeneous textures [6]. Moreover, some analytical results based on nonparametric resampling of random fields and a variational point of view are presented in [25] and [26], respectively. One problem with the approach is the selection of the correct patch size because using different-sized patches tends to generate different textures [15].…”