Variational Framework for Non-Local Inpainting

Fedorov, Vadim; Facciolo, Gabriele; Arias, Pablo

doi:10.5201/ipol.2015.136

Cited by 29 publications

(32 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We minimize a patch-based functional taken from the video inpainting [19,20] and texture synthesis [13] literature. Although the core of our algorithm is similar to that of Arias et al, [9] there are several important differences, which we list here. Firstly, we identify an important problem concerning the correct inpainting of textures with patch-based methods, and modify the patch distance to address this problem, in a similar manner to that of Liu and Caselles [14] in image inpainting and Newson et al [17] in video inpainting.…”

Section: Introductionmentioning

confidence: 75%

See 1 more Smart Citation

Non-Local Patch-Based Image Inpainting

Newson¹,

Almansa²,

Gousseau³

et al. 2017

Image Processing on Line

View full text Add to dashboard Cite

Image inpainting is the process of filling in missing regions in an image in a plausible way. In this contribution, we propose and describe an implementation of a patch-based image inpainting algorithm. The method is actually a two-dimensional version of our video inpainting algorithm The functional specifies that a good solution to the inpainting problem should be an image where each patch is very similar to its nearest neighbor in the unoccluded area. Iterations are performed in a multi-scale framework which yields globally coherent results. In this manner two of the major goals of image inpainting, the correct reconstruction of textures and structures, are addressed. We address a series of important practical issues which arise when using such an approach. In particular, we reduce execution times by using the PatchMatch [C. Barnes, PatchMatch: a randomized correspondence algorithm for structural image editing, ACM Transactions on Graphics, (2009)] algorithm for nearest neighbor searches, and we propose a modified patch distance which improves the comparison of textured patches. We address the crucial issue of initialization and the choice of the number of pyramid levels, two points which are rarely discussed in such approaches. We provide several examples which illustrate the advantages of our algorithm, and compare our results with those of state-of-the-art methods. Source CodeThe reviewed source code and documentation for this algorithm are available from the web page of this article 1 .

show abstract

Section: Introductionmentioning

confidence: 75%

“…This was first done using discrete minimization [12,18], and then in a continuous context by Arias et al [1,9]. One important element of such methods is that they use a multi-scale approach to converge to better inpainting solutions.…”

Section: Introductionmentioning

confidence: 99%

Non-Local Patch-Based Image Inpainting

Newson¹,

Almansa²,

Gousseau³

et al. 2017

Image Processing on Line

View full text Add to dashboard Cite

show abstract

“…• We propose several improvements to the architecture based on an improved WGAN such as the introduction of the residual learning framework in both the generator and discriminator, the removal of the fully connected layers on top of convolutional features and the replacement of the widely used batch normalization by a layer normalization. These improvements ease the training of the (Fedorov et al, 2015). (c) Results with the local method (Getreuer, 2012).…”

Section: Introductionmentioning

confidence: 99%

Semantic Image Inpainting through Improved Wasserstein Generative Adversarial Networks

Vitoria

Sintes

Ballester

2019

Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

View full text Add to dashboard Cite

Image inpainting is the task of filling-in missing regions of a damaged or incomplete image. In this work we tackle this problem not only by using the available visual data but also by incorporating image semantics through the use of generative models. Our contribution is twofold: First, we learn a data latent space by training an improved version of the Wasserstein generative adversarial network, for which we incorporate a new generator and discriminator architecture. Second, the learned semantic information is combined with a new optimization loss for inpainting whose minimization infers the missing content conditioned by the available data. It takes into account powerful contextual and perceptual content inherent in the image itself. The benefits include the ability to recover large regions by accumulating semantic information even it is not fully present in the damaged image. Experiments show that the presented method obtains qualitative and quantitative top-tier results in different experimental situations and also achieves accurate photo-realism comparable to state-of-the-art works.

show abstract

“…However, if the size of the unknown regions separating thin objects are too large, then even the fourth-order PDE-based model [40] will leave the thin structures broken (see Figure 1, 'TV-H −1 ' inpainting). Moreover, we see on Figure 1, 'NLinpaint' image, that even non-local methods that do not construct v with enough care [21] have difficulty connecting lines, when there are many large missing areas. One motivation for considering RNLTV is that, because it smoothly inpaints v in the missing regions and since large weights values are concentrated and aligned along the direction of the thin structures, RNLTV has the ability to connect the broken lines.…”

Section: Insight On the Modelmentioning

confidence: 99%

“…-inpaintn: the fully automated MATLAB function inpaintn [22,43] minimizes an energy functional involving a regularization by a squared finite difference and is solved with a Discrete Cosine Transform; -TV inpainting; -TV-H −1 inpainting [9,40]; -NLTVG [37]; -NLinpaint [2,21]: NLinpaint implemented three exemplar-based inpainting schemes, namely patch NL-means, patch NL-medians and patch NLPoisson, under the variational framework suggested by [2]. We provide, in the following experiments, the best result of the three methods for patch sizes between 5 and 15, using the default parameter of the 'Image Processing On Line' [2].…”

Section: Inpaintingmentioning

confidence: 99%

Regularized Non-local Total Variation and Application in Image Restoration

Malgouyres

Zeng

2017

J Math Imaging Vis

View full text Add to dashboard Cite

In the usual non-local variational models, such as the non-local total variations (NLTV), the image is regularized by minimizing an energy term that penalizes gray-levels discrepancy between some specified pairs of pixels, a weight value is computed between these two pixels to penalize their dissimilarity. In this paper, we impose some regularity to those weight values. More precisely, we minimize a function involving a regularization term, analogous to an H 1 term, on weights. Doing so, the finite differences defining the image regularity depend on their environment. When the weights are difficult to define, they can be restored by the proposed stable regularization scheme.We provide all the details necessary for the implementation of a PALM algorithm with proved convergence. We illustrate the ability of the model to restore relevant unknown edges from the neighboring edges on an image inpainting problem. We also argue on inpainting, zooming and denoising problems that the model better recovers thin structures.

show abstract

Variational Framework for Non-Local Inpainting

Cited by 29 publications

References 40 publications

Non-Local Patch-Based Image Inpainting

Non-Local Patch-Based Image Inpainting

Semantic Image Inpainting through Improved Wasserstein Generative Adversarial Networks

Regularized Non-local Total Variation and Application in Image Restoration

Contact Info

Product

Resources

About