Affine Invariant Self-similarity for Exemplar-based Inpainting

Fedorov, Vadim; Arias, Pablo; Facciolo, Gabriele; Ballester, Coloma

doi:10.5220/0005728100480058

Cited by 12 publications

(5 citation statements)

References 31 publications

(75 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus, some researchers have exploited, also using a variational approach, the efficiency of PatchMatch [Barnes et al, 2009] in computing a probabilistic approximation of correspondence maps between patches to average the contribution of multiple source patches during the synthesis step. For example, [Arias et al, 2011], [Newson et al, 2014] use it in a non-local mean fashion [Wexler et al, 2004], to inpaint rescaled versions of the original image with results propagated from the coarser to the finer scale, [Cao et al, 2011] to guide the inpainting with geometric-sketches, [Sun et al, 2005] to guide structures or [Mansfield et al, 2011], [Eller and Fornasier, 2016], [Fedorov et al, 2016] to account for geometric transformations of patches. However, these mathematical and numerical advances may result computationally expensive while suffering from the single-imaging source, and dependence on the initialization quality and the selection of associated parameters (e.g.…”

Section: Model-based Inpaintingmentioning

confidence: 99%

An Analysis of Generative Methods for Multiple Image Inpainting

Ballester¹,

Bugeau²,

Hurault³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Image inpainting refers to the restoration of an image with missing regions in a way that is not detectable by the observer. The inpainting regions can be of any size and shape. This is an ill-posed inverse problem that does not have a unique solution. In this work, we focus on learning-based image completion methods for multiple and diverse inpainting which goal is to provide a set of distinct solutions for a given damaged image. These methods capitalize on the probabilistic nature of certain generative models to sample various solutions that coherently restore the missing content. Along the chapter, we will analyze the underlying theory and analyze the recent proposals for multiple inpainting. To investigate the pros and cons of each method, we present quantitative and qualitative comparisons, on common datasets, regarding both the quality and the diversity of the set of inpainted solutions. Our analysis allows us to identify the most successful generative strategies in both inpainting quality and inpainting diversity. This task is closely related to the learning of an accurate probability distribution of images. Depending on the dataset in use, the challenges that entail the training of such a model will be discussed through the analysis.

show abstract

Section: Model-based Inpaintingmentioning

confidence: 99%

An Analysis of Generative Methods for Multiple Image Inpainting

Ballester¹,

Bugeau²,

Hurault³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…These methods show good performance in propagating smooth level lines or gradients, but fail in the presence of texture or for large missing regions. Non-local methods (also called exemplaror patch-based) exploit the self-similarity prior by directly sampling the desired texture to perform the synthesis (Efros and Leung, 1999;Demanet et al, 2003;Criminisi et al, 2004;Wang, 2008;Kawai et al, 2009;Aujol et al, 2010;Arias et al, 2011;Huang et al, 2014;Fedorov et al, 2016). They provide impressive results in inpainting textures and repetitive structures even in the case of large holes.…”

Section: Related Workmentioning

confidence: 99%

Semantic Image Inpainting through Improved Wasserstein Generative Adversarial Networks

Vitoria

Sintes

Ballester

2019

Proceedings of the 14th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Application

Self Cite

View full text Add to dashboard Cite

Image inpainting is the task of filling-in missing regions of a damaged or incomplete image. In this work we tackle this problem not only by using the available visual data but also by incorporating image semantics through the use of generative models. Our contribution is twofold: First, we learn a data latent space by training an improved version of the Wasserstein generative adversarial network, for which we incorporate a new generator and discriminator architecture. Second, the learned semantic information is combined with a new optimization loss for inpainting whose minimization infers the missing content conditioned by the available data. It takes into account powerful contextual and perceptual content inherent in the image itself. The benefits include the ability to recover large regions by accumulating semantic information even it is not fully present in the damaged image. Experiments show that the presented method obtains qualitative and quantitative top-tier results in different experimental situations and also achieves accurate photo-realism comparable to state-of-the-art works.

show abstract

“…Non-local or patch-based approaches are used in most of the state-of-the-art methods for image denoising, restoration, super-resolution, inpainting and object recognition [6], [28], [33], [31], [14], [30], [17], [12]. Image denoising has gone along with the advances in patch-based techniques.…”

Section: Related Workmentioning

confidence: 99%

Affine Non-Local Means Image Denoising

Fedorov

Ballester

2017

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

This paper presents an extension of the Non-Local Means denoising method, that effectively exploits the affine invariant self-similarities present in the images of real scenes. Our method provides a better image denoising result by grounding on the fact that in many occasions similar patches exist in the image but have undergone a transformation. The proposal uses an affine invariant patch similarity measure that performs an appropriate patch comparison by automatically and intrinsically adapting the size and shape of the patches. As a result, more similar patches are found and appropriately used. We show that this image denoising method achieves top-tier performance in terms of PSNR, outperforming consistently the results of the regular Non-Local Means, and that it provides state-of-the-art qualitative results.

show abstract

Affine Invariant Self-similarity for Exemplar-based Inpainting

Cited by 12 publications

References 31 publications

An Analysis of Generative Methods for Multiple Image Inpainting

An Analysis of Generative Methods for Multiple Image Inpainting

Semantic Image Inpainting through Improved Wasserstein Generative Adversarial Networks

Affine Non-Local Means Image Denoising

Contact Info

Product

Resources

About