Optimal Patch Assignment for Statistically Constrained Texture Synthesis

Gutiérrez, J. A.; Rabin, Julien; Galerne, Bruno; Hurtut, Thomas

doi:10.1007/978-3-319-58771-4_14

Cited by 18 publications

(18 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Indeed, a counter-example is given by one non-trivial texture u in which by chance happens a constant patch with color c; then a constant image with color c would be considered as a perfect synthesis for this energy E NN but a complete failure for a human observer. Therefore, this experiment underlies the need of a statistical control in the synthesis process, as was already stated by [59] and [21]. The poor quality of the images generated by iterated NN may explain why the authors of [28] used several patch sizes at each scale and relied on a more sophisticated patch aggregation than a weighted average.…”

Section: Comparison With Iterated Nn Projectionsmentioning

confidence: 76%

“…Tartavel et al [60] extended variational texture synthesis by combining discrete OT distances computed on several (non-linear) filter responses. Finally, Gutierrez et al [21] proposed a texture synthesis method that enforces the patch distribution at multiple scales by applying a discrete OT plan. This method can be understood as an elaborate improvement of the texture optimization method [28] with a global statistical control.…”

Section: Global Statistical Controlmentioning

confidence: 99%

“…Once estimated, the OT maps are stored, and can be used for the synthesis of possibly very large images. Our method can be seen as a localized version of [21]. Indeed, in contrast to [21], our model leads to a much faster (and highly parallel) synthesis algorithm because the local patch transformations are computed once and for all, and are applied independently to all patches at each scale.…”

Section: Global Statistical Controlmentioning

confidence: 99%

See 2 more Smart Citations

A Texture Synthesis Model Based on Semi-Discrete Optimal Transport in Patch Space

Galerne

Leclaire²,

Rabin³

2018

SIAM J. Imaging Sci.

View full text Add to dashboard Cite

Exemplar-based texture synthesis consists in producing new synthetic images which have the same perceptual characteristics than a given texture sample while exhibiting sufficient innovation (to avoid verbatim copy). In this paper, we propose to address this problem with a model obtained as local transformations of Gaussian random fields. The local transformations operate on 3 × 3 patches and are designed to solve a semi-discrete optimal transport problem in order to reimpose the patch distribution of the exemplar texture. The semi-discrete optimal transport problem is solved with a stochastic gradient algorithm, whose convergence speed is evaluated on several practical transport cases. After studying the properties of such transformed Gaussian random fields, we propose a multiscale extension of the model which aims at preserving the patch distribution of the exemplar texture at multiple scales. Experiments demonstrate that this multiscale model is able to synthesize structured textures while keeping several mathematical guarantees, and with low requirements in synthesis time and memory storage. In particular, a single patch optimal transport map is shown to be better than iterated nearest neighbor assignments in terms of statistical guarantees. Besides, once the model is estimated, the resulting synthesis algorithm is fast and highly parallel since it amounts to perform weighted nearest neighbor patch assignments at each scale.

show abstract

Section: Comparison With Iterated Nn Projectionsmentioning

confidence: 76%

Section: Global Statistical Controlmentioning

confidence: 99%

Section: Global Statistical Controlmentioning

confidence: 99%

See 1 more Smart Citation

A Texture Synthesis Model Based on Semi-Discrete Optimal Transport in Patch Space

Galerne

Leclaire²,

Rabin³

2018

SIAM J. Imaging Sci.

View full text Add to dashboard Cite

show abstract

“…As reported in Gutierrez et al . [GRGH17], the (unwanted) optimal solution of methods based on patch optimization is the input image itself. For most of these methods though, the local copies are sufficiently randomized to deliver enough diversity.…”

Section: Resultsmentioning

confidence: 99%

“…Yet, observe that many methods in the literature for 2D texture synthesis generate images that are local copies of the input image [WL00, KEBK05], which strongly limits the diversity. As reported in Gutierrez et al [GRGH17], the (unwanted) optimal solution of methods based on patch optimization is the input image itself. For most of these methods though, the local copies are sufficiently randomized to deliver enough diversity.…”

Section: Diversitymentioning

confidence: 99%

On Demand Solid Texture Synthesis Using Deep 3D Networks

Gutiérrez

Rabin

Galerne

et al. 2019

Computer Graphics Forum

View full text Add to dashboard Cite

This paper describes a novel approach for on demand volumetric texture synthesis based on a deep learning framework that allows for the generation of high‐quality three‐dimensional (3D) data at interactive rates. Based on a few example images of textures, a generative network is trained to synthesize coherent portions of solid textures of arbitrary sizes that reproduce the visual characteristics of the examples along some directions. To cope with memory limitations and computation complexity that are inherent to both high resolution and 3D processing on the GPU, only 2D textures referred to as ‘slices’ are generated during the training stage. These synthetic textures are compared to exemplar images via a perceptual loss function based on a pre‐trained deep network. The proposed network is very light (less than 100k parameters), therefore it only requires sustainable training (i.e. few hours) and is capable of very fast generation (around a second for 2563 voxels) on a single GPU. Integrated with a spatially seeded pseudo‐random number generator (PRNG) the proposed generator network directly returns a color value given a set of 3D coordinates. The synthesized volumes have good visual results that are at least equivalent to the state‐of‐the‐art patch‐based approaches. They are naturally seamlessly tileable and can be fully generated in parallel.

show abstract

Learning from small data sets: Patch‐based regularizers in inverse problems for image reconstruction

Piening,

Altekrüger,

Hertrich

et al. 2024

GAMM-Mitteilungen

View full text Add to dashboard Cite

The solution of inverse problems is of fundamental interest in medical and astronomical imaging, geophysics as well as engineering and life sciences. Recent advances were made by using methods from machine learning, in particular deep neural networks. Most of these methods require a huge amount of data and computer capacity to train the networks, which often may not be available. Our paper addresses the issue of learning from small data sets by taking patches of very few images into account. We focus on the combination of model‐based and data‐driven methods by approximating just the image prior, also known as regularizer in the variational model. We review two methodically different approaches, namely optimizing the maximum log‐likelihood of the patch distribution, and penalizing Wasserstein‐like discrepancies of whole empirical patch distributions. From the point of view of Bayesian inverse problems, we show how we can achieve uncertainty quantification by approximating the posterior using Langevin Monte Carlo methods. We demonstrate the power of the methods in computed tomography, image super‐resolution, and inpainting. Indeed, the approach provides also high‐quality results in zero‐shot super‐resolution, where only a low‐resolution image is available. The article is accompanied by a GitHub repository containing implementations of all methods as well as data examples so that the reader can get their own insight into the performance.

show abstract

Optimal Patch Assignment for Statistically Constrained Texture Synthesis

Cited by 18 publications

References 33 publications

A Texture Synthesis Model Based on Semi-Discrete Optimal Transport in Patch Space

A Texture Synthesis Model Based on Semi-Discrete Optimal Transport in Patch Space

On Demand Solid Texture Synthesis Using Deep 3D Networks

Learning from small data sets: Patch‐based regularizers in inverse problems for image reconstruction

Contact Info

Product

Resources

About