Inverse texture synthesis

Abstract

original texture 890 2 × 11 original control map 890 2 × 1 inverse synthesis 128 2 × 11 128 2 × 1 texture compaction forward synthesis target control from original 14776 sec from compaction 1719 sec Figure 1: Inverse texture synthesis. Given a large globally-varying texture with an auxiliary control map (patina sequence from [Lu et al. 2007] in this case),our algorithm automatically computes a small texture compaction that best summarizes the original, including both texture and control. This small texture compaction can be used to reconstruct the original texture from its original control map, or to re-synthesize a new texture under a user-supplied control map. Due to the reduced data size, re-synthesis from our compaction is much faster than from the original without compromising image quality (right two images).In this example we use [Kwatra et al. 2005] for forward synthesis, but other algorithms can also be used since our compactions are just ordinary images. AbstractThe quality and speed of most texture synthesis algorithms depend on a 2D input sample that is small and contains enough texture variations. However, little research exists on how to acquire such a sample. For homogeneous patterns this can be achieved via manual cropping, but no adequate solution exists for inhomogeneous or globally varying textures, i.e. patterns that are local but not stationary, such as rusting over an iron statue with appearance conditioned on varying moisture levels.We present inverse texture synthesis to address this issue. Our inverse synthesis runs in the opposite direction with respect to traditional forward synthesis: given a large globally varying texture, our algorithm automatically produces a small texture compaction that best summarizes the original. This small compaction can be used to reconstruct the original texture or to re-synthesize new textures under user-supplied controls. More important, our technique allows real-time synthesis of globally varying textures on a GPU, where the texture memory is usually too small for large textures. We propose an optimization framework for inverse texture synthesis, ensuring that each input region is properly encoded in the output compaction. Our optimization process also automatically computes orientation fields for anisotropic textures containing both low-and high-frequency regions, a situation difficult to handle via existing techniques.

Repeatable texture sampling with interchangeable patches

“…Unlike [21], the algorithm does not place a spatial dependency on sampled pixels, so their performance cannot be compared in practice. Any algorithm which places dependency on sampled pixels would rely on cache, making it slower for larger output textures, so that sampling the texture millions of pixels apart would give our method an advantage with predictable results.…”

“…Instead of performing the process once, patches can be placed iteratively over the output until desired quality is achieved [8,21]. However, sequential algorithms are not suitable for simultaneous synthesis of disjoint regions, because the space between them needs to be synthesised as well.…”

Fast multi-scale joint bilateral texture upsampling

“…More recently, Kopf et al [13] extended global texture optimization method [10,12] to the task of solid texture synthesis; in addition, Kopf et al [13] integrated histogram matching the texture optimization, which improved the convergence of the synthesis process and partially addressed the issue that the optimization process could get stuck in a local minimum. More recently, differently from the traditional texture synthesis, Wei et al [18] presented an inverse texture synthesis method, which used an optimization framework to produce a small texture compaction that best summarized original large globally varying texture.…”

“…A great deal of existing works synthesize texture using either parametric [1,2], nonparametric [3][4][5], or patch-based [6][7][8][9] approaches. Among these methods, optimization-based methods [10,12,13], which belong to a kind of patch-based methods, have been proved very successful in terms of the quality of the synthesized results, and have been efficiently applied in image and video synthesis [10,12], solid texture synthesis [13], inverse texture synthesis [18], and geometry-surface texture synthesis [11]. However, the synthesis procedure of these optimization-based methods is done by minimizing a global texture energy function, which is a time and memory consuming process, and it limits the wider applications of optimization-based methods.…”

Mixed-Resolution Patch-Matching

“…The local or sparse matches may only provide approximate solutions for the given performance requirements, with the optimal solution requiring dense or global matching. Interactive applications such as image editing/re-targeting [4,6,7] need fast matching to maintain user interest. Sparse matching is used by them as dense matching is slow.…”