StyleBlit: Fast Example‐Based Stylization with Local Guidance

Abstract

We present StyleBlit—an efficient example‐based style transfer algorithm that can deliver high‐quality stylized renderings in real‐time on a single‐core CPU. Our technique is especially suitable for style transfer applications that use local guidance ‐ descriptive guiding channels containing large spatial variations. Local guidance encourages transfer of content from the source exemplar to the target image in a semantically meaningful way. Typical local guidance includes, e.g., normal values, texture coordinates or a displacement field. Contrary to previous style transfer techniques, our approach does not involve any computationally expensive optimization. We demonstrate that when local guidance is used, optimization‐based techniques converge to solutions that can be well approximated by simple pixel‐level operations. Inspired by this observation, we designed an algorithm that produces results visually similar to, if not better than, the state‐of‐the‐art, and is several orders of magnitude faster. Our approach is suitable for scenarios with low computational budget such as games and mobile applications.

SNeRF: Stylized Neural Implicit Representations for 3D Scenes

“…However, they require the content and style image to be semantically similar. Stylize-by-example approaches also adopt a patch-based approach using high-quality examples as guidance [Fišer et al 2016;Sýkora et al 2019b;]. Despite impressive results, these methods require dedicated guiding examples provided by professional artists.…”

“…We propose to combine NeRF and image-based neural style transfer to perform 3D scene stylization. While NeRF provides a strong inductive bias to maintain multi-view consistency, neural style transfer enables a flexible stylization approach that does not require dedicated example inputs from professional artists [Fišer et al 2016;Sýkora et al 2019a;]. Additionally, we address the memory limitations of NeRF by splitting the 3D scene style transfer process into two steps that run alternatingly.…”

Controlling Material Appearance by Examples

“…In recent years, neural style transfer [GEB16] and neural texture synthesis [GEB15] have been used in a variety of contexts (e.g. sketching [TTK∗21,SJT∗19], video [JvST∗19, TFK∗20], painting style [TFF∗20]). These methods are based on the matching of the statistics extracted by a pre‐trained neural network between output and target images.…”

SelectionConv: Convolutional Neural Networks for Non-rectilinear Image Data

“…Some have explored generating new texture from a smaller example texture through classical texture synthesis methods [3,8,40] and neural approaches [12,13,31,33,49]. Textures have also been manipulated through lighting-based style transfer [11,35] rather than through a purely image-based approach like the one we present in Sec. 6.4.…”