Stylizing video by example

Jamriška, Ondřej; Sochorová, Šárka; Texler, Ondřej; Lukáč, Michal; Fišer, Jakub; Lu, Jingwan; Shechtman, Eli; Sýkora, Daniel

doi:10.1145/3306346.3323006

Cited by 59 publications

(47 citation statements)

References 54 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…See Figures 9 and 10 and our supplementary video for results in this scenario. As in the video stylization case when compared to other techniques [GEB16, KSS19, JST*19, TFK*20] our approach better preserves the style exemplar (c.f. Fig.…”

Section: Resultsmentioning

confidence: 92%

STALP: Style Transfer with Auxiliary Limited Pairing

Futschik

Kučera

Lukáč

et al. 2021

Computer Graphics Forum

View full text Add to dashboard Cite

source style source frame Jamriška et al. target frame our approach Texler et al. Figure 1: An example of style transfer with auxiliary pairing-an artist prepares a stylized version (source style) of a selected frame from input video (source frame). Then a network is trained to transfer artist's style to remaining video frames (target frame). During the training phase a subset of input video frames as well as the source frame and its stylized counterpart are taken into account. Once the network is trained, the entire sequence can be stylized in real-time (our approach). In contrast to current state-of-the-art in example-based video stylzation (Jamriška et al. [JST * 19] and Texler et al. [TFK * 20]) our approach better preserves important visual characteristics of the style exemplar even though the scene structure changed considerably (head rotation). Input video frames and source style c Zuzana Studená, used with permission.

show abstract

Section: Resultsmentioning

confidence: 92%

STALP: Style Transfer with Auxiliary Limited Pairing

Futschik

Kučera

Lukáč

et al. 2021

Computer Graphics Forum

View full text Add to dashboard Cite

show abstract

“…This problem can be formulated within the context of Image Analogies [3], in which the goal is to stylize a target unstylized image B, given a pair of images A (un-stylized) and A' (stylized). The most common approach to tackle this problem has been via patch-based texture synthesis [7], [8], [9]. Nevertheless, recent approaches have leveraged the capabilities of deep latent spaces within convolutional neural networks to disentangle style from content [10].…”

Section: Related Workmentioning

confidence: 99%

Neural Photometry-guided Visual Attribute Transfer

Rodriguez-Pardo,

Garces

2021

Preprint

View full text Add to dashboard Cite

We present a deep learning-based method for propagating spatially-varying visual material attributes (e.g. texture maps or image stylizations) to larger samples of the same or similar materials. For training, we leverage images of the material taken under multiple illuminations and a dedicated data augmentation policy, making the transfer robust to novel illumination conditions and affine deformations. Our model relies on a supervised image-to-image translation framework and is agnostic to the transferred domain; we showcase a semantic segmentation, a normal map, and a stylization. Following an image analogies approach, the method only requires the training data to contain the same visual structures as the input guidance. Our approach works at interactive rates, making it suitable for material edit applications. We thoroughly evaluate our learning methodology in a controlled setup providing quantitative measures of performance. Last, we demonstrate that training the model on a single material is enough to generalize to materials of the same type without the need for massive datasets.

show abstract

“…The depth loss is computed as the per pixel loss between the depth estimated from the input content image and the stylized image, using a pre-trained single-image depth perception network [10]. Following the success of style transfer for 2D images, applications to other imaging modalities have emerged, such as video [11,12], omnidirectional imaging [13], or stereo imaging [14,15]. The video style transfer approach of Ruder et al [11] builds on the optimization approach of Gatys et al, enforcing the temporal consistency by initialising a stylized frame from a warped stylized previous frame and introducing a temporal con-sistency loss penalizing deviations between consecutive stylized frames.…”

Section: Introductionmentioning

confidence: 99%

Light Field Style Transfer with Local Angular Consistency

Egan

Alain

Smolić

2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

Style transfer involves combining the style of one image with the content of another to form a new image. Unlike traditional two-dimensional images which only capture the spatial intensity of light rays, four-dimensional light fields also capture the angular direction of the light rays. Thus, applying style transfer to a light field requires to not only render convincing style transfer for each view, but also to preserve its angular structure. In this paper, we present a novel optimization-based method for light field style transfer which iteratively propagates the style from the centre view towards the outer views while enforcing local angular consistency. For this purpose, a new initialisation method and angular loss function is proposed for the optimization process. In addition, since style transfer for light field is an emerging topic, no clear evaluation procedure is available. Thus, we investigate the use of a recently proposed metric designed to evaluate light field angular consistency, as well as a proposed variant.

show abstract

Stylizing video by example

Cited by 59 publications

References 54 publications

STALP: Style Transfer with Auxiliary Limited Pairing

STALP: Style Transfer with Auxiliary Limited Pairing

Neural Photometry-guided Visual Attribute Transfer

Light Field Style Transfer with Local Angular Consistency

Contact Info

Product

Resources

About