Interactive video stylization using few-shot patch-based training

Abstract

In this paper, we present a learning-based method to the keyframe-based video stylization that allows an artist to propagate the style from a few selected keyframes to the rest of the sequence. Its key advantage is that the resulting stylization is semantically meaningful, i.e., specific parts of moving objects are stylized according to the artist's intention. In contrast to previous style transfer techniques, our approach does not require any lengthy pre-training process nor a large training dataset. We demonstrate how to train an appearance translation network from scratch using only a few stylized exemplars while implicitly preserving temporal consistency. This leads to a video stylization framework that supports real-time inference, parallel processing, and random access to an arbitrary output frame. It can also merge the content from multiple keyframes without the need to perform an explicit blending operation. We demonstrate its practical utility in various interactive scenarios, where the user paints over a selected keyframe and sees her style transferred to an existing recorded sequence or a live video stream.

STALP: Style Transfer with Auxiliary Limited Pairing

“…The final image translation model can be used for real‐time stylization of a new video conference call that contains the same person and have similar lightihg conditions (target frames). Note that in contrast to the method of Texler et al [TFK*20] our approach better preserves style details and keeps the stylization more consistent in time (see also our supplementary video). Video frames and source style © Zuzana Studená, used with permission.…”

“…As visible from the results and comparisons, our approach can better preserve style details during a longer time frame even if the scene structure changes considerably with respect to X. Also, note how the resulting stylized sequence has better temporal stability implicitly without performing any additional treatment, which contrasts with previous techniques [JST*19, TFK*20] that need to handle temporal consistency explicitly.…”

“…See Figures 9 and 10 and our supplementary video for results in this scenario. As in the video stylization case when compared to other techniques [GEB16, KSS19, JST*19, TFK*20] our approach better preserves the style exemplar (c.f. Fig.…”

Application of Feature Extraction Algorithm in the Construction of Interactive English Chinese Translation Mode

“…And the corresponding calling interface is designed for the classification system, which ensures the usability of the classification module. At the same time, a Classified Thesaurus is designed to preserve the unique feature words of each category and to prioritize the categories of documents to be classified [10].…”

Interactive Layout Transfer

“…This has found applications in colour adaptation [33] and in transferring the overall style of portrait images [36]. Convolutional neural networks (CNNs) were presented to transfer a painter's artistic style to natural images [11], which has inspired to further research in neural style transfer to improve quality and expand the scope of applications [22,23,25,38]. By adding a structural component to address UI details and usability, CNNs have been recently used to restyle GUIs with artistic images [8].…”