Painting style transfer for head portraits using convolutional neural networks

Selim, Ahmed; Elgharib, Mohamed; Doyle, Linda

doi:10.1145/2897824.2925968

Cited by 181 publications

(89 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As they do not impose spatial constraints, directly applying these existing algorithms to head portraits will deform facial structures, which is unacceptable for the human visual system. Selim et al [73] address this problem and extend [4] to head portrait painting transfer. They propose to use the notion of gain maps to constrain spatial configurations, which can preserve the facial structures while transferring the texture of the style image.…”

Section: Improvements and Extensionsmentioning

confidence: 99%

Neural Style Transfer: A Review

Jing

Yang

Feng

et al. 2020

IEEE Trans. Visual. Comput. Graphics

555

310

View full text Add to dashboard Cite

The seminal work of Gatys et al. demonstrated the power of Convolutional Neural Networks (CNNs) in creating artistic imagery by separating and recombining image content and style. This process of using CNNs to render a content image in different styles is referred to as Neural Style Transfer (NST). Since then, NST has become a trending topic both in academic literature and industrial applications. It is receiving increasing attention and a variety of approaches are proposed to either improve or extend the original NST algorithm. In this paper, we aim to provide a comprehensive overview of the current progress towards NST. We first propose a taxonomy of current algorithms in the field of NST. Then, we present several evaluation methods and compare different NST algorithms both qualitatively and quantitatively. The review concludes with a discussion of various applications of NST and open problems for future research. A list of papers discussed in this review, corresponding codes, pre-trained models and more comparison results are publicly available at: https://github.com/ycjing/Neural-Style-Transfer-Papers. IndexTerms-Neural style transfer (NST), convolutional neural network ! Neural Style Transfer Example-Based Techniques Colour Image Analogy Texture Model-Optimisation-Based Offline Neural Methods Multiple-Style-Per-Model Neural Methods Dumoulin'17 [53] Chen'17 [54] Li'17 [55] Zhang'17 [56] Luan'17 [84] Mechrez'17 [85]Photorealistic Liao'17 [88] Attribute Champandard'16 [65] Doodle Ruder'16 [74] Video Selim'16 [73] Portrait Castillo'17 [71] Instance Gatys'17 [60] Improvement Image Gatys'16 [10] Li'17 [42] Risser'17 [44] Li'17 [45] Li'16 [46] Image Champandard'16 [65] Chen'16 [68] Mechrez'18 [69]

show abstract

Section: Improvements and Extensionsmentioning

confidence: 99%

Neural Style Transfer: A Review

Jing

Yang

Feng

et al. 2020

IEEE Trans. Visual. Comput. Graphics

555

310

View full text Add to dashboard Cite

show abstract

“…Recently, convolutional neural network (CNN) [13] based style transfer methods have shown successful applications in transferring the style of a certain type of artistic painting, e.g, Vincent van Gogh's "The Starry Night", to a real world photograph, e.g., an image taken by iPhone. Since the seminal work of Gatys et al [7], it has attracted a lot of attentions from both academia [10,15,23,24,8,28,31,6,5,4] and industry [26,1,11,3]. Although the work of neural style transfer has shown promising progress on transferring artistic images with rich textures and colors, e.g., the oil paintings, we observe that it is less effective in transferring Chinese traditional painting.…”

Section: Introductionmentioning

confidence: 91%

Neural Abstract Style Transfer for Chinese Traditional Painting

Xiong

et al. 2019

Computer Vision – ACCV 2018

View full text Add to dashboard Cite

Chinese traditional painting is one of the most historical artworks in the world. It is very popular in Eastern and Southeast Asia due to being aesthetically appealing. Compared with western artistic painting, it is usually more visually abstract and textureless. Recently, neural network based style transfer methods have shown promising and appealing results which are mainly focused on western painting. It remains a challenging problem to preserve abstraction in neural style transfer. In this paper, we present a Neural Abstract Style Transfer method for Chinese traditional painting. It learns to preserve abstraction and other style jointly end-to-end via a novel MXDoG-guided filter (Modified version of the eXtended Difference-of-Gaussians) and three fully differentiable loss terms. To the best of our knowledge, there is little work study on neural style transfer of Chinese traditional painting. To promote research on this direction, we collect a new dataset with diverse photo-realistic images and Chinese traditional paintings 6 . In experiments, the proposed method shows more appealing stylized results in transferring the style of Chinese traditional painting than state-of-the-art neural style transfer methods.

show abstract

“…It uses semantic labels to prevent semantic inconsistency so that style transfer is carried out between same semantic regions. Style transfer networks are also specialized for the editing face images and portraits [Kemelmacher-Shlizerman 2016;Liao et al 2017;Selim et al 2016] with new objectives. Style transfer works limit the users to find an reference photo in which desired style effects exist for desired attributes.…”

Section: Image Editing With Deepmentioning

confidence: 99%

Manipulating Attributes of Natural Scenes via Hallucination

et al. 2019

View full text Add to dashboard Cite

Fig. 1. Given a natural image, the proposed approach can hallucinate different versions of the same scene in a wide range of conditions, e.g.night, sunset, winter, spring, rain, fog or even a combination of those. First, our method utilizes a single generator network to imagine the scene with respect to its semantic layout and the desired set of attributes. Then, it directly transfers the look from the hallucinated output to the input image, without a need to have access to a reference style image.In this study, we explore building a two-stage framework for enabling users to directly manipulate high-level attributes of a natural scene. The key to our approach is a deep generative network which can hallucinate images of a scene as if they were taken at a different season (e.g. during winter), weather condition (e.g. in a cloudy day) or time of the day (e.g. at sunset). Once the scene is hallucinated with the given attributes, the corresponding look is then transferred to the input image while preserving the semantic details intact, giving a photo-realistic manipulation result. As the proposed framework hallucinates what the scene will look like, it does not require any reference style image as commonly utilized in most of the appearance or style transfer approaches. Moreover, it allows to simultaneously manipulate a given scene according to a diverse set of transient attributes within a single model, eliminating the need of training multiple networks per each translation task. Our comprehensive set of qualitative and quantitative results demonstrate the effectiveness of our approach against the competing methods.

show abstract

Painting style transfer for head portraits using convolutional neural networks

Cited by 181 publications

References 43 publications

Neural Style Transfer: A Review

Neural Style Transfer: A Review

Neural Abstract Style Transfer for Chinese Traditional Painting

Manipulating Attributes of Natural Scenes via Hallucination

Contact Info

Product

Resources

About