Decoder Network over Lightweight Reconstructed Feature for Fast Semantic Style Transfer

Lu, Ming; Zhao, Hao; Yao, Anbang; Xu, Feng; Chen, Yurong; Zhang, Li

doi:10.1109/iccv.2017.270

Cited by 50 publications

(38 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Jiang'17 [89] Fashion Chen'18 [72] 3D 2D Johnson'16 [47] Ulyanov'16 [48] Ulyanov'17 [50] Li'16 [52] Liu'17 [63] Depth Wang'17 [62] Jing'18 [61] Stroke Size Improvement Image Video Huang'17 [78] Gupta'17 [79] Chen'17 [80] Lu'17 [70] Semantic Non-Photorealistic Zhang'17 [87] Photorealistic Azadi'18 [83] Character Figure 2: A taxonomy of NST techniques. Our proposed NST taxonomy extends the IB-AR taxonomy proposed by Kyprianidis et al [14].…”

Section: Per-style-per-model Neural Methodsmentioning

confidence: 99%

“…Both [65] and [68] are based on IOB-NST algorithms and therefore leave much room for improvement. Lu et al [70] speed up the process by optimising the objective function in feature space, instead of in pixel space. More specifically, they propose to do feature reconstruction, instead of image reconstruction as previous algorithms do.…”

Section: Improvements and Extensionsmentioning

confidence: 99%

See 1 more Smart Citation

Neural Style Transfer: A Review

Jing

Yang

Feng

et al. 2020

IEEE Trans. Visual. Comput. Graphics

507

309

View full text Add to dashboard Cite

The seminal work of Gatys et al. demonstrated the power of Convolutional Neural Networks (CNNs) in creating artistic imagery by separating and recombining image content and style. This process of using CNNs to render a content image in different styles is referred to as Neural Style Transfer (NST). Since then, NST has become a trending topic both in academic literature and industrial applications. It is receiving increasing attention and a variety of approaches are proposed to either improve or extend the original NST algorithm. In this paper, we aim to provide a comprehensive overview of the current progress towards NST. We first propose a taxonomy of current algorithms in the field of NST. Then, we present several evaluation methods and compare different NST algorithms both qualitatively and quantitatively. The review concludes with a discussion of various applications of NST and open problems for future research. A list of papers discussed in this review, corresponding codes, pre-trained models and more comparison results are publicly available at: https://github.com/ycjing/Neural-Style-Transfer-Papers. IndexTerms-Neural style transfer (NST), convolutional neural network ! Neural Style Transfer Example-Based Techniques Colour Image Analogy Texture Model-Optimisation-Based Offline Neural Methods Multiple-Style-Per-Model Neural Methods Dumoulin'17 [53] Chen'17 [54] Li'17 [55] Zhang'17 [56] Luan'17 [84] Mechrez'17 [85]Photorealistic Liao'17 [88] Attribute Champandard'16 [65] Doodle Ruder'16 [74] Video Selim'16 [73] Portrait Castillo'17 [71] Instance Gatys'17 [60] Improvement Image Gatys'16 [10] Li'17 [42] Risser'17 [44] Li'17 [45] Li'16 [46] Image Champandard'16 [65] Chen'16 [68] Mechrez'18 [69]

show abstract

Section: Per-style-per-model Neural Methodsmentioning

confidence: 99%

Section: Improvements and Extensionsmentioning

confidence: 99%

Neural Style Transfer: A Review

Jing

Yang

Feng

et al. 2020

IEEE Trans. Visual. Comput. Graphics

507

309

View full text Add to dashboard Cite

show abstract

“…Another type of control is spatial, allowing users to ensure that certain regions of the output should be stylized using only features from a manually selected region of the style image (or that different regions of the output image should be stylized based on different style images). In [5,18] the authors propose forms of spatial control based on the user defining matched regions of the image by creating a dense mask for both the style and content image. We demonstrate that it is straightforward to incorporate this type of user-control into our formulation of style transfer.…”

Section: Stylementioning

confidence: 99%

Style Transfer by Relaxed Optimal Transport and Self-Similarity

Kolkin

Salavon

Shakhnarovich

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

212

226

View full text Add to dashboard Cite

Style transfer algorithms strive to render the content of one image using the style of another. We propose Style Transfer by Relaxed Optimal Transport and Self-Similarity (STROTSS), a new optimization-based style transfer algorithm. We extend our method to allow user-specified pointto-point or region-to-region control over visual similarity between the style image and the output. Such guidance can be used to either achieve a particular visual effect or correct errors made by unconstrained style transfer. In order to quantitatively compare our method to prior work, we conduct a large-scale user study designed to assess the stylecontent tradeoff across settings in style transfer algorithms.Our results indicate that for any desired level of content preservation, our method provides higher quality stylization than prior work. Code is available here.

show abstract

“…Once F l o is achieved, we will get the output image I o by decoding F l o to image domain. The decoder can be pretrained for efficiency [30,20,33]. By contrast, we adopt an different decoder learning from theirs.…”

Section: Single Layer Optimizationmentioning

confidence: 99%

Arbitrary Style Transfer with Deep Feature Reshuffle

Chen

Liao

et al. 2018

2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition

167

162

View full text Add to dashboard Cite

This paper introduces a novel method by reshuffling deep features (i.e., permuting the spacial locations of a feature map) of the style image for arbitrary style transfer. We theoretically prove that our new style loss based on reshuffle connects both global and local style losses respectively used by most parametric and non-parametric neural style transfer methods. This simple idea can effectively address the challenging issues in existing style transfer methods. On one hand, it can avoid distortions in local style patterns, and allow semantic-level transfer, compared with neural parametric methods. On the other hand, it can preserve globally similar appearance to the style image, and avoid wash-out artifacts, compared with neural non-parametric methods. Based on the proposed loss, we also present a progressive feature-domain optimization approach. The experiments show that our method is widely applicable to various styles, and produces better quality than existing methods.

show abstract

Decoder Network over Lightweight Reconstructed Feature for Fast Semantic Style Transfer

Cited by 50 publications

References 9 publications

Neural Style Transfer: A Review

Neural Style Transfer: A Review

Style Transfer by Relaxed Optimal Transport and Self-Similarity

Arbitrary Style Transfer with Deep Feature Reshuffle

Contact Info

Product

Resources

About