Photographic style transfer

Wang, Li; Zhao, Wang; Yang, Xiaosong; Hu, Shi‐Min; Zhang, Jianjun

doi:10.1007/s00371-018-1609-4

Cited by 16 publications

(8 citation statements)

References 27 publications

(111 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gatys et al [156] demonstrated that visual features of models could be combined to represent image styles. It arises in a context of strong growth in DNNs for several applications, including art and painting [157,158]. For example, Lian et al [157] proposed a style transfer-based method that takes any natural portrait of a human and transforms it into Picasso's cubism style.…”

Section: Style Transfermentioning

confidence: 99%

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

et al. 2021

View full text Add to dashboard Cite

The research progress in multimodal learning has grown rapidly over the last decade in several areas, especially in computer vision. The growing potential of multimodal data streams and deep learning algorithms has contributed to the increasing universality of deep multimodal learning. This involves the development of models capable of processing and analyzing the multimodal information uniformly. Unstructured real-world data can inherently take many forms, also known as modalities, often including visual and textual content. Extracting relevant patterns from this kind of data is still a motivating goal for researchers in deep learning. In this paper, we seek to improve the understanding of key concepts and algorithms of deep multimodal learning for the computer vision community by exploring how to generate deep models that consider the integration and combination of heterogeneous visual cues across sensory modalities. In particular, we summarize six perspectives from the current literature on deep multimodal learning, namely: multimodal data representation, multimodal fusion (i.e., both traditional and deep learning-based schemes), multitask learning, multimodal alignment, multimodal transfer learning, and zero-shot learning. We also survey current multimodal applications and present a collection of benchmark datasets for solving problems in various vision domains. Finally, we highlight the limitations and challenges of deep multimodal learning and provide insights and directions for future research.

show abstract

Section: Style Transfermentioning

confidence: 99%

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

et al. 2021

View full text Add to dashboard Cite

show abstract

“…The Neural Algorithm of Artistic Style [35][36][37] uses a trained VGGNet, which is a network that adopts deep CNN layers [27] to obtain content spatial information (content representation) and style texture information (style information) in the middle layers. By repeating feed-forward and back propagation, the algorithm trains the VGGNet to construct an image that has the shapes of the content image and the style of the style image.…”

Section: Neural Style Transfermentioning

confidence: 99%

Translation of Real-World Photographs into Artistic Images via Conditional CycleGAN and StarGAN

Komatsu

Gonsalves

2021

SN COMPUT. SCI.

View full text Add to dashboard Cite

To translate a real-world photograph into an artistic image in the style of a famous artist, the selection of colors and brushstrokes should reflect those of the artist. A one-to-one domain translation architecture, CycleGAN, trained with an unpaired dataset can be used to translate a real-world photograph into an artistic image. However, to translate images in N number of multi-artistic styles, the disadvantage is that more than one CycleGAN must be trained corresponding to each style. Here, we develop a single deep learning architecture that can be controlled to yield multiple artistic styles by adding a conditional vector. The overall architecture includes a one-to-N domain translation architecture, namely, a conditional CycleGAN, and an N-to-N domain translation architecture, namely, StarGAN, for translating into five different artistic styles. An evaluation of the trained models reveal that multiple artistic styles can be produced from a single real-world photograph only by adjusting the conditional input.

show abstract

“…Follow-up study [13] has successfully applied it to image style transfer. The Gram-based optimization method [13] presented by Gatys et al inspired many subsequent studies [18,34,41] on style transfer, which mainly focused on replacing the iterative optimization to one feed-forward calculation. Recently, Chen et al [6] proposed a fast algorithm to transfer image style.…”

Section: Related Workmentioning

confidence: 99%

Example-based rapid generation of vegetation on terrain via CNN-based distribution learning

Zhang

Wang

et al. 2019

Vis Comput

View full text Add to dashboard Cite

Modeling large-scale vegetation on terrain is an important and challenging task in computer games, movie production and other digital entertainment applications. In this work, we propose a novel example-based method for rapid generation of vegetation in outdoor natural environments. Its central idea is to learn the vegetation distribution on terrain via deep convolution neural networks. We first use a pre-trained deep neural network to extract rich local information from the terrain pertinent to vegetation distribution. Second, we produce the initial features of the target vegetation distribution based on patch matching and further introduce a network that generates a vegetation density map based on the initial features. Third, during the synthesis stage, we propose a procedural method to generate the vegetation distribution data corresponding to the terrain data. Our research work confirms that the image features extracted by the pre-trained deep neural network could be utilized to explore the connection between vegetation and terrain. We validate our new method over various outdoor scenes, including procedural generated scenes and scenes with manual control on tree patterns. The experimental results demonstrate that our method can rapidly produce new realistic scenes for outdoor natural environments, which relies on the mechanism of learning correlationship between vegetation distribution and terrain data.

show abstract

Photographic style transfer

Cited by 16 publications

References 27 publications

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

Translation of Real-World Photographs into Artistic Images via Conditional CycleGAN and StarGAN

Example-based rapid generation of vegetation on terrain via CNN-based distribution learning

Contact Info

Product

Resources

About