A Neural Algorithm of Artistic Style

Gatys, Leon A.; Ecker, Alexander; Bethge, Matthias

doi:10.48550/arxiv.1508.06576

Cited by 274 publications

(538 citation statements)

References 0 publications

Supporting

Mentioning

533

Contrasting

Unclassified

Order By: Relevance

“…The term "style transfer" was proposed by [21], who transferred the style of an input image to another specific style. A large body of work has since emerged in this field [22].…”

Section: A Music Style Transfermentioning

confidence: 99%

Can Machines Generate Personalized Music? A Hybrid Favorite-aware Method for User Preference Music Transfer

Hu¹,

Liu²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

User preference music transfer (UPMT) is a new problem in music style transfer that can be applied to many scenarios but remains understudied. Transferring an arbitrary song to fit a user's preferences increases musical diversity and improves user engagement, which can greatly benefit individuals' mental health. Most music style transfer approaches rely on datadriven methods. In general, however, constructing a large training dataset is challenging because users can rarely provide enough of their favorite songs. To address this problem, this paper proposes a novel hybrid method called User Preference Transformer (UP-Transformer) which uses prior knowledge of only one piece of a user's favorite music. Based on the distribution of music events in the provided music, we propose a new favorite-aware loss function to fine-tune the Transformer-based model. Two steps are proposed in the transfer phase to achieve UPMT based on the extracted music pattern in a user's favorite music. Additionally, to alleviate the problem of evaluating melodic similarity in music style transfer, we propose a new concept called pattern similarity (PS) to measure the similarity between two pieces of music. Statistical tests indicate that the results of PS are consistent with the similarity score in a qualitative experiment. Furthermore, experimental results on subjects show that the transferred music achieves better performance in musicality, similarity, and user preferences.

show abstract

“…The term "style transfer" was proposed by [21], who transferred the style of an input image to another specific style. A large body of work has since emerged in this field [22].…”

Section: A Music Style Transfermentioning

confidence: 99%

Can Machines Generate Personalized Music? A Hybrid Favorite-aware Method for User Preference Music Transfer

Hu¹,

Liu²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Style Encoder Pre-training: Gram matrices have been introduced to represent the stylistic features of a reference image in neural style transfer [23]. Many models use learned style encoding that is similar to the style encoding obtained from gram matrices to enforce condition on generated image [18,24].…”

Section: Frameworkmentioning

confidence: 99%

Unsupervised Image to Image Translation for Multiple Retinal Pathology Synthesis in Optical Coherence Tomography Scans

Hemanth¹,

Girish²

2021

Preprint

View full text Add to dashboard Cite

Image to Image Translation (I2I) is a challenging computer vision problem used in numerous domains for multiple tasks. Recently, ophthalmology became one of the major fields where the application of I2I is increasing rapidly. One such application is the generation of synthetic retinal optical coherence tomographic (OCT) scans. Existing I2I methods require training of multiple models to translate images from normal scans to a specific pathology: limiting the use of these models due to their complexity. To address this issue, we propose an unsupervised multi-domain I2I network with pre-trained style encoder that translates retinal OCT images in one domain to multiple domains. We assume that the image splits into domain-invariant content and domain-specific style codes, and pre-train these style codes. The performed experiments show that the proposed model outperforms state-of-the-art models like MUNIT and CycleGAN synthesizing diverse pathological scans.

show abstract

“…Nevertheless, recent approaches have leveraged the capabilities of deep latent spaces within convolutional neural networks to disentangle style from content [10]. A seminal work by Gatys et al [11] uses a VGG-19 convolutional neural network [12] pre-trained on ImageNet [13] as a feature descriptor for images, in which style and content are related to different layers of the network, and transferred by gradient descent optimization. Their work on style transfer has been extended for single images [14], [15], [16], [17] and video [18], as well as for developing image-space distance metrics that resemble human perception [19].…”

Section: Related Workmentioning

confidence: 99%

“…Figure 11 shows additional results of material stylizations. In these examples, we used the method of Gatys et al [11] to stylize a small patch of the material. Then, we trained a model using photometricNet and diffuseNet.…”

Section: Interactive Stylizationsmentioning

confidence: 99%

“…11: Our method allows for interactive material-aware visual attribute transfers. Using an off-the-shelf style transfer algorithm [11], we can transfer the style of one image I style to the content of another I content , obtaining a visual attribute ω. Training a M ω to learn this relationship, we can find predictable style mappings, that we can transfer to guidance images X, obtaining style transfers M ω (X).…”

Section: Creation Of Large Scale Digital Materials Assetsmentioning

confidence: 99%

See 1 more Smart Citation

Neural Photometry-guided Visual Attribute Transfer

Rodriguez-Pardo,

Garces

2021

Preprint

View full text Add to dashboard Cite

We present a deep learning-based method for propagating spatially-varying visual material attributes (e.g. texture maps or image stylizations) to larger samples of the same or similar materials. For training, we leverage images of the material taken under multiple illuminations and a dedicated data augmentation policy, making the transfer robust to novel illumination conditions and affine deformations. Our model relies on a supervised image-to-image translation framework and is agnostic to the transferred domain; we showcase a semantic segmentation, a normal map, and a stylization. Following an image analogies approach, the method only requires the training data to contain the same visual structures as the input guidance. Our approach works at interactive rates, making it suitable for material edit applications. We thoroughly evaluate our learning methodology in a controlled setup providing quantitative measures of performance. Last, we demonstrate that training the model on a single material is enough to generalize to materials of the same type without the need for massive datasets.

show abstract

A Neural Algorithm of Artistic Style

Cited by 274 publications

References 0 publications

Can Machines Generate Personalized Music? A Hybrid Favorite-aware Method for User Preference Music Transfer

Can Machines Generate Personalized Music? A Hybrid Favorite-aware Method for User Preference Music Transfer

Unsupervised Image to Image Translation for Multiple Retinal Pathology Synthesis in Optical Coherence Tomography Scans

Neural Photometry-guided Visual Attribute Transfer

Contact Info

Product

Resources

About