Neural Style Palette: A Multimodal and Interactive Style Transfer From a Single Style Image

Virtusio, John Jethro; Ople, Jose Jaena Mari; Tan, Daniel Stanley; Tanveer, M.; Kumar, Neeraj; Hua, Kai-Lung

doi:10.1109/tmm.2021.3087026

Cited by 25 publications

(9 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Some related works have introduced user interactions, in which the interactive part is mainly some ways to adjust the style. Virtusio et al [44] allow users to control the ratio of different sub-styles separated from a style image. Akimoto et al [1] provide the fast soft color segmentation, which can extract and change the color distribution by users.…”

Section: Style Transfer With Interactionsmentioning

confidence: 99%

Interactive Style Transfer: All is Your Palette

Zheng¹,

Zhang²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural style transfer (NST) can create impressive artworks by transferring reference style to content image. Current imageto-image NST methods are short of fine-grained controls, which are often demanded by artistic editing. To mitigate this limitation, we propose a drawing-like interactive style transfer (IST) method, by which users can interactively create a harmonious-style image. Our IST method can serve as a brush, dip style from anywhere, and then paint to any region of the target content image. To determine the action scope, we formulate a fluid simulation algorithm, which takes styles as pigments around the position of brush interaction, and diffusion in style or content images according to the similarity maps. Our IST method expands the creative dimension of NST. By dipping and painting, even employing one style image can produce thousands of eye-catching works. The demo video is available in supplementary files or in http://mmcheng.net/ist.

show abstract

Section: Style Transfer With Interactionsmentioning

confidence: 99%

Interactive Style Transfer: All is Your Palette

Zheng¹,

Zhang²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…After that, a series of arbitrary style transfer methods [27]- [33] are proposed. Virtusio et al [34] present a method for interactively generating a variety of stylized images from only a single style input. Based on arbitrary style transfer methods, some video style transfer methods [35]- [37] with time consistency have emerged.…”

Section: A Texture Synthesis and Style Transfermentioning

confidence: 99%

Location-Free Camouflage Generation Network

Li¹,

Zhai²,

Cao³

et al. 2022

Preprint

View full text Add to dashboard Cite

Camouflage is a common visual phenomenon, which refers to hiding the foreground objects into the background images, making them briefly invisible to the human eye. Previous work has typically been implemented by an iterative optimization process. However, these methods struggle in 1) efficiently generating camouflage images using foreground and background with arbitrary structure; 2) camouflaging foreground objects to regions with multiple appearances (e.g. the junction of the vegetation and the mountains), which limit their practical application.To address these problems, this paper proposes a novel Locationfree Camouflage Generation Network (LCG-Net) that fuse highlevel features of foreground and background image, and generate result by one inference. Specifically, a Position-aligned Structure Fusion (PSF) module is devised to guide structure feature fusion based on the point-to-point structure similarity of foreground and background, and introduce local appearance features point-bypoint. To retain the necessary identifiable features, a new immerse loss is adopted under our pipeline, while a background patch appearance loss is utilized to ensure that the hidden objects look continuous and natural at regions with multiple appearances. Experiments show that our method has results as satisfactory as state-of-the-art in the single-appearance regions and are less likely to be completely invisible, but far exceed the quality of the state-of-the-art in the multi-appearance regions. Moreover, our method is hundreds of times faster than previous methods. Benefitting from the unique advantages of our method, we provide some downstream applications for camouflage generation, which show its potential. The related code and dataset will be released at https://github.com/Tale17/LCG-Net.

show abstract

“…Kim et al [24] utilized an uncorrelation loss to finetune the encoder and acquire the superior performance. Virtusio et al [25] adapted the style separation loss to encourage diversity, and utilized a unification loss to maintain the original style. Ma et al [26] took the connection between the subject and the art style into account and introduced a dual consistency loss that can reinforce the stylization effect adapted to the semantics of the content image and increase its diversity.…”

Section: A Arbitrary Style Transfermentioning

confidence: 99%

MNCAA: Balanced Style Transfer Based on Multi-level Normalized Cross Attention Alignment

Liu¹,

zhipeng²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

<p>Given a content image and an artistic style one, style transfer usually refers to applying the patterns learned from the style image to the content image to generate a new stylized image. Despite the thrilling success achieved by existing style transfer methods, most of them are in the mire of two limitations: 1) cannot preserve the structure of the content image well; 2) cannot generate delicate enough style effects or may produce significant artifacts. It is a challenge to maintain a balance between content structure preservation and style pattern transformation. In this work, we observe that multi-level content-style cross attention can extract the content features matching to the style characteristics at different feature levels. In addition, we also find that through multi-level dynamic normalization and alignment, hierarchical content-style cross attention can effectively transform the content image through the style characteristics of different levels while preserving its local structure and semantics as much as possible. The perceptual loss and the contextual loss are introduced individually to ensure the generated stylized image is close both to the content image and the style image in the feature space. At the same time, the identity loss of the content image and the style one is deployed to encourage the proposed model to retain the global appearance and the feature semantics of the input images without overall statistical deviation. A large number of qualitative and quantitative experiments and evaluations on the benchmark MSCOCO and WikiArt datasets demonstrate that, compared with other state-of-the-art (SOTA) methods, the proposed approach can obtain high-quality stylized images with structure-style balance. The project code is available at https://github.com/hengliusky/MNCAA.</p>

show abstract

Neural Style Palette: A Multimodal and Interactive Style Transfer From a Single Style Image

Cited by 25 publications

References 32 publications

Interactive Style Transfer: All is Your Palette

Interactive Style Transfer: All is Your Palette

Location-Free Camouflage Generation Network

MNCAA: Balanced Style Transfer Based on Multi-level Normalized Cross Attention Alignment

Contact Info

Product

Resources

About