Adaptive Convolutions for Structure-Aware Style Transfer

Chandran, Prashanth; Zoss, Gaspard; Gotardo, Paulo; Groß, Markus; Bradley, Derek

doi:10.1109/cvpr46437.2021.00788

Cited by 47 publications

(41 citation statements)

References 30 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As such their latent space is different between training and inversion while ours remains as it was. Also similar to our work is AdaConv [9] for style transfer, but our modified style injection stays much closer to the original StyleGAN and they do not address inversion.…”

Section: Previous Workmentioning

confidence: 86%

Overparameterization Improves StyleGAN Inversion

Poirier-Ginter¹,

Lessard²,

Smith³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Section: Previous Workmentioning

confidence: 86%

Overparameterization Improves StyleGAN Inversion

Poirier-Ginter¹,

Lessard²,

Smith³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Different from the traditional CNN, where the filter weights stay fixed once trained, the dynamic filters are generated by separate network branches and can be changed according to the input on-the-fly. Due to its adaptive nature, it can increase the flexibility of a network and has been applied to various tasks, such as super-resolution (Hu et al 2019), point cloud segmentation (Xu et al 2020), image deblurring (Lee et al 2021), and style transfer (Chandran et al 2021). However, generating such depthwise-separable and spatially-varying filters usually entailed memory intensive network architectures, which was computation-heavy and time-consuming.…”

Section: Dynamic Filtersmentioning

confidence: 99%

Generative Adaptive Convolutions for Real-World Noisy Image Denoising

Zhang

et al. 2022

AAAI

View full text Add to dashboard Cite

Recently, deep learning techniques are soaring and have shown dramatic improvements in real-world noisy image denoising. However, the statistics of real noise generally vary with different camera sensors and in-camera signal processing pipelines. This will induce problems of most deep denoisers for the overfitting or degrading performance due to the noise discrepancy between the training and test sets. To remedy this issue, we propose a novel flexible and adaptive denoising network, coined as FADNet. Our FADNet is equipped with a plane dynamic filter module, which generates weight filters with flexibility that can adapt to the specific input and thereby impedes the FADNet from overfitting to the training data. Specifically, we exploit the advantage of the spatial and channel attention, and utilize this to devise a decoupling filter generation scheme. The generated filters are conditioned on the input and collaboratively applied to the decoded features for representation capability enhancement. We additionally introduce the Fourier transform and its inverse to guide the predicted weight filters to adapt to the noisy input with respect to the image contents. Experimental results demonstrate the superior denoising performances of the proposed FADNet versus the state-of-the-art. In contrast to the existing deep denoisers, our FADNet is not only flexible and efficient, but also exhibits a compelling generalization capability, enjoying tremendous potential for practical usage.

show abstract

“…A popular practice [7], [16], [25], [46] directly aligns the statistics between the new artwork and the given style image in the instance feature space. Some works [5], [15], [35] learn to predict dynamic parameters for any specified style. Although such methods greatly improve the flexibility, the processing granularity is still full image and cannot meet the needs of all users.…”

Section: Neural Style Transfermentioning

confidence: 99%

“…N EURAL style transfer (NST) is a widely concerned artistic creation tool, which aims at transferring the artistic style from a reference image to a content image. Many impressive NST methods have been proposed by exploring content retention [7], [18], robustness analysis [9], [46], ceiling of style types [5], [16], geometric changes [28], brushstroke simulation [19], etc. Recently, several works have introduced the user's intention, making NST can be manually adjusted in the degree of stylization [14], semantic guidance [20], [30], or masked style splicing [40],…”

Section: Introductionmentioning

confidence: 99%

Interactive Style Transfer: All is Your Palette

Zheng¹,

Zhang²,

Zhang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Neural style transfer (NST) can create impressive artworks by transferring reference style to content image. Current imageto-image NST methods are short of fine-grained controls, which are often demanded by artistic editing. To mitigate this limitation, we propose a drawing-like interactive style transfer (IST) method, by which users can interactively create a harmonious-style image. Our IST method can serve as a brush, dip style from anywhere, and then paint to any region of the target content image. To determine the action scope, we formulate a fluid simulation algorithm, which takes styles as pigments around the position of brush interaction, and diffusion in style or content images according to the similarity maps. Our IST method expands the creative dimension of NST. By dipping and painting, even employing one style image can produce thousands of eye-catching works. The demo video is available in supplementary files or in http://mmcheng.net/ist.

show abstract

Adaptive Convolutions for Structure-Aware Style Transfer

Cited by 47 publications

References 30 publications

Overparameterization Improves StyleGAN Inversion

Overparameterization Improves StyleGAN Inversion

Generative Adaptive Convolutions for Real-World Noisy Image Denoising

Interactive Style Transfer: All is Your Palette

Contact Info

Product

Resources

About