S2WAT: Image Style Transfer via Hierarchical Vision Transformer using Strips Window Attention

Zhang, Chiyu; Yang, Jing; Wang, Lei; Dai, Zaiyan

doi:10.48550/arxiv.2210.12381

Cited by 2 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to verify the effectiveness of the method in this research, the model was compared with PAMA [15], S2WAT [13], StyTr-2 [12], and AesUST [11]. The transfer efficiency was first assessed, and the time required to generate single stylized images of two different size sizes, 256 × 256 and 512 × 512, is shown in Table 1.…”

Section: Objective Evaluation Of Resultsmentioning

confidence: 99%

“…Wang et al [11] incorporated aesthetic features into the style transfer network with an aesthetic discriminator. Both Deng et al [12] and Zhang et al [13] used transformer-based approaches [21] for arbitrary image style transfer. Since then, several style transfer methods based on contrastive learning have emerged [22][23][24].…”

Section: Image Style Transfermentioning

confidence: 99%

“…Figure 1. The stylized comparison graph, comparing our method with PAMA [15], AesUST [11], StyTr-2 [12], and S2WAT [13], showing that the stylized image is square; therefore, the stylized image generated by the model should also be square.…”

Section: Introductionmentioning

confidence: 97%

“…However, its generated stylized images are more colorful, resulting in an insufficient stylization effect. Deng et al [12] and Zhang et al [13] both used transformer-based approaches [14] for arbitrary image style migration, but the number of transformer-based model parameters was too large, and thus the approaches consumed too much training time. Luo et al [15] proposed PAMA (progressive attentional manifold alignment), using attentional operations and spatially aware interpolation to perform alignment operations on the content and style space.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Image Style Transfer Based on Dynamic Convolutional Manifold Alignment of Halo Attention

Yang

2023

Electronics

View full text Add to dashboard Cite

The objective of image style transfer is to render an image with artistic features of a style reference while preserving the details of the content image. With the development of deep learning, many arbitrary style transfer methods have emerged. From the recent arbitrary style transfer algorithms, it has been found that the images generated suffer from the problem of poorly stylized quality. To solve this problem, we propose an arbitrary style transfer algorithm based on halo attention dynamic convolutional manifold alignment. First, the features of the content image and style image are extracted by a pre-trained VGG encoder. Then, the features are extracted by halo attention and dynamic convolution, and then the content feature space and style feature space are aligned by attention operations and spatial perception interpolation. The output is achieved through dynamic convolution and halo attention. During this process, multi-level loss functions are used, and total variation loss is introduced to eliminate noise. The manifold alignment process is then repeated three times. Finally, the pre-trained VGG decoder is used to output the stylized image. The experimental results show that our proposed method can generate high-quality stylized images, achieving values of 33.861, 2.516, and 3.602 for ArtFID, style loss, and content loss, respectively. A qualitative comparison with existing algorithms showed that it achieved good results. In future work, we will aim to make the model lightweight.

show abstract

Section: Objective Evaluation Of Resultsmentioning

confidence: 99%

Section: Image Style Transfermentioning

confidence: 99%

Section: Introductionmentioning

confidence: 97%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Image Style Transfer Based on Dynamic Convolutional Manifold Alignment of Halo Attention

Yang

2023

Electronics

View full text Add to dashboard Cite

show abstract

“…Recently, an increasing number of reports on the cross-fusion between style transfer algorithms and classic models have emerged. Zhang et al [ 26 ] proposed a hierarchical vision transformer using strip window attention. This approach realizes accurate style transfer by focusing on local image domains and adapting to a wide range of styles.…”

Section: Introductionmentioning

confidence: 99%

An improved defocusing adaptive style transfer method based on a stroke pyramid

et al. 2023

View full text Add to dashboard Cite

Image style transfer aims to assign a specified artist’s style to a real image. However, most existing methods cannot generate textures of various thicknesses due to the rich semantic information of the input image. The image loses some semantic information through style transfer with a uniform stroke size. To address the above problems, we propose an improved multi-stroke defocus adaptive style transfer framework based on a stroke pyramid, which mainly fuses various stroke sizes in the image spatial dimension to enhance the image content interpretability. We expand the receptive field of each branch and then fuse the features generated by the multiple branches based on defocus degree. Finally, we add an additional loss term to enhance the structural features of the generated image. The proposed model is trained using the Common Objects in Context (COCO) and Synthetic Depth of Field (SYNDOF) datasets, and the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) are used to evaluate the overall quality of the output image and its structural similarity with the content image, respectively. To validate the feasibility of the proposed algorithm, we compare the average PSNR and SSIM values of the output of the modified model and those of the original model. The experimental results show that the modified model improves the PSNR and SSIM values of the outputs by 1.43 and 0.12 on average, respectively. Compared with the single-stroke style transfer method, the framework proposed in this study improves the readability of the output images with more abundant visual expression.

show abstract

S2WAT: Image Style Transfer via Hierarchical Vision Transformer using Strips Window Attention

Cited by 2 publications

References 0 publications

Image Style Transfer Based on Dynamic Convolutional Manifold Alignment of Halo Attention

Image Style Transfer Based on Dynamic Convolutional Manifold Alignment of Halo Attention

An improved defocusing adaptive style transfer method based on a stroke pyramid

Contact Info

Product

Resources

About