Rethinking the Truly Unsupervised Image-to-Image Translation

Baek, Kyungjune; Choi, Yunjey; Uh, Youngjung; Yoo, Jaejun; Shim, Hyunjung

doi:10.1109/iccv48922.2021.01389

Cited by 81 publications

(29 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…MX-Font [23] employs multiple encoders for each reference image with disentanglement between content and style which makes the cross-lingual task possible. DG-Font [31] is an unsupervised framework based on TUNIT [2] by replacing the traditional convolutional blocks with Deformable blocks which enables the model to perform better on cursive characters which are more difficult to generate.…”

Section: Few-shot Font Generationmentioning

confidence: 99%

Few-Shot Font Generation by Learning Fine-Grained Local Styles

Tang¹,

Yiyang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Few-shot font generation (FFG), which aims to generate a new font with a few examples, is gaining increasing attention due to the significant reduction in labor cost. A typical FFG pipeline considers characters in a standard font library as content glyphs and transfers them to a new target font by extracting style information from the reference glyphs. Most existing solutions explicitly disentangle content and style of reference glyphs globally or componentwisely. However, the style of glyphs mainly lies in the local details, i.e. the styles of radicals, components, and strokes together depict the style of a glyph. Therefore, even a single character can contain different styles distributed over spatial locations. In this paper, we propose a new font generation approach by learning 1) the fine-grained local styles from references, and 2) the spatial correspondence between the content and reference glyphs. Therefore, each spatial location in the content glyph can be assigned with the right fine-grained style. To this end, we adopt cross-attention over the representation of the content glyphs as the queries and the representations of the reference glyphs as the keys and values. Instead of explicitly disentangling global or component-wise modeling, the cross-attention mechanism can attend to the right local styles in the reference glyphs and aggregate the reference styles into a fine-grained style representation for the given content glyphs. The experiments show that the proposed method outperforms the stateof-the-art methods in FFG. In particular, the user studies also demonstrate the style consistency of our approach significantly outperforms previous methods.

show abstract

Section: Few-shot Font Generationmentioning

confidence: 99%

Few-Shot Font Generation by Learning Fine-Grained Local Styles

Tang¹,

Yiyang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Image-to-Image (I2I) translation. Image-to-image translation aims to learn the style of images from a source domain into a target domain [32,20,36,1,38], which can be classified into two groups: the paired setting [19,19,35,29] (supervised) and an unpaired setting (unsupervised). Paired setting means the training set is supervised, each image from source domain has a corresponding label from target domain.…”

Section: Related Workmentioning

confidence: 99%

Exploring Negatives in Contrastive Learning for Unpaired Image-to-Image Translation

Lin¹,

Zhang²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

Figure 1: Visualization of the learned similarity by the feature extractor. Given an input and output image, we extract the features of these images through a feature extractor. We compute the learned similarities between the feature vectors of [(v, v − 1 ), ..., (v, v − N )] by using exp(v • v − /τ ). Specifically, v is a query element (the highlighted red dot in the output) and [v − 1 , ..., v − N ] are all the candidate patches in the input. Compared with other I2I translation method [28], the feature extractor of our model learns the cross-domain correspondence with a better saliency effect.

show abstract

“…CUT proposes patch-wise contrastive learning by cropping input and output images into patches and maximizing the mutual information between patches. Following CUT, TUNIT [Baek et al 2021] adopts contrastive learning on images with similar semantic structures. However, the semantic similarity assumption does not hold for arbitrary style transfer tasks, which leads the learned style representations to a significant performance drop.…”

Section: Related Workmentioning

confidence: 99%

Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning

Zhang¹,

Tang²,

Dong³

et al. 2022

Preprint

View full text Add to dashboard Cite

propose to learn style representation directly from image features instead of their second-order statistics, by analyzing the similarities and differences between multiple styles and considering the style distribution. Specifically, we present Contrastive Arbitrary Style Transfer (CAST), which is a new style representation learning and style transfer method via contrastive learning. Our framework consists of three key components, i.e., a multi-layer style projector for style code encoding, a domain enhancement module for effective learning of style distribution, and a generative network for image style transfer. We conduct qualitative and quantitative evaluations comprehensively to demonstrate that our approach achieves significantly better results compared to those obtained via state-of-the-art methods. Code and models are available at https://github.com/zyxElsa/CAST_pytorch. CCS CONCEPTS• Computing methodologies → Image processing.

show abstract

Rethinking the Truly Unsupervised Image-to-Image Translation

Cited by 81 publications

References 15 publications

Few-Shot Font Generation by Learning Fine-Grained Local Styles

Few-Shot Font Generation by Learning Fine-Grained Local Styles

Exploring Negatives in Contrastive Learning for Unpaired Image-to-Image Translation

Domain Enhanced Arbitrary Image Style Transfer via Contrastive Learning

Contact Info

Product

Resources

About