Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts

Park, Song; Chun, Sanghyuk; Cha, Junbum; Lee, Bado; Shim, Hyunjung

doi:10.48550/arxiv.2104.00887

Cited by 3 publications

(14 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Most Few-shot font generation (FFG) methods focus on disentangling the content feature and style feature from the given glyphs. Based on different kinds of feature representation, FFG methods can be divided into two main categories: global feature representation [1,8,29,33] and component-based feature representation [4,13,22,23]. In methods that apply global feature representation, vectors related to content and style are extracted from content glyphs and reference glyphs respectively.…”

Section: Few-shot Font Generationmentioning

confidence: 99%

“…LF-Font [22] designs a component-conditioned reference encoder to extract component-wise features from reference images. MX-Font [23] employs multiple encoders for each reference image with disentanglement between content and style which makes the cross-lingual task possible. DG-Font [31] is an unsupervised framework based on TUNIT [2] by replacing the traditional convolutional blocks with Deformable blocks which enables the model to perform better on cursive characters which are more difficult to generate.…”

Section: Few-shot Font Generationmentioning

confidence: 99%

“…However, the previous FFG works fail to fully explore the style maps from k-shot references. When receiving kshot reference images, they tend to explicitly disentangle style and content of images globally or component-wisely and conduct a average operation among extracted features [22,23]. The local details extracted from each reference are significantly weakened by disentanglement and the average operation.…”

Section: Few-shot Font Generationmentioning

confidence: 99%

“…In recent years, some works tried to tackle few-shot font generation (FFG) with a few-shot Image-to-Image translation (I2I) scheme [4,8,22,23,31]. Unlike zi2zi, the font style representation is learned from a few reference exemplars, rather than learning embeddings from different font labels.…”

Section: Introductionmentioning

confidence: 99%

“…global style representation and componentwise style representation. The former one models the glyph style as universal representation for each font [8,18,31], while the latter one utilizes component-wise style representation from different reference exemplars in the same font [4,13,22,23].…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Few-Shot Font Generation by Learning Fine-Grained Local Styles

Tang¹,

Yiyang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

Few-shot font generation (FFG), which aims to generate a new font with a few examples, is gaining increasing attention due to the significant reduction in labor cost. A typical FFG pipeline considers characters in a standard font library as content glyphs and transfers them to a new target font by extracting style information from the reference glyphs. Most existing solutions explicitly disentangle content and style of reference glyphs globally or componentwisely. However, the style of glyphs mainly lies in the local details, i.e. the styles of radicals, components, and strokes together depict the style of a glyph. Therefore, even a single character can contain different styles distributed over spatial locations. In this paper, we propose a new font generation approach by learning 1) the fine-grained local styles from references, and 2) the spatial correspondence between the content and reference glyphs. Therefore, each spatial location in the content glyph can be assigned with the right fine-grained style. To this end, we adopt cross-attention over the representation of the content glyphs as the queries and the representations of the reference glyphs as the keys and values. Instead of explicitly disentangling global or component-wise modeling, the cross-attention mechanism can attend to the right local styles in the reference glyphs and aggregate the reference styles into a fine-grained style representation for the given content glyphs. The experiments show that the proposed method outperforms the stateof-the-art methods in FFG. In particular, the user studies also demonstrate the style consistency of our approach significantly outperforms previous methods.

show abstract

Section: Few-shot Font Generationmentioning

confidence: 99%

Section: Few-shot Font Generationmentioning

confidence: 99%

Section: Few-shot Font Generationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Few-Shot Font Generation by Learning Fine-Grained Local Styles

Tang¹,

Yiyang²,

Liu³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

DeepCalliFont: Few-Shot Chinese Calligraphy Font Synthesis by Integrating Dual-Modality Generative Models

Liu,

Lian

2024

AAAI

View full text Add to dashboard Cite

Few-shot font generation, especially for Chinese calligraphy fonts, is a challenging and ongoing problem. With the help of prior knowledge that is mainly based on glyph consistency assumptions, some recently proposed methods can synthesize high-quality Chinese glyph images. However, glyphs in calligraphy font styles often do not meet these assumptions. To address this problem, we propose a novel model, DeepCalliFont, for few-shot Chinese calligraphy font synthesis by integrating dual-modality generative models. Specifically, the proposed model consists of image synthesis and sequence generation branches, generating consistent results via a dual-modality representation learning strategy. The two modalities (i.e., glyph images and writing sequences) are properly integrated using a feature recombination module and a rasterization loss function. Furthermore, a new pre-training strategy is adopted to improve the performance by exploiting large amounts of uni-modality data. Both qualitative and quantitative experiments have been conducted to demonstrate the superiority of our method to other state-of-the-art approaches in the task of few-shot Chinese calligraphy font synthesis. The source code can be found at https://github.com/lsflyt-pku/DeepCalliFont.

show abstract

CCFont: Component-Based Chinese Font Generation Model Using Generative Adversarial Networks (GANs)

2022

View full text Add to dashboard Cite

Font generation using deep learning has made considerable progress using image style transfer, but the automatic conversion/generation of Chinese characters still remains a difficult task owing to the complex character shape and large number of Chinese characters. Most known Chinese character generation models use the image conversion method of the Chinese character shape itself; however, it is difficult to reproduce complex Chinese characters. Recent methods have utilized character compositionality by separating up to three or four components to improve the quality of generated characters, but it is still difficult to generate high-quality results for complex Chinese characters with many components. In this study, we proposed the CCFont model (component-based Chinese font generation model using generative adversarial networks (GANs)) that automatically generates all Chinese characters using Chinese character components (up to 17 components). The CCFont model generates all Chinese characters in various styles using the components of Chinese characters based on conditional GAN. By acquiring local style information from the components, the information is more accurate and there is less information loss than when global information is obtained from the image of the entire character, reducing the failure of style conversion and improving quality to produce high-quality results. Additionally, the CCFont model generates high-quality results without any additional training (zero-shot font generation without any additional training) for the first-seen characters and styles. For example, the CCFont model, which was trained with only traditional Chinese (TC) characters, generates high-quality results for languages that can be divided into components, such as Korean and Thai, as well as simplified Chinese (SC) characters that are only seen during inference. CCFont can be adopted as a multi-lingual font-generation model that can be applied to all languages, which can be divided into components. To the best of our knowledge, the proposed method is the first to generate a zero-shot multilingual generation model using components. Qualitative and quantitative experiments were conducted to demonstrate the effectiveness of the proposed method.

show abstract

Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts

Cited by 3 publications

References 0 publications

Few-Shot Font Generation by Learning Fine-Grained Local Styles

Few-Shot Font Generation by Learning Fine-Grained Local Styles

DeepCalliFont: Few-Shot Chinese Calligraphy Font Synthesis by Integrating Dual-Modality Generative Models

CCFont: Component-Based Chinese Font Generation Model Using Generative Adversarial Networks (GANs)

Contact Info

Product

Resources

About