Few-Shot Font Generation by Learning Fine-Grained Local Styles

Tang, Licheng; Cai, Yiyang; Liu, Jiaming; Hong, Zhibin; Gong, Maoguo; Fan, Minhu; Han, Junyu; Liu, Jingtuo; Ding, Errui; Wang, Jingdong

doi:10.1109/cvpr52688.2022.00774

Cited by 36 publications

(15 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We compare our method with seven state-of-the-art methods: one image-to-image translation method (FUNIT (Liu et al 2019)) and six Chinese font generation methods (LF-Font (Park et al 2021a), MX-Font (Park et al 2021b), DG-Font (Xie et al 2021), CG-GAN (Kong et al 2022), Fs-Font (Tang et al 2022), and CF-Font (Wang et al 2023) 18.67 .4823 .1688.3400 DG-Font 19.81 .4532 .2047.3646 MX-Font 9.32 .4605 .1603.3571 Fs-Font 31.40 .4270 .2160.3855 CG-GAN 7.72 .4655 .1721.3544 CF-Font 14.20 .4396 .2139.3713 Diff-Font 12.08 .4192 .2022.3877 Ours 7.67 .4942 .1426 Table 2: Quantitative Results on UFSC. we use the font of Song as the source, and all methods are trained based on their official codes.…”

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

“…Although these methods have achieved remarkable success in font generation, they still suffer from complex character generation and large style variation transfer, leading to severe stroke missing, artifacts, blurriness, layout errors, and style inconsistency as shown in Figure 1(b)(c). Retrospectively, most font generation approaches (Park et al 2021a,b;Xie et al 2021;Tang et al 2022;Liu et al 2022;Kong et al 2022;Wang et al 2023) adopt a GANbased (Goodfellow et al 2014) framework which potentially suffers from unstable training due to their adversarial training nature. Moreover, most of these methods perceive content information through only single-scale highlevel features, omitting the fine-grained details that are crucial to preserving the source content, especially for complex characters.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning

Yang,

Peng,

Kong

et al. 2024

AAAI

View full text Add to dashboard Cite

Automatic font generation is an imitation task, which aims to create a font library that mimics the style of reference images while preserving the content from source images. Although existing font generation methods have achieved satisfactory performance, they still struggle with complex characters and large style variations. To address these issues, we propose FontDiffuser, a diffusion-based image-to-image one-shot font generation method, which innovatively models the font imitation task as a noise-to-denoise paradigm. In our method, we introduce a Multi-scale Content Aggregation (MCA) block, which effectively combines global and local content cues across different scales, leading to enhanced preservation of intricate strokes of complex characters. Moreover, to better manage the large variations in style transfer, we propose a Style Contrastive Refinement (SCR) module, which is a novel structure for style representation learning. It utilizes a style extractor to disentangle styles from images, subsequently supervising the diffusion model via a meticulously designed style contrastive loss. Extensive experiments demonstrate FontDiffuser's state-of-the-art performance in generating diverse characters and styles. It consistently excels on complex characters and large style changes compared to previous methods. The code is available at https://github.com/yeungchenwa/FontDiffuser.

show abstract

Section: Comparison With State-of-the-art Methodsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning

Yang,

Peng,

Kong

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Based on the same components in characters, Park et al proposed LF-Font (Park et al 2020) and MX-Font (Park et al 2021), establishing a relation between the local features and components. Similarly, Tang et al (2022) developed an algorithm to search more suitable reference sets based on the prior knowledge of the existence of repeating components in FS-Font. Although these existing methods can synthesize visually-pleasing glyph images, they typically fail to synthesize satisfactory calligraphy fonts, which do not meet the consistency assumptions in prior knowledge.…”

Section: Glyph Image Synthesismentioning

confidence: 99%

DeepCalliFont: Few-Shot Chinese Calligraphy Font Synthesis by Integrating Dual-Modality Generative Models

Liu,

Lian

2024

AAAI

View full text Add to dashboard Cite

Few-shot font generation, especially for Chinese calligraphy fonts, is a challenging and ongoing problem. With the help of prior knowledge that is mainly based on glyph consistency assumptions, some recently proposed methods can synthesize high-quality Chinese glyph images. However, glyphs in calligraphy font styles often do not meet these assumptions. To address this problem, we propose a novel model, DeepCalliFont, for few-shot Chinese calligraphy font synthesis by integrating dual-modality generative models. Specifically, the proposed model consists of image synthesis and sequence generation branches, generating consistent results via a dual-modality representation learning strategy. The two modalities (i.e., glyph images and writing sequences) are properly integrated using a feature recombination module and a rasterization loss function. Furthermore, a new pre-training strategy is adopted to improve the performance by exploiting large amounts of uni-modality data. Both qualitative and quantitative experiments have been conducted to demonstrate the superiority of our method to other state-of-the-art approaches in the task of few-shot Chinese calligraphy font synthesis. The source code can be found at https://github.com/lsflyt-pku/DeepCalliFont.

show abstract

“…To improve the quality of the generated images, [13] proposed the Deep Feature Similarity (DFS) architecture to leverage the feature similarity between the input content and style images to synthesize target images. Recently, researchers [9,19,20,[44][45][46] have made significant progress by exploiting the compositionality of compositional scripts. However, our experimental results indicate poor performance for the constructed multi-language dataset.…”

Section: Font Generationmentioning

confidence: 99%

Cross-language font style transfer

Taniguchi

Min

et al. 2023

Appl Intell

View full text Add to dashboard Cite

In this paper, we propose a cross-language font style transfer system that can synthesize a new font by observing only a few samples from another language. Automatic font synthesis is a challenging task and has attracted much research interest. Most previous works addressed this problem by transferring the style of the given subset to the content of unseen ones. Nevertheless, they only focused on the font style transfer in the same language. In many cases, we need to learn font style from one language and then apply it to other languages. Existing methods make this difficult to accomplish because of the abstraction of style and language differences. To address this problem, we specifically designed the network into a multi-level attention form to capture both local and global features of the font style. To validate the generative ability of our model, we constructed an experimental font dataset of 847 fonts, each containing English and Chinese characters with the same style. Results show that our model generates 80.3% of users’ preferred images compared with state-of-the-art models.

show abstract

Few-Shot Font Generation by Learning Fine-Grained Local Styles

Cited by 36 publications

References 20 publications

FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning

FontDiffuser: One-Shot Font Generation via Denoising Diffusion with Multi-Scale Content Aggregation and Style Contrastive Learning

DeepCalliFont: Few-Shot Chinese Calligraphy Font Synthesis by Integrating Dual-Modality Generative Models

Cross-language font style transfer

Contact Info

Product

Resources

About