A Closer Look at Few-shot Image Generation

Zhao, Yunqing; Ding, Henghui; Huang, Houjing; Cheung, Ngai-Man

doi:10.1109/cvpr52688.2022.00893

Cited by 40 publications

(33 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DCL [19] Preservation of multilevel semantic diversity of the generated images by the source generative model pre-trained on the source domain.…”

Section: Methodsmentioning

confidence: 99%

“…As only very limited samples are provided to define the underlying distribution, standard fine-tuning of a pre-trained GAN suffers from mode collapse: the adapted model can only generate samples closely resembling the given few shot target samples [16,14]. Therefore, recent works [18,14,19] have proposed to augment standard fine-tuning with different criteria to carefully preserve subset of source model's knowledge into the adapted model. Various criteria has been proposed (Table 1), and these knowledge preserving criteria have been central in recent FSIG research.…”

Section: Adam (Our Work)mentioning

confidence: 99%

“…These target/adaptation-agnostic knowledge preserving criteria in recent works raise question regarding their suitability in different source/target domain setups. It should be noted that existing FSIG works (under very limited target samples) focus largely on setups where source and target domains are in close proximity (semantically) e.g., Human faces (FFHQ)ÑBaby faces [14,19], or CarsÑAbandoned Cars [14,19]. It is unclear about their performance when source/target domains are more apart (e.g., Human faces (FFHQ) Ñ Animal faces [5]).…”

Section: Adam (Our Work)mentioning

confidence: 99%

“…Conventional few-shot learning [24][25][26] aims at learning a discriminative classifier for classification [27][28][29][30], segmentation [31,32] or detection [33][34][35] tasks. Differently, few-shot image generation (FSIG) [14,18,19] aims at learning a generator for new and diverse samples given extremely limited samples (e.g., 10 shots). Transfer learning has been applied to FSIG.…”

Section: Related Workmentioning

confidence: 99%

“…Cross-domain Correspondence (CDC) [14] preserves pair-wise distance of generated images from the source model to alleviate mode collapse. Dual Contrastive Learning (DCL) [19] applies mutual information maximization to preserve multi-level diversity of the generated images by the source model. In this work, we observe that these SOTA methods perform poorly when source and target domains are more apart.…”

Section: Related Workmentioning

confidence: 99%

See 4 more Smart Citations

Few-shot Image Generation via Adaptation-Aware Kernel Modulation

Zhao¹,

Keshigeyan²,

Abdollahzadeh³

et al. 2022

Preprint

View full text Add to dashboard Cite

Few-shot image generation (FSIG) aims to learn to generate new and diverse samples given an extremely limited number of samples from a domain, e.g., 10 training samples. Recent work has addressed the problem using transfer learning approach, leveraging a GAN pretrained on a large-scale source domain dataset and adapting that model to the target domain based on very limited target domain samples. Central to recent FSIG methods are knowledge preserving criteria, which aim to select a subset of source model's knowledge to be preserved into the adapted model. However, a major limitation of existing methods is that their knowledge preserving criteria consider only source domain/source task, and they fail to consider target domain/adaptation task in selecting source model's knowledge, casting doubt on their suitability for setups of different proximity between source and target domain.Our work makes two contributions. As our first contribution, we revisit recent FSIG works and their experiments. Our important finding is that, under setups which assumption of close proximity between source and target domains is relaxed, existing state-of-the-art (SOTA) methods which consider only source domain in knowledge preserving perform no better than a baseline fine-tuning method. To address the limitation of existing methods, as our second contribution, we propose Adaptation-Aware kernel Modulation (AdAM) to address general FSIG of different source-target domain proximity. Extensive experimental results show that the proposed method consistently achieves SOTA performance across source/target domains of different proximity, including challenging setups when source and target domains are more apart. Project Page: https://yunqing-me.github.io/AdAM/ * Equal Contribution : Corresponding Author Preprint. Under review.

show abstract

“…DCL [19] Preservation of multilevel semantic diversity of the generated images by the source generative model pre-trained on the source domain.…”

Section: Methodsmentioning

confidence: 99%

Section: Adam (Our Work)mentioning

confidence: 99%

Section: Adam (Our Work)mentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

Few-shot Image Generation via Adaptation-Aware Kernel Modulation

Zhao¹,

Keshigeyan²,

Abdollahzadeh³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

Discovering Transferable Forensic Features for CNN-Generated Images Detection

Keshigeyan¹,

Tran²,

Binder³

et al. 2022

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Unifying Conditional and Unconditional Semantic Image Synthesis with OCO-GAN

Marlène

Lathuilière

Verbeek³

2023

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Semantic image synthesis aims to generate photo realistic images given a semantic segmentation map. Despite much recent progress, training them still requires large datasets of images annotated with per-pixel label maps that are extremely tedious to obtain. To alleviate the high annotation cost, we propose a transfer method that leverages a model trained on a large source dataset to improve the learning ability on small target datasets via estimated pairwise relations between source and target classes. The class affinity matrix is introduced as a first layer to the source model to make it compatible with the target label maps, and the source model is then further finetuned for the target domain. To estimate the class affinities we consider different approaches to leverage prior knowledge: semantic segmentation on the source domain, textual label embeddings, and self-supervised vision features. We apply our approach to GAN-based and diffusion-based architectures for semantic synthesis. Our experiments show that the different ways to estimate class affinity can be effectively combined, and that our approach significantly improves over existing state-ofthe-art transfer approaches for generative image models.

show abstract

A Closer Look at Few-shot Image Generation

Cited by 40 publications

References 31 publications

Few-shot Image Generation via Adaptation-Aware Kernel Modulation

Few-shot Image Generation via Adaptation-Aware Kernel Modulation

Discovering Transferable Forensic Features for CNN-Generated Images Detection

Unifying Conditional and Unconditional Semantic Image Synthesis with OCO-GAN

Contact Info

Product

Resources

About