Dynamic Weighted Semantic Correspondence for Few-Shot Image Generative Adaptation

Hou, Xingzhong; Liu, Boxiao; Zhang, Shuai; Shi, Lulin; Jiang, Zite; You, Haihang

doi:10.1145/3503161.3548270

Cited by 3 publications

(4 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We detail the experimental settings in Section 4. (Zhao et al 2022b), RSSA (Xiao et al 2022), and DWSC (Hou et al 2022). We also include the latest finetuning-based methods AdAM (Zhao et al 2022a) and RICK (Zhao et al 2023).…”

Section: Methodsmentioning

confidence: 99%

“…DCL (Zhao et al 2022b) proposed contrastive losses between source/target features of both generator and discriminator. DWSC (Hou et al 2022) designed perceptual/contextual loss respectively for easy/hard-to-generate patches. These methods kept the characteristics of the source domain by imposing strong regularization thus inherited the diversity from the source dataset.…”

Section: Related Workmentioning

confidence: 99%

“…Finetuning-based methods reduce the number of trainable parameters trying to alleviate the overfitting issue (Wang et al 2018;Noguchi and Harada 2019;Robb et al 2020;Mo, Cho, and ), yet the effects are usually limited. Other methods propose regularization terms imposing penalties on feature/parameter changes during the transfer processes Ojha et al 2021;Xiao et al 2022;Zhao et al 2022b;Hou et al 2022). By minimally updating the model, these methods seek to keep some characteristics of the source images, hence inherit the diversity of the source domain.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

WeditGAN: Few-Shot Image Generation via Latent Space Relocation

Duan,

Niu,

Hong

et al. 2024

AAAI

View full text Add to dashboard Cite

In few-shot image generation, directly training GAN models on just a handful of images faces the risk of overfitting. A popular solution is to transfer the models pretrained on large source domains to small target ones. In this work, we introduce WeditGAN, which realizes model transfer by editing the intermediate latent codes w in StyleGANs with learned constant offsets (delta w), discovering and constructing target latent spaces via simply relocating the distribution of source latent spaces. The established one-to-one mapping between latent spaces can naturally prevents mode collapse and overfitting. Besides, we also propose variants of WeditGAN to further enhance the relocation process by regularizing the direction or finetuning the intensity of delta w. Experiments on a collection of widely used source/target datasets manifest the capability of WeditGAN in generating realistic and diverse images, which is simple yet highly effective in the research area of few-shot image generation. Codes are available at https://github.com/Ldhlwh/WeditGAN.

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

WeditGAN: Few-Shot Image Generation via Latent Space Relocation

Duan,

Niu,

Hong

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…These include scale and shift parameters [32], updating only the higher discriminator layers [28], linear combinations of scale and shift parameters [41], modulating kernels or convolutions [59,58,10,2] and singular values [38], mapping networks from noise to latents [46,29,53] and latent offsets [12]. Various works apply regularization losses by enforcing constraints to samples/weights by the source generator including elastic weight regularization [27], domain correspondence [33,16,22], contrastive learning [60], spatial alignment [51], inversion [49,23,44], random masks on discriminators [61] and alignment free spatial correlation [30]. Given the increasing popularity of VQ-VAE and diffusion based models, recent work [43] and [61] explore few-shot finetuning on VQ-VAE tokens and diffusion models.…”

Section: Generative Transfermentioning

confidence: 99%

A nonlinear anisotropic diffusion model with forward-backward diffusivities for image denoising

Kumar¹,

Alam²,

Kumar³

2021

Smart Computing

View full text Add to dashboard Cite

We investigate the task of adapting image generative models to different datasets without finetuneing. To this end, we introduce Semantica, an image-conditioned diffusion model capable of generating images based on the semantics of a conditioning image. Semantica is trained exclusively on web-scale image pairs, that is it receives a random image from a webpage as conditional input and models another random image from the same webpage. Our experiments highlight the expressivity of pretrained image encoders and necessity of semantic-based data filtering in achieving high-quality image generation. Once trained, it can adaptively generate new images from a dataset by simply using images from that dataset as input. We study the transfer properties of Semantica on ImageNet, LSUN Churches, LSUN Bedroom and SUN-397.

show abstract

Frequency-Auxiliary One-Shot Domain Adaptation of Generative Adversarial Networks

Cheng,

Liu,

Liu

et al. 2024

Electronics

View full text Add to dashboard Cite

Generative domain adaptation in a one-shot scenario involves transferring a pretrained generator from one domain to another using only a single reference image. To address the issue of extremely scarce data, existing methods resort to complex parameter constraints and leverage additional semantic knowledge from CLIP models to mitigate it. However, these methods still suffer from overfitting and underfitting issues due to the lack of prior knowledge about the domain adaptation task. In this paper, we firstly introduce the perspective of the frequency domain into the generative domain adaptation task to support the model in understanding the adaptation goals in a one-shot scenario and propose a method called frequency-auxiliary GAN (FAGAN). The FAGAN contains two core modules: a low-frequency fusion module (LFF-Module) and high-frequency guide module (HFG-Module). Specifically, the LFF-Module aims to inherit the domain-sharing information of the source module by fusing the low-frequency features of the source model. In addition, the HFG-Module is designed to select the domain-specific information of the reference image and guide the model to fit them by utilizing high-frequency guidance. These two modules are dedicated to alleviating overfitting and underfitting issues, thereby enchancing the diversity and fidelity of generated images. Extensive experimental results showed that our method leads to better quantitative and qualitative results than the existing methods under a wide range of task settings.

show abstract

Dynamic Weighted Semantic Correspondence for Few-Shot Image Generative Adaptation

Cited by 3 publications

References 15 publications

WeditGAN: Few-Shot Image Generation via Latent Space Relocation

WeditGAN: Few-Shot Image Generation via Latent Space Relocation

A nonlinear anisotropic diffusion model with forward-backward diffusivities for image denoising

Frequency-Auxiliary One-Shot Domain Adaptation of Generative Adversarial Networks

Contact Info

Product

Resources

About