Manifold-preserved GANs

Abstract

Generative Adversarial Networks (GANs) have been widely adopted in various fields. However, existing GANs generally are not able to preserve the manifold of data space, mainly due to the simple representation of discriminator for the real/generated data. To address such open challenges, this paper proposes Manifold-preserved GANs (MaF-GANs), which generalize Wasserstein GANs into high-dimensional form. Specifically, to improve the representation of data, the discriminator in MaF-GANs is designed to map data into a high-dimensional manifold. Furthermore, to stabilize the training of MaF-GANs, an operation with precise and universal solution for any K-Lipschitz continuity, called Topological Consistency is proposed. The effectiveness of the proposed method is justified by both theoretical analysis and empirical results. When adopting DCGAN as the backbone on CelebA (256×256), the proposed method achieved 12.43 FID, which outperforms the state-of-the-art model like Realness GAN (23.51 FID) by a large margin. Code will be made publicly available.

Combating Mode Collapse via Offline Manifold Entropy Estimation

“…Benefiting from DLLE and DIsoMap, our method, namely MaEM-GAN, maximizes the entropy in the well-learned embedding space to combat the mode collapse in GANs. Experimental results show that the proposed MaEM-GAN outperforms the recent advanced GAN method MaF-GAN (Liu et al 2021) on CelebA (9.13 vs. 12.43 in FID) and surpasses the recent state-of-the-art EBM (Geng et al 2021) • We propose a novel training pipeline to address the mode collapse issue in GANs, which effectively alleviates mode collapse without sacrificing the image quality of generated images.…”

“…The empirical and theoretical study pointed out that the learning objective of GANs might neglect an intractable entropy term for maximum likelihood, which plays a key point in the mode collapse and can be effectively tackled by high-dimensional GANs. Manifold-preserved GAN (Liu et al 2021) is the most similar work to ours; however, it neglects the entropy term.…”

“…By dynamically modeling the distance between the generated and real distributions, WGANs can reduce the risk for local minima and mitigate the mode collapse to a certain extent. More recently, Realness GANs (Xiangli et al 2020) and Manifoldpreserved GANs (Liu et al 2021) generalize GANs into a high-dimensional form by mapping the output of the discriminator into a vector to enhance the realness. Since the fidelity of the given image can be judged from different views like ensemble learning, the generator can synthesize images with different attributes to fit the discriminator.…”

“…The above works indicate that the discriminator is critical for GANs and can be a potential target to combat mode collapse. Motivated by these progresses (AZhang et al 2020;Xiangli et al 2020;Liu et al 2021) with regard to the discriminator, we propose a new pipeline, focusing on designing the simple yet effective constraints on the discriminator side to prevent the mode collapse. Different from the previous methods, we explore how to maximize the entropy of distribution in the embedding space supported by the discriminator, and how to well learn the embedding space.…”

Robust Representation via Dynamic Feature Aggregation

“…The recent advances in convolutional neural networks (CNNs) have yielded significant improvements to various computer vision tasks, such as image classification [14,31,39] and image generation [10,19,30]. However, recent studies [16,38] revealed that CNN based models are vulnerable to the adversarial attacks and out-of-distribution samples, which seriously limits their applications in security-critical scenarios, e.g., self-driving [40], medical imaging [33] and biometrics [28,29].…”