How Does Lipschitz Regularization Influence GAN Training?

Qin, Yipeng; Mitra, Niloy J.; Wonka, Peter

doi:10.1007/978-3-030-58517-4_19

Cited by 16 publications

(9 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Instead, it is trained to learn a K -Lipschitz continuous function, which makes the neural network gradient smaller than a threshold value K , such that ||Δ f || ≤ K . The primary rationale for applying this condition is that gradient behaves better, making generator optimization easier(Qin et al, 2018). As the loss function decreases in training, the Wasserstein distance gets smaller, and the generator model’s output grows closer to the actual data distribution.…”

Section: Methodsmentioning

confidence: 99%

scAEGAN: Unification of Single-Cell Genomics Data by Adversarial Learning of Latent Space Correspondences

Khan

Lehman

Martinez-de-Morentin

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent progress in Single-Cell Genomics have produced different library protocols and techniques for profiling of one or more data modalities in individual cells. Machine learning methods have separately addressed specific integration challenges (libraries, samples, paired-unpaired data modalities). We formulate a unifying data-driven methodology addressing all these challenges. To this end, we design a hybrid architecture using an autoencoder (AE) network together with adversarial learning by a cycleGAN (cGAN) network, jointly referred to as scAEGAN. The AE learns a low-dimensional embedding of each condition, whereas the cGAN learns a non-linear mapping between the AE representations. The core insight is that the AE respects each sample’s uniqueness, whereas the cGAN exploits the distributional data similarity in the latent space. We evaluate scAEGAN using simulated data and real datasets of a single-modality (scRNA-seq), different library preparations (Fluidigm C1, CelSeq, CelSeq2, SmartSeq), and several data modalities such as paired scRNA-seq and scATAC-seq. We find that scAEGAN outperforms Seurat3 in library integration, is more robust against data sparsity, and beats Seurat 4 in integrating paired data from the same cell. Furthermore, in predicting one data modality from another, scAEGAN outperforms Babel. We conclude scAEGAN surpasses current state-of-the-art methods across several seemingly different integration challenges.

show abstract

Section: Methodsmentioning

confidence: 99%

scAEGAN: Unification of Single-Cell Genomics Data by Adversarial Learning of Latent Space Correspondences

Khan

Lehman

Martinez-de-Morentin

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Conversely, adversarial defenses either enforce smoothness around embedding space or on potentially perturbed inputs themselves (Das et al, 2018). Similarly, GANs can enforce 1-Lipschitzness to improve coverage of sample generation (Qin et al, 2020) .…”

Section: Definition (Lipshitznessmentioning

confidence: 99%

MiDAS: Multi-integrated Domain Adaptive Supervision for Fake News Detection

Suprem¹,

Pu²

2022

Preprint

View full text Add to dashboard Cite

Covid-19 related misinformation and fake news, coined an 'infodemic', has dramatically increased over the past few years. This misinformation exhibits concept drift, where the distribution of fake news changes over time, reducing effectiveness of previously trained models for fake news detection. Given a set of fake news models trained on multiple domains, we propose an adaptive decision module to select the best-fit model for a new sample. We propose MIDAS, a multi-domain adaptative approach for fake news detection that ranks relevancy of existing models to new samples. MIDAS contains 2 components: a doman-invariant encoder, and an adaptive model selector. MIDAS integrates multiple pre-trained and fine-tuned models with their training data to create a domain-invariant representation. Then, MIDAS uses local Lipschitz smoothness of the invariant embedding space to estimate each model's relevance to a new sample. Higher ranked models provide predictions, and lower ranked models abstain. We evaluate MIDAS on generalization to drifted data with 9 fake news datasets, each obtained from different domains and modalities. MIDAS achieves new state-of-the-art performance on multi-domain adaptation for out-of-distribution fake news classification.Preprint. Under review.

show abstract

“…Lipschitz regularization can be used to optimize a loss function to ensure that the learned function is Lipschitz continuous (smooth). It has been used to regularize the discriminator in GAN (Gulrajani et al 2017;Miyato et al 2018;Qin, Mitra, and Wonka 2020) and to defend against attacks in adversarial learning (Hein and Andriushchenko 2017). One way to constrain a function to be Lipschitz continuous is to penalize its gradient norm (Gulrajani et al 2017).…”

Section: Related Workmentioning

confidence: 99%

Unsupervised Anomaly Detection by Robust Density Estimation

Liu

Tan

Zhou

2022

AAAI

View full text Add to dashboard Cite

Density estimation is a widely used method to perform unsupervised anomaly detection. By learning the density function, data points with relatively low densities are classified as anomalies. Unfortunately, the presence of anomalies in training data may significantly impact the density estimation process, thereby imposing significant challenges to the use of more sophisticated density estimation methods such as those based on deep neural networks. In this work, we propose RobustRealNVP, a deep density estimation framework that enhances the robustness of flow-based density estimation methods, enabling their application to unsupervised anomaly detection. RobustRealNVP differs from existing flow-based models from two perspectives. First, RobustRealNVP discards data points with low estimated densities during optimization to prevent them from corrupting the density estimation process. Furthermore, it imposes Lipschitz regularization to the flow-based model to enforce smoothness in the estimated density function. We demonstrate the robustness of our algorithm against anomalies in training data from both theoretical and empirical perspectives. The results show that our algorithm achieves competitive results as compared to state-of-the-art unsupervised anomaly detection methods.

show abstract

How Does Lipschitz Regularization Influence GAN Training?

Cited by 16 publications

References 10 publications

scAEGAN: Unification of Single-Cell Genomics Data by Adversarial Learning of Latent Space Correspondences

scAEGAN: Unification of Single-Cell Genomics Data by Adversarial Learning of Latent Space Correspondences

MiDAS: Multi-integrated Domain Adaptive Supervision for Fake News Detection

Unsupervised Anomaly Detection by Robust Density Estimation

Contact Info

Product

Resources

About