Some Theoretical Properties of GANs

Biau, Gérard; Cadre, Benoı̂t; Sangnier, Maxime; Tanielian, Ugo

doi:10.48550/arxiv.1803.07819

Cited by 6 publications

(6 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This idea was then used in several other problems in robust statistics, for instance in sparse functionals estimation [Du et al, 2017], clustering [Kothari et al, 2018], mixtures of spherical Gaussians learning [Diakonikolas et al, 2018a], and robust linear regression [Diakonikolas et al, 2018b]. [Gao et al, 2019] offers a different perspective on robust estimation and connects the robust normal mean estimation problem with Generative Adversarial Networks (GANs) [Goodfellow et al, 2014, Biau et al, 2018, what enables computing robust estimators using efficient tools developed for training GANs. Hence, the authors compute depth-like estimators that retain the same appealing robustness properties than Tukey's median and that can be trained using stochastic gradient descent (SGD) algorithms that were originally designed for GANs.…”

Section: Literaturementioning

confidence: 99%

Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence

Chérief-Abdellatif,

Alquier

2019

Preprint

View full text Add to dashboard Cite

Many works in statistics aim at designing a universal estimation procedure. This question is of major interest, in particular because it leads to robust estimators, a very hot topic in statistics and machine learning. In this paper, we tackle the problem of universal estimation using a minimum distance estimator presented in [Briol et al., 2019] based on the Maximum Mean Discrepancy. We show that the estimator is robust to both dependence and to the presence of outliers in the dataset. We also highlight the connections that may exist with minimum distance estimators using L2-distance. Finally, we provide a theoretical study of the stochastic gradient descent algorithm used to compute the estimator, and we support our findings with numerical simulations.

show abstract

Section: Literaturementioning

confidence: 99%

Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence

Chérief-Abdellatif,

Alquier

2019

Preprint

View full text Add to dashboard Cite

show abstract

“…On the theoretical side, [Aro+17; Sin+18; Lia17; BMR18; Fei+17] consider finite sample issues in GANs under different objective functions in various parametric and non-parametric settings, and provide bounds on their sample requirement. [Bia+18] provides asymptotic convergence bounds on GAN solutions when both generators and discriminators are unrestricted. [Bot+18] provide an analysis of the geometry of different GAN objective functions, with a view towards explaining their relative performance.…”

Section: Related Workmentioning

confidence: 99%

The Inductive Bias of Restricted f-GANs

Liu¹,

Chaudhuri²

2018

Preprint

View full text Add to dashboard Cite

Generative adversarial networks are a novel method for statistical inference that have achieved much empirical success; however, the factors contributing to this success remain ill-understood. In this work, we attempt to analyze generative adversarial learning -that is, statistical inference as the result of a game between a generator and a discriminator -with the view of understanding how it differs from classical statistical inference solutions such as maximum likelihood inference and the method of moments.Specifically, we provide a theoretical characterization of the distribution inferred by a simple form of generative adversarial learning called restricted f -GANs -where the discriminator is a function in a given function class, the distribution induced by the generator is restricted to lie in a pre-specified distribution class and the objective is similar to a variational form of the f -divergence. A consequence of our result is that for linear KL-GANs -that is, when the discriminator is a linear function over some feature space and f corresponds to the KL-divergence -the distribution induced by the optimal generator is neither the maximum likelihood nor the method of moments solution, but an interesting combination of both.

show abstract

“…It was shown in [3] that standard GAN losses of Jensen-Shannon divergence and Wasserstein distance both fail to generalize with a finite number of samples. On the other hand, more recent advances in analyzing GANs in [56,6,4] show promising generalization bounds by either assuming Lipschitz conditions on the generator model or by restricting the analysis to certain classes of distributions. Under those assumptions, where JS divergence generalizes, Theorem 1 justifies the use of the proposed RCGAN.…”

Section: Theoretical Analysis Of Rcganmentioning

confidence: 99%

Robustness of Conditional GANs to Noisy Labels

Thekumparampil,

Khetan,

Lin

et al. 2018

Preprint

View full text Add to dashboard Cite

We study the problem of learning conditional generators from noisy labeled samples, where the labels are corrupted by random noise. A standard training of conditional GANs will not only produce samples with wrong labels, but also generate poor quality samples. We consider two scenarios, depending on whether the noise model is known or not. When the distribution of the noise is known, we introduce a novel architecture which we call Robust Conditional GAN (RCGAN). The main idea is to corrupt the label of the generated sample before feeding to the adversarial discriminator, forcing the generator to produce samples with clean labels. This approach of passing through a matching noisy channel is justified by corresponding multiplicative approximation bounds between the loss of the RCGAN and the distance between the clean real distribution and the generator distribution. This shows that the proposed approach is robust, when used with a carefully chosen discriminator architecture, known as projection discriminator. When the distribution of the noise is not known, we provide an extension of our architecture, which we call RCGAN-U, that learns the noise model simultaneously while training the generator. We show experimentally on MNIST and CIFAR-10 datasets that both the approaches consistently improve upon baseline approaches, and RCGAN-U closely matches the performance of RCGAN.

show abstract

Some Theoretical Properties of GANs

Cited by 6 publications

References 5 publications

Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence

Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence

The Inductive Bias of Restricted f-GANs

Robustness of Conditional GANs to Noisy Labels

Contact Info

Product

Resources

About