On Duality Gap as a Measure for Monitoring GAN Training

Sidheekh, Sahil; Aimen, Aroof; Madan, Vineet; Krishnan, Narayanan C.

doi:10.1109/ijcnn52387.2021.9534186

Cited by 4 publications

(3 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…7(a) and 7(c) are due to the unstable training of GANs, which is a highly non-trivial non-convex non-concave minimax optimization. 50,51 Since all settings of dynamic PDGAN and original StyleGAN2/StyleGAN3 remain identical, except for the objective function Eq. ( 8) for the generator, it can be guaranteed that the performance gain is brought by the introduction of the pretrained teacher discriminator D and the sample-specific strategy.…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Dynamic PDGAN: discriminator-boosted knowledge distillation for StyleGANs

Tian,

Shen,

Tian

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

Generative adversarial networks have shown remarkable success in image synthesis, especially StyleGANs. Equipped with delicate and specific designs, StyleGANs are capable of synthesizing high-resolution and high-fidelity images. Previous works aiming at improving StyleGANs mainly focus on modifying the architecture of StyleGANs or transferring knowledge from other domains. However, the knowledge contained in StyleGANs trained in the same domain is still unexplored. We aim to further boost the performance of StyleGANs from the perspective of knowledge distillation, i.e., improving uncompressed StyleGANs with the aid of teacher StyleGANs trained in the same domain. Motivated by the implicit distribution contained in the pretrained teacher discriminator, we propose to exploit the teacher discriminator to additionally supervise the student generator of StyleGANs so as to leverage the knowledge in the teacher discriminator. With the proposed distillation scheme, our method can outperform original StyleGANs on several large-scale datasets, achieving state-of-the-art on AFHQv2.

show abstract

Section: Resultsmentioning

confidence: 99%

“…The spikes of FID curves in Figs. 7(a) and 7(c) are due to the unstable training of GANs, which is a highly non-trivial non-convex non-concave minimax optimization 50 , 51…”

Section: Resultsmentioning

confidence: 99%

Dynamic PDGAN: discriminator-boosted knowledge distillation for StyleGANs

Tian,

Shen,

Tian

et al. 2024

J. Electron. Imag.

View full text Add to dashboard Cite

show abstract

“…Note that duality gap is tailored towards GANs and primarily indicates a model's convergence or divergence, not the quality of generated samples [62]. Recently, Sidheekh et al [63,64] proposed two variants called perturbed and proximal duality gap, which are more accurate than the plain version, especially in cases where the two-player game need not converge to a Nash equilibrium for the generator to model P.…”

Section: Distribution Of Reconstruction Errors (Dre)mentioning

confidence: 99%

Evaluation is key: a survey on evaluation measures for synthetic time series

Stenger,

Leppich,

Foster

et al. 2024

J Big Data

View full text Add to dashboard Cite

Synthetic data generation describes the process of learning the underlying distribution of a given real dataset in a model, which is, in turn, sampled to produce new data objects still adhering to the original distribution. This approach often finds application where circumstances limit the availability or usability of real-world datasets, for instance, in health care due to privacy concerns. While image synthesis has received much attention in the past, time series are key for many practical (e.g., industrial) applications. To date, numerous different generative models and measures to evaluate time series syntheses have been proposed. However, regarding the defining features of high-quality synthetic time series and how to quantify quality, no consensus has yet been reached among researchers. Hence, we propose a comprehensive survey on evaluation measures for time series generation to assist users in evaluating synthetic time series. For one, we provide brief descriptions or - where applicable - precise definitions. Further, we order the measures in a taxonomy and examine applicability and usage. To assist in the selection of the most appropriate measures, we provide a concise guide for fast lookup. Notably, our findings reveal a lack of a universally accepted approach for an evaluation procedure, including the selection of appropriate measures. We believe this situation hinders progress and may even erode evaluation standards to a “do as you like”-approach to synthetic data evaluation. Therefore, this survey is a preliminary step to advance the field of synthetic data evaluation.

show abstract

Imitation learning by state-only distribution matching

Boborzi,

Straehle,

Buchner

et al. 2023

Appl Intell

View full text Add to dashboard Cite

Imitation Learning from observation describes policy learning in a similar way to human learning. An agent’s policy is trained by observing an expert performing a task. Although many state-only imitation learning approaches are based on adversarial imitation learning, one main drawback is that adversarial training is often unstable and lacks a reliable convergence estimator. If the true environment reward is unknown and cannot be used to select the best-performing model, this can result in bad real-world policy performance. We propose a non-adversarial learning-from-observations approach, together with an interpretable convergence and performance metric. Our training objective minimizes the Kulback-Leibler divergence (KLD) between the policy and expert state transition trajectories which can be optimized in a non-adversarial fashion. Such methods demonstrate improved robustness when learned density models guide the optimization. We further improve the sample efficiency by rewriting the KLD minimization as the Soft Actor Critic objective based on a modified reward using additional density models that estimate the environment’s forward and backward dynamics. Finally, we evaluate the effectiveness of our approach on well-known continuous control environments and show state-of-the-art performance while having a reliable performance estimator compared to several recent learning-from-observation methods.

show abstract

On Duality Gap as a Measure for Monitoring GAN Training

Cited by 4 publications

References 7 publications

Dynamic PDGAN: discriminator-boosted knowledge distillation for StyleGANs

Dynamic PDGAN: discriminator-boosted knowledge distillation for StyleGANs

Evaluation is key: a survey on evaluation measures for synthetic time series

Imitation learning by state-only distribution matching

Contact Info

Product

Resources

About