LEAD: Least-Action Dynamics for Min-Max Optimization

Hemmat, Reyhane Askari; Mitra, Amartya; Lajoie, Guillaume; Mitliagkas, Ioannis

doi:10.48550/arxiv.2010.13846

Search citation statements

Order By: Relevance

Paper Sections

Select...

Related Work1

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2021

Publication Types

Select...

Other1

Relationship

Self Cite0

Independent1

Authors

Journals

Cited by 1 publication

(1 citation statement)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Sometimes response gradients can be unstable, motivating methods like Stable Opponent Shaping (Letcher et al, 2019). Various works use higher-order information to help games, or approximate a large matrix inversion (Hemmat et al, 2020;Wang et al, 2019; Schäfer & Anandkumar,…”

Section: Related Workmentioning

confidence: 99%

Complex Momentum for Optimization in Games

Lorraine,

Acuna,

Vicol

et al. 2021

Preprint

View full text Add to dashboard Cite

We generalize gradient descent with momentum for learning in differentiable games to have complex-valued momentum. We give theoretical motivation for our method by proving convergence on bilinear zero-sum games for simultaneous and alternating updates. Our method gives real-valued parameter updates, making it a dropin replacement for standard optimizers. We empirically demonstrate that complex-valued momentum can improve convergence in adversarial games-like generative adversarial networks-by showing we can find better solutions with an almost identical computational cost. We also show a practical generalization to a complex-valued Adam variant, which we use to train BigGAN to better inception scores on CIFAR-10.

show abstract

Section: Related Workmentioning

confidence: 99%