Two Steps at a Time---Taking GAN Training in Stride with Tseng's Method

Böhm, Axel; Sedlmayer, Michael; Csetnek, Ernö Robert; Boţ, Radu Ioan

doi:10.1137/21m1420939

Cited by 11 publications

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Then we intend to combine this technique with Tseng's algorithm and call it Tseng's algorithm with extrapolation from the past. We obtain a general scheme as (see [4])…”

Section: Introductionmentioning

confidence: 99%

Tseng's Algorithm with Extrapolation from the Past Endowed with Variable Metrics and Error Terms

Tongnoi¹

2022

Preprint

View full text Add to dashboard Cite

In this paper, we propose a variable metric version of Tseng's algorithm (the forward-backward-forward algorithm: FBF) combined with extrapolation from the past that includes error terms for finding a zero of the sum of a maximally monotone operator and a monotone Lipschitzian operator in Hilbert spaces. This can be seen as the optimistic gradient descent ascent (OGDA) algorithm endowed with variable metrics and error terms. Primal-dual algorithms are also proposed for monotone inclusion problems involving compositions with linear operators. The primal-dual problem occurring in image deblurring demonstrates an application of our theoretical results.

show abstract

“…Then we intend to combine this technique with Tseng's algorithm and call it Tseng's algorithm with extrapolation from the past. We obtain a general scheme as (see [4])…”

Section: Introductionmentioning

confidence: 99%

Tseng's Algorithm with Extrapolation from the Past Endowed with Variable Metrics and Error Terms

Tongnoi¹

2022

Preprint

View full text Add to dashboard Cite

show abstract

Fast Optimistic Gradient Descent Ascent (OGDA) Method in Continuous and Discrete Time

Boţ,

Csetnek,

Nguyen

2023

Found Comput Math

View full text Add to dashboard Cite

In the framework of real Hilbert spaces, we study continuous in time dynamics as well as numerical algorithms for the problem of approaching the set of zeros of a single-valued monotone and continuous operator V. The starting point of our investigations is a second-order dynamical system that combines a vanishing damping term with the time derivative of V along the trajectory, which can be seen as an analogous of the Hessian-driven damping in case the operator is originating from a potential. Our method exhibits fast convergence rates of order $$o \left( \frac{1}{t\beta (t)} \right) $$ o 1 t β ( t ) for $$\Vert V(z(t))\Vert $$ ‖ V ( z ( t ) ) ‖ , where $$z(\cdot )$$ z ( · ) denotes the generated trajectory and $$\beta (\cdot )$$ β ( · ) is a positive nondecreasing function satisfying a growth condition, and also for the restricted gap function, which is a measure of optimality for variational inequalities. We also prove the weak convergence of the trajectory to a zero of V. Temporal discretizations of the dynamical system generate implicit and explicit numerical algorithms, which can be both seen as accelerated versions of the Optimistic Gradient Descent Ascent (OGDA) method for monotone operators, for which we prove that the generated sequence of iterates $$(z_k)_{k \ge 0}$$ ( z k ) k ≥ 0 shares the asymptotic features of the continuous dynamics. In particular we show for the implicit numerical algorithm convergence rates of order $$o \left( \frac{1}{k\beta _k} \right) $$ o 1 k β k for $$\Vert V(z^k)\Vert $$ ‖ V ( z k ) ‖ and the restricted gap function, where $$(\beta _k)_{k \ge 0}$$ ( β k ) k ≥ 0 is a positive nondecreasing sequence satisfying a growth condition. For the explicit numerical algorithm, we show by additionally assuming that the operator V is Lipschitz continuous convergence rates of order $$o \left( \frac{1}{k} \right) $$ o 1 k for $$\Vert V(z^k)\Vert $$ ‖ V ( z k ) ‖ and the restricted gap function. All convergence rate statements are last iterate convergence results; in addition to these, we prove for both algorithms the convergence of the iterates to a zero of V. To our knowledge, our study exhibits the best-known convergence rate results for monotone equations. Numerical experiments indicate the overwhelming superiority of our explicit numerical algorithm over other methods designed to solve monotone equations governed by monotone and Lipschitz continuous operators.

show abstract

Variable sample-size optimistic mirror descent algorithm for stochastic mixed variational inequalities

Yang,

Zhao,

Lin

2023

J Glob Optim

View full text Add to dashboard Cite

Two Steps at a Time---Taking GAN Training in Stride with Tseng's Method

Cited by 11 publications

References 14 publications

Tseng's Algorithm with Extrapolation from the Past Endowed with Variable Metrics and Error Terms

Tseng's Algorithm with Extrapolation from the Past Endowed with Variable Metrics and Error Terms

Fast Optimistic Gradient Descent Ascent (OGDA) Method in Continuous and Discrete Time

Variable sample-size optimistic mirror descent algorithm for stochastic mixed variational inequalities

Contact Info

Product

Resources

About