On the ergodic convergence rates of a first-order primal–dual algorithm

Abstract

We revisit the proofs of convergence for a first order primal-dual algorithm for convex optimization which we have studied a few years ago. In particular, we prove rates of convergence for a more general version, with simpler proofs and more complete results. The new results can deal with explicit terms and nonlinear proximity operators in spaces with quite general norms. MSC Classification: 49M29 65K10 65Y20 90C25

Techniques for Gradient-Based Bilevel Optimization with Non-smooth Lower Level Problems

“…Since the primal-dual algorithm with Bregman proximity functions from [9] provides us with a flexible tool, for n to 0 :…”

“…The saddle-point problem (23) can be solved using the ergodic primal-dual algorithm [9], which leads to an iterative algorithm with totally differentiable iterations. The primal update in (13) is discussed in Example 6 and the dual update of (13) is essentially Example 7.…”

Primal-Dual Extragradient Methods for Nonlinear Nonsmooth PDE-Constrained Optimization

“…In addition, Moreau-Yosida regularization results in a strongly convex functional, which can be exploited for accelerating Algorithm 1 as in [4] via adaptive step length and extrapolation parameters. This leads to the following iteration.…”

“…The appropriate choice for µ > 0 is related to the constant of strong convexity of F * , and in the convex case yields the optimal convergence rate of O(1/k 2 ) for the functional values rather than the rate O(1/k) for the original version; see [3,4,21]. A similar acceleration is possible if G is strongly convex by swapping the roles of σ i and τ i in line 4; we will refer to both variants as Algorithm 2 in the following.…”

Accelerated Alternating Descent Methods for Dykstra-Like Problems

“…Such problems arise for instance in the computation of the proximity operators of sums of simple functions, for which in some setting (as we illustrate in an experimental section) it might be beneficial to perform such a splitting which decomposes the problem into tiny parallel subproblems, rather than tackle the global problem by an accelerated descent or primal-dual algorithm such as [19,3,9,10].…”