Extragradient Method in Optimization: Convergence and Complexity

Abstract

We consider the extragradient method to minimize the sum of two functions, the first one being smooth and the second being convex. Under the Kurdyka-Lojasiewicz assumption, we prove that the sequence produced by the extragradient method converges to a critical point of the problem and has finite length. The analysis is extended to the case when both functions are convex. We provide, in this case, a sublinear convergence rate, as for gradient-based methods. Furthermore, we show that the recent small-prox complexity result can be applied to this method. Considering the extragradient method is an occasion to describe an exact line search scheme for proximal decomposition methods. We provide details for the implementation of this scheme for the one norm regularized least squares problem and demonstrate numerical results which suggest that combining nonaccelerated methods with exact line search can be a competitive choice.

Convergence Results of Forward-Backward Algorithms for Sum of Monotone Operators in Banach Spaces

“…• The minimization problem (35) in this section extends the problem studied in [8,15,34,50] and other related papers from Hilbert spaces to Banach spaces.…”

Probabilistic feasibility guarantees for solution sets to uncertain variational inequalities

“…1. Every solution is computed by means of a typical extragradient algorithm [33], initialized with a different condition and fixed step size, whose convergence is guaranteed as F is monotone and Lipschitz continuous with constant αN T , for any step size (0, α −1 /N T ). We would like to emphasize that, in this context, we are interested in computing a set of solution(s) to VI(X δ K , F ), and this motivates us to partially neglect the multi-agent nature of the problem addressed by adopting an extragradient method, eventually with a centralized structure.…”

Pursuing robust decisions in uncertain traffic equilibrium problems

“…Remark 1: We do not address the computation of network traffic equilibria, as it is per se an interesting question beyond the scope of the current paper -see, e.g., [10], [23]- [26].…”

“…2, we consider a traffic network digraph consisting of 22 nodes and 36 edges, with OD pairs specified in L = { (1,6), (1,7), (1,8), (2,5), (2,7), (2,8), (3,5), (3,6), (3,8), (4,5), (4,6), (4, 7)}, and hence = 12. By enumerating all Then, to numerically corroborate Lemma 2, we run an extragradient algorithm [26] with fixed step-size α = 0.3 10) and empirical violation probability of the set of agents' traffic equilibria for the randomized routing game (G, L, C, P ω K , ω K ). from 10 4 initial points, randomly sampled in P 0 , to estimate the "nominal" set of agents' traffic equilibria S ω0 , thus obtaining Sω0 .…”