On the Complexity of the Hybrid Proximal Extragradient Method for the Iterates and the Ergodic Mean

Abstract

In this paper we analyze the iteration complexity of the hybrid proximal extragradient (HPE) method for finding a zero of a maximal monotone operator recently proposed by Solodov and Svaiter. One of the key points of our analysis is the use of new termination criteria based on the ε-enlargement of a maximal monotone operator. The advantage of using these termination criteria is that their definition do not depend on the boundedness of the domain of the operator. We then show that Korpelevich's extragradient method for solving monotone variational inequalities falls in the framework of the HPE method. As a consequence, using the complexity analysis of the HPE method, we obtain new complexity bounds for Korpelevich's extragradient method which do not require the feasible set to be bounded, as assumed in a recent paper by Nemirovski. Another feature of our analysis is that the derived iteration-complexity bounds are proportional to the distance of the initial point to the solution set. The HPE framework is also used to obtain the first iterationcomplexity result for Tseng's modified forward-backward splitting method for finding a zero of the sum of a monotone Lipschitz continuous map with an arbitrary maximal monotone operator whose resolvent is assumed to be easily computable. Also using the framework of the HPE method, we study the complexity of a variant of a Newton-type extragradient algorithm proposed by Solodov and Svaiter for finding a zero of a smooth monotone function with Lipschitz continuous Jacobian.

An Accelerated Hybrid Proximal Extragradient Method for Convex Optimization and Its Implications to Second-Order Methods

“…We will now apply the theory outlined in Sections 3 and 4 to analyze an accelerated Newton proximal extragradient (A-NPE) method, which is an accelerated version of the method presented in [10] for solving a monotone nonlinear equation.…”

“…In [2], Burachik, Iusem and Svaiter introduced the ε-enlargement of maximal monotone operators. In [10] this concept was extended to a generic point-to-set operator in E as follows. Given T : E ⇒ E and a scalar ε, define T ε : E ⇒ E as…”

“…Iteration complexity results for one of these inexact versions of the proximal point method introduced in [18,19], namely the hybrid proximal extragradient (HPE) method, were established in [10]. Application of this framework in the iteration-complexity analysis of several zero-order (or, in the context of optimization, first-order) methods for solving monotone variational inequalities, and monotone inclusion and saddle-point problems, are discussed in [10] and in the subsequent papers [8,9].…”

“…The HPE framework was also used to study the iteration-complexity of a first-order (or, in the context of optimization, second-order) method for solving monotone nonlinear equations (see [10]) and, more generally, for monotone smooth variational inequalities and inclusion problems consisting of the sum of a smooth monotone map and a maximal monotone point-to-set operator (see [11]). …”

“…The purpose of this paper is to present an accelerated variant of the HPE method for convex optimization (based on a relative error condition), which we refer to as the accelerated HPE (A-HPE) framework. This framework builds on the ideas introduced in [10,18,19,12]. Iteration-complexity results are established for the A-HPE method, as well as a special version of it, where a large stepsize condition is imposed.…”