A variant of forward-backward splitting method for the sum of two monotone operators with a new search strategy

Abstract

In this paper, we propose variants of Forward-Backward splitting method for finding a zero of the sum of two operators. A classical modification of ForwardBackward method was proposed by Tseng, which is known to converge when the forward and the backward operators are monotone and with Lipschitz continuity of the forward operator. The conceptual algorithm proposed here improves Tseng's method in some instances. The first and main part of our approach, contains an explicit Armijo-type search in the spirit of the extragradient-like methods for variational inequalities. During the iteration process, the search performs only one calculation of the forward-backward operator in each tentative of the step. This achieves a considerable computational saving when the forwardbackward operator is computationally expensive. The second part of the scheme consists in special projection steps. The convergence analysis of the proposed scheme is given assuming monotonicity on both operators, without Lipschitz continuity assumption on the forward operator.

Shadow Douglas–Rachford Splitting for Monotone Inclusions

“…Until recently, this was the only known method with these properties, however there has been progress in the area with the discovery of further methods having this property [16,17,20]. In this connection, see also [8,12].…”

A Forward-Backward Splitting Method for Monotone Inclusions Without Cocoercivity

“…Problems of this form naturally arise in machine learning, statistics, etc., where the dual (maximization) problem comes from either dualizing the constraints in the primal problem or from using the Fenchel-Legendre transform to leverage a nonsmooth composite part. Through its first-order optimality condition, the saddle point problem (6) can expressed as the monotone inclusion find x y ∈ H × H such that 0 0 ∈ ∂g(x) ∂f (y) + ∇ x Φ(x, y) −∇ y Φ(x, y) , (7) which is of the form specified by (1). By using the definitions of the respective subdifferentials, (7) can also be expressed in terms of the variational inequality (VI): find z * = (x * , y * ) ∈ H × H such that…”

“…For instance, apart from the trivial case when K = 0, the skew-symmetric operator in ( 4) is never cocoercive. Furthermore, without cocoercivity, convergence of (9) can only be guaranteed in the presence of similarly strong assumptions such as strong monotonicity of A + B [12], or at the cost of incorporating a backtracking strategy [6] (even when the Lipschitz constant is known).…”

A Projection Algorithm for Non-Monotone Variational Inequalities

“…These halfspaces (as well as their intersections) have been widely used in the literature, e.g., [3,5,7,32,40]. Now we describe the Algorithm.…”