Finding the Forward-Douglas–Rachford-Forward Method

Abstract

We consider the monotone inclusion problem with a sum of 3 operators, in which 2 are monotone and 1 is monotone-Lipschitz. The classical Douglas-Rachford and Forward-backward-forward methods respectively solve the monotone inclusion problem with a sum of 2 monotone operators and a sum of 1 monotone and 1 monotone-Lipschitz operators. We first present a method that naturally combines Douglas-Rachford and Forward-backwardforward and show that it solves the 3 operator problem under further assumptions, but fails in general. We then present a method that naturally combines Douglas-Rachford and forward-reflected-backward, a recently proposed alternative to Forward-backward-forward by Malitsky and Tam [arXiv:1808.04162, 2018]. We show that this second method solves the 3 operator problem generally, without further assumptions. IntroductionWe consider the monotone inclusion problem of finding a zero of the sum of 2 maximal monotone and 1 monotone-Lipschitz operators. The classical Communicated by Jalal Fadili. Recently, there has been much work developing splitting methods combining and unifying classical ones. Another classical method is forward-backward (FB) splitting [3,4], which solves the problem with a sum of 1 monotone and 1 cocoercive operators. The effort of combining DR and FB was started by Raguet, Fadili, and Peyré [5,6], extended by Briceño-Arias [7], and completed by Davis and Yin [8] as they proved convergence for the sum of 2 monotone and 1 cocoercive operators. FB and FBF were combined by Briceño-Arias and Davis [9] as they proved convergence for 1 monotone, 1 cocoercive, and 1 monotone-Lipschitz operators. These combined splitting methods can efficiently solve monotone inclusion problems with more complex structure.On the other hand, DR and FBF have not been fully combined, to the best of our knowledge. Banert's relaxed forward backward (in the thesis [10]) and Briceño-Arias's forward-partial inverse-forward [11] combine DR and FBF in the setup where one operator is a normal cone operator with respect to a closed subspace. However, neither method applies to the general setup with 2 maximal monotone and 1 monotone-Lipschitz operators.In this work, we first present a method that naturally combines and unifies DR and FBF. We prove convergence under further assumptions, and we prove, through a counterexample, that convergence cannot be established in full generality. We then propose a second method that naturally combines and unifies DR and forward-reflected-backward (FRB), a recently proposed alternative to FBF by Malitsky and Tam [12]. We show that this combination of DR and FRB does converge in full generality.The paper is organized as follows. Section 2 states the problem formally. Section 3 reviews preliminary information and sets up the notation. Section 4 presents our first proposed method combining DR and FBF, proves convergence under certain further assumptions, and proves divergence in the fully general case. Section 5 presents our second proposed method combining DR and FRB and proves convergen...

Relaxed Forward–Backward Splitting Methods for Solving Variational Inclusions and Applications

“…Several other methods for solving the problem (VI) can be found, for example, as the Douglas-Rachford splitting method (DRSM) [13], the forward-reflected-backward splitting method (FRBSM) [14] and others [6,7,9]. A general form of the problem (VI), which involves the sum of three operator, can be found in [3,10,14,18,20].…”

Yapa: Accelerated Proximal Algorithm for Convex Composite Problems

“…Their algorithms are however significantly different from Algorithm 1, and hence all worthy of investigation. Note finally that we base our algorithm on Davis-Yin splitting for its simplicity, although we could have chosen other three-operator splitting methods [18][19][20][21].…”

“…Their goal is to split the optimization process into elementary operations like gradient steps, "simple" proximity operators, and linear operators. Proximal algorithms can be classified in primal methods [12][13][14][15][16][17][18][19][20][21] and primal-dual methods [22][23][24][25]. The main difference is that the latter do not need to invert the linear operators involved in the optimization problem, even though the former may converge faster when such an inversion can be efficiently computed.…”

“…The proposed algorithm can activate smooth functions via their gradients, and allows for linear operators in nonsmooth functions. This distinguishes it from many other primal proximal methods, which can only activate smooth functions through their proximity operator [14,15], or do not allow for linear operators in nonsmooth functions [18][19][20][21]. Moreover, unlike primaldual proximal methods [23], the proposed algorithm only requires to set a single stepsize, and can be accelerated with great effect via Anderson extrapolation [26] (see Section 3).…”

A Direct Proof of Convergence of Davis–Yin Splitting Algorithm Allowing Larger Stepsizes

“…Only recently, three-operator splitting algorithms have been developed [18,21,[27][28][29]. This note is devoted to one of them, which was introduced by Damek Davis and Wotao Yin in [18], and is commonly referred as Davis-Yin splitting algorithm.…”