Forward-Douglas–Rachford splitting and forward-partial inverse method for solving monotone inclusions

Briceño-Arias, Luis M.

doi:10.1080/02331934.2013.855210

Cited by 67 publications

(82 citation statements)

References 44 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The following theorem recalls several results on convergence rates for the iteration of averaged operators [16]. In addition, we show that (λ j ∇h(z j ) − ∇h(z * ) 2 ) j≥0 is a summable sequence [7] whenever (λ j ) j≥0 is chosen properly.…”

Section: Damek Davismentioning

confidence: 97%

“…Convergence properties of FDRS. The paper [7] assumed the step size constraint γ ∈ (0, 2β) in order to guarantee convergence of Algorithm 1. We now show that the parameter γ can (possibly) be increased beyond 2β, which can result in faster practical performance.…”

Section: The Fdrs Algorithmmentioning

confidence: 99%

“…Our analysis builds on the techniques and results of [7,16,17]. The rest of this section contains a brief review of these results.…”

Section: Introductionmentioning

confidence: 98%

See 2 more Smart Citations

Convergence Rate Analysis of the Forward-Douglas-Rachford Splitting Scheme

Davis¹

2015

SIAM J. Optim.

View full text Add to dashboard Cite

Operator splitting schemes are a class of powerful algorithms that solve complicated monotone inclusion and convex optimization problems that are built from many simpler pieces. They give rise to algorithms in which all simple pieces of the decomposition are processed individually. This leads to easily implementable and highly parallelizable or distributed algorithms, which often obtain nearly state-of-the-art performance. In this paper, we analyze the convergence rate of the forwardDouglas-Rachford splitting (FDRS) algorithm, which is a generalization of the forward-backward splitting (FBS) and Douglas-Rachford splitting algorithms. Under general convexity assumptions, we derive the ergodic and nonergodic convergence rates of the FDRS algorithm, and show that these rates are the best possible. Under Lipschitz differentiability assumptions, we show that the best iterate of FDRS converges as quickly as the last iterate of the FBS algorithm. Under strong convexity assumptions, we derive convergence rates for a sequence that strongly converges to a minimizer. Under strong convexity and Lipschitz differentiability assumptions, we show that FDRS converges linearly. We also provide examples where the objective is strongly convex, yet FDRS converges arbitrarily slowly. Finally, we relate the FDRS algorithm to a primal-dual FBS scheme and clarify its place among existing splitting methods. Our results show that the FDRS algorithm automatically adapts to the regularity of the objective functions and achieves rates that improve upon the sharp worst case rates that hold in the absence of smoothness and strong convexity. 1761The forward-backward splitting (FBS) algorithm [23] is another technique for solving (1.1) when g is known to be smooth. In this case, the proximal operator of g is never evaluated. Instead, FBS combines gradient (forward) steps with respect to g and proximal (backward) steps with respect to f . FBS is especially useful when the proximal operator of g is complex and its gradient is simple to compute.Recently, the forward-Douglas-Rachford splitting (FDRS) algorithm [7] was proposed to combine DRS and FBS and extend their applicability (see Algorithm 1). More specifically, let V ⊆ H be a closed vector space and suppose g is smooth. Then FDRS applies to the following constrained problem:Problem (1.5) arises in the dual form soft-margin kernelized support vector machine classifier [14] in which C is a box constraint, b is 0, and A has rank one. Note that by the argument in (1.3), we can always assume that b = 0. Define the smooth function g(x) := (1/2) Qx, x + c, x , the indicator function f (x) := χ C (x) (which is 0 on C and ∞ elsewhere), and the vector space V := {x ∈ R d | Ax = 0}. With this notation, (1.5) is in the form (1.2) and, thus, FDRS can be applied. This splitting is nice because ∇g(x) = Qx + c is simple whereas the proximal operator of g requires a matrix inversion prox γg = (I R d +γQ) −1 •(I R d −γc), which is expensive for large-scale problems. Goals, challenges, and approaches.This work se...

show abstract

Section: Damek Davismentioning

confidence: 97%

Section: The Fdrs Algorithmmentioning

confidence: 99%

See 1 more Smart Citation

Convergence Rate Analysis of the Forward-Douglas-Rachford Splitting Scheme

Davis¹

2015

SIAM J. Optim.

View full text Add to dashboard Cite

show abstract

“…Smoothness is better exploited by ForwardBackward schemes and Briceño-Arias has proposed a Forward-DRA scheme in [12], further improved by Davis in [25] (see too [82] or [18] for similar related algorithms). We will present first some algorithmic enhancements relative to convergence issues and, in the second part, discuss numerical scaling issues.…”

Section: Convergence Results and Complexity Issuesmentioning

confidence: 99%

A survey on operator splitting and decomposition of convex programs

Lenoir

Mahey

2016

RAIRO-Oper. Res.

View full text Add to dashboard Cite

To cite this version:Philippe Mahey, Arnaud Lenoir. A survey on operator splitting and decomposition of convex programs. RAIRO -Operations Research, EDP Sciences, 2017, 51 (1) Abstract Many structured convex minimization problems can be modeled by the search of a zero of the sum of two monotone operators. Operator splitting methods have been designed to decompose and regularize at the same time these kind of models. We review here these models and the classical splitting methods. We focus on the numerical sensitivity of these algorithms with respect to the scaling parameters that drive the regularizing terms, in order to accelerate convergence rates for different classes of models.

show abstract

“…Combettes and Pesquet's PPXA [13], Boţ-Hendrich [16], Latafat and Patrinos's AFBA [17], and Ryu's 3-operator resolvent-splitting [18] solve the problem with 3 or more monotone operators by activating the operators through their individual resolvents. Condat-Vũ [19,20], FDR [5,6,7,8], and Yan's PD3O [21] solve the problem with 2 monotone and 1 cocoercive operators by activating the 2 monotone operators through their resolvents and the cocoercive operator through forward evaluations. FBHF [9] solves the problem with 1 monotone, 1 cocoercive, and 1 monotone-Lipschitz operators by activating the monotone operator through its resolvent and the cocoercive and monotone-Lipschitz operators through forward evaluations.…”

Section: Douglas-rachford (Dr) Splitting By Lions and Merciermentioning

confidence: 99%

Finding the Forward-Douglas–Rachford-Forward Method

Ryu

Vũ

2019

J Optim Theory Appl

View full text Add to dashboard Cite

We consider the monotone inclusion problem with a sum of 3 operators, in which 2 are monotone and 1 is monotone-Lipschitz. The classical Douglas-Rachford and Forward-backward-forward methods respectively solve the monotone inclusion problem with a sum of 2 monotone operators and a sum of 1 monotone and 1 monotone-Lipschitz operators. We first present a method that naturally combines Douglas-Rachford and Forward-backwardforward and show that it solves the 3 operator problem under further assumptions, but fails in general. We then present a method that naturally combines Douglas-Rachford and forward-reflected-backward, a recently proposed alternative to Forward-backward-forward by Malitsky and Tam [arXiv:1808.04162, 2018]. We show that this second method solves the 3 operator problem generally, without further assumptions. IntroductionWe consider the monotone inclusion problem of finding a zero of the sum of 2 maximal monotone and 1 monotone-Lipschitz operators. The classical Communicated by Jalal Fadili. Recently, there has been much work developing splitting methods combining and unifying classical ones. Another classical method is forward-backward (FB) splitting [3,4], which solves the problem with a sum of 1 monotone and 1 cocoercive operators. The effort of combining DR and FB was started by Raguet, Fadili, and Peyré [5,6], extended by Briceño-Arias [7], and completed by Davis and Yin [8] as they proved convergence for the sum of 2 monotone and 1 cocoercive operators. FB and FBF were combined by Briceño-Arias and Davis [9] as they proved convergence for 1 monotone, 1 cocoercive, and 1 monotone-Lipschitz operators. These combined splitting methods can efficiently solve monotone inclusion problems with more complex structure.On the other hand, DR and FBF have not been fully combined, to the best of our knowledge. Banert's relaxed forward backward (in the thesis [10]) and Briceño-Arias's forward-partial inverse-forward [11] combine DR and FBF in the setup where one operator is a normal cone operator with respect to a closed subspace. However, neither method applies to the general setup with 2 maximal monotone and 1 monotone-Lipschitz operators.In this work, we first present a method that naturally combines and unifies DR and FBF. We prove convergence under further assumptions, and we prove, through a counterexample, that convergence cannot be established in full generality. We then propose a second method that naturally combines and unifies DR and forward-reflected-backward (FRB), a recently proposed alternative to FBF by Malitsky and Tam [12]. We show that this combination of DR and FRB does converge in full generality.The paper is organized as follows. Section 2 states the problem formally. Section 3 reviews preliminary information and sets up the notation. Section 4 presents our first proposed method combining DR and FBF, proves convergence under certain further assumptions, and proves divergence in the fully general case. Section 5 presents our second proposed method combining DR and FRB and proves convergen...

show abstract

Forward-Douglas–Rachford splitting and forward-partial inverse method for solving monotone inclusions

Cited by 67 publications

References 44 publications

Convergence Rate Analysis of the Forward-Douglas-Rachford Splitting Scheme

Convergence Rate Analysis of the Forward-Douglas-Rachford Splitting Scheme

A survey on operator splitting and decomposition of convex programs

Finding the Forward-Douglas–Rachford-Forward Method

Contact Info

Product

Resources

About