The Douglas–Rachford algorithm for convex and nonconvex feasibility problems

Artacho, Francisco J. Aragón; Campoy, Rubén; Tam, Matthew K.

doi:10.1007/s00186-019-00691-9

Cited by 32 publications

(8 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While the convergence of DR for convex problems is well known, the method also solves many nonconvex problems. In addition to the survey of Lindstrom and Sims [35], we refer the interested reader to the excellent survey of Aragón Artacho, Campoy, and Tam [2]. Li and Pong have also provided some local convergence guarantees for the more general optimization problem (1) in [32].…”

Section: The Douglas-rachford Operator and Methodsmentioning

confidence: 99%

Computable Centering Methods for Spiraling Algorithms and their Duals, with Motivations from the theory of Lyapunov Functions

Lindstrom¹

2020

Preprint

View full text Add to dashboard Cite

Splitting methods like Douglas-Rachford (DR), ADMM, and FISTA solve problems whose objectives are sums of functions that may be evaluated separately, and all frequently show signs of spiraling. Circumcentering reflection methods (CRMs) have been shown to obviate spiraling for DR for certain feasibility problems. Under conditions thought to typify local convergence for splitting methods, we show that Lyapunov functions generically exist. We then show for prototypical feasibility problems that CRMs, subgradient descent, and Newton-Raphson are all describable as gradient-based methods for minimizing Lyapunov functions constructed for DR operators. Motivated thereby, we introduce a centering method that shares this property but with the added advantages that it: 1) does not rely on subproblems (e.g. reflections) and so may be applied for any operator whose iterates spiral; 2) provably has the aforementioned Lyapunov properties with few structural assumptions and so is generically suitable for primal/dual implementation; and 3) maps spaces of reduced dimension into themselves whenever the original operator does. We then describe a general approach to primal/dual implementation and provide, as an example (basis pursuit), the first such application of centering. The new centering operator we introduce works well, while a similar primal/dual adaptation of CRM does not, for reasons we explain.

show abstract

Section: The Douglas-rachford Operator and Methodsmentioning

confidence: 99%

Computable Centering Methods for Spiraling Algorithms and their Duals, with Motivations from the theory of Lyapunov Functions

Lindstrom¹

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…Since ω ∈ (0, 1) implies that β 2 ( 1+β 2 ) ∈ (0, 1), it follows immediately from (3.26) that the scalar sequence (dist(z k , X ∩ Y )) k∈N converges q-linearly to 0 with asymptotic constant bounded above by β 2 1+β 2 . Now, the operator T is the composition of the operators P X , P Y , 1 2 (Id +P X ) and P S X ∩S Y , taking into account Lemma 2.3. All these operators are nonexpansive and their sets of fixed points contain X ∩ Y .…”

Section: Linear Convergence Of Ccrm Under Ebmentioning

confidence: 99%

“…In fact, z C := 1 2 (z MAP + P X (z MAP )) , (1.4) is a centralized point, that is, it satisfies R X (z C ) − z C , R Y (z C ) − z C ≤ 0, where •, • stands for the Euclidean inner product and z MAP := P Y P X (z). A pCRM step (1.3) represents an acceleration of the Simultaneous Projections Method (SPM), also called Cimmino's method [22], given by z k+1 SPM = T SPM (z k ) := 1 2 P X (z k ) + P Y (z k ) , (1.5) known to converge to a point in X ∩ Y whenever X ∩ Y = ∅. We mention, parenthetically, that this method is devised for several convex sets with different weights in the average of the projections; (1.5) corresponds to the case of two sets with equal weights.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

On the linear convergence of the circumcentered-reflection method

Behling

Bello-Cruz

Santos

2018

Operations Research Letters

View full text Add to dashboard Cite

In order to accelerate the Douglas-Rachford method we recently developed the circumcentered-reflection method, which provides the closest iterate to the solution among all points relying on successive reflections, for the best approximation problem related to two affine subspaces. We now prove that this is still the case when considering a family of finitely many affine subspaces. This property yields linear convergence and incites embedding of circumcenters within classical reflection and projection based methods for more general feasibility problems.

show abstract

“…For a more comprehensive overview of the history, including the broader context of DR as a splitting method for solving optimization problems, see for example, [31]. For more on the use of DR for solving both nonconvex and convex feasibility problems, see also [8]. The aforementioned seminal work of Elser and Gravel [28] piqued the interests of Borwein and Sims, who in 2011 made the first rigorous attempt at analysing the behaviour of DR in the nonconvex setting of hypersurfaces [17].…”

Section: Introductionmentioning

confidence: 99%

Circumcentering Reflection Methods for Nonconvex Feasibility Problems

Dizon¹,

Hogan²,

Lindstrom³

2019

Preprint

View full text Add to dashboard Cite

The Douglas-Rachford method (DR) and its product space variant are often employed as iterated maps for solving the feasibility problem of the form: Find x ∈ N k=1 S k . The sets S k typically represent constraints that are easy to satisfy individually, but more challenging when imposed together. When the constraints under consideration are modeled by closed, convex, nonempty sets, convergence is well-understood. The method also demonstrates surprising performance with nonconvex sets. Recently, the method of circumcentering reflections has been introduced, with the aim of accelerating convergence of averaged reflection methods like DR in the convex setting of hypersurfaces. We introduce a generalization, GCR, that is amenable to employment when the circumcentering reflections operator fails to be proper. We prove local convergence for certain plane curves together with lines, the natural prototypical setting of most theoretical analysis of regular nonconvex DR. In particular, for GCR, we demonstrate local convergence to feasible points in cases where DR only converges to fixed points. For those cases where DR is proven to converge to a feasible point, we show that GCR provides a better convergence rate. Finally, as a root finder, we show that GCR has local convergence whenever Newton-Raphson does, exhibits quadratic convergence whenever Newton-Raphson does so, and exhibits superlinear convergence in many cases where Newton-Raphson fails to converge at all. Motivated by our theoretical results, we introduce a new 2 stage DR-GCR search algorithm, and we apply it to wavelet construction recast as a feasibility problem, demonstrating its acceleration over regular DR.

show abstract

The Douglas–Rachford algorithm for convex and nonconvex feasibility problems

Cited by 32 publications

References 55 publications

Computable Centering Methods for Spiraling Algorithms and their Duals, with Motivations from the theory of Lyapunov Functions

Computable Centering Methods for Spiraling Algorithms and their Duals, with Motivations from the theory of Lyapunov Functions

On the linear convergence of the circumcentered-reflection method

Circumcentering Reflection Methods for Nonconvex Feasibility Problems

Contact Info

Product

Resources

About