Systems of Structured Monotone Inclusions: Duality, Algorithms, and Applications

Combettes, Patrick L.

doi:10.1137/130904160

Cited by 64 publications

(82 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the complexity of Problem 2 can be increased in various ways, e.g., by precomposing each of h i and l i with linear operators [3,10] or by solving systems of such inclusions [17,8]. We choose to discuss this relatively simple formulation for clarity of exposition.…”

Section: Damek Davismentioning

confidence: 99%

“…In this paper, we are mainly concerned with the line of work that began in [41,15,25] and the many generalizations and enhancements of the basic framework that followed [19,22,46,12,9,10,17,31,6,18]. Thus, we consider the following prototypical convex optimization problem as our guiding example: (1.1) where denotes the infimal convolution operation (see section 1.2), n ∈ N, n ≥ 1, H i are Hilbert spaces for i = 0, .…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convergence Rate Analysis of Primal-Dual Splitting Schemes

Davis

2015

SIAM J. Optim.

View full text Add to dashboard Cite

Primal-dual splitting schemes are a class of powerful algorithms that solve complicated monotone inclusions and convex optimization problems that are built from many simpler pieces. They decompose problems that are built from sums, linear compositions, and infimal convolutions of simple functions so that each simple term is processed individually via proximal mappings, gradient mappings, and multiplications by the linear maps. This leads to easily implementable and highly parallelizable or distributed algorithms, which often obtain nearly state-of-the-art performance. In this paper, we analyze a monotone inclusion problem that captures a large class of primal-dual splittings as a special case. We introduce a unifying scheme and use some abstract analysis of the algorithm to prove convergence rates of the proximal point algorithm, forward-backward splitting, Peaceman-Rachford splitting, and forward-backward-forward splitting applied to the model problem. Our ergodic convergence rates are deduced under variable metrics, stepsizes, and relaxation parameters. Our nonergodic convergence rates are the first shown in the literature. Finally, we apply our results to a large class of primal-dual algorithms that are a special case of our scheme and deduce their convergence rates.higher order regularizations of the components in each group, a task which might otherwise be intractable in large-scale applications.Finally, we note that the use of infimal convolutions in applications is not widespread, so we list a few instances where they may be useful: Infimal convolutions are used in image recovery [14, section 5] to remove staircasing effects in the total variation model. The infimal convolution of the indicator functions of two closed convex sets is the indicator function of their Minkowski sum, which has applications in motion planning for robotics [32, section 4.3.2]. In convex analysis, the Moreau envelope of a function arises as an infimal convolution with a multiple of the squared norm [2, section 12.4]. More generally, the infimal convolution of h i and l i can be interpreted as a regularization or smoothing of h i by l i and vice versa [2, section 18.3].1.1. Goals, challenges, and approaches. This work seeks to improve the theoretical understanding of the convergence rates of primal-dual splitting schemes. In this paper, we study primal-dual algorithms that are applications of standard operator splitting algorithms in product spaces consisting of primal and dual variables. Consequently, the convergence theory for these algorithms is well-developed, and they are known to converge (weakly) under mild conditions.Although we understand when these algorithms converge, relatively little is known about their rate of convergence. For convex optimization algorithms, the ergodic convergence rate of the primal-dual gap has been analyzed in a few cases [15,7,6,43]. However, even in cases where convergence rates are known, variable metrics and stepsizes, which can significantly improve practical performance of the algorithms [40...

show abstract

Section: Damek Davismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Convergence Rate Analysis of Primal-Dual Splitting Schemes

Davis

2015

SIAM J. Optim.

View full text Add to dashboard Cite

show abstract

“…This problem arises in many applications, such as variational inequalities and equilibrium problems [29], signal and image processing [21,23], game theory [10], and statistical learning [25,27,39,50,60]. A necessarily incomplete list of related works include [8,9,11,12,15,14,22,24,43,55].…”

Section: Introductionmentioning

confidence: 99%

A First-Order Stochastic Primal-Dual Algorithm with Correction Step

Rosasco

Villa

Vũ

2017

Numerical Functional Analysis and Optimization

View full text Add to dashboard Cite

We investigate the convergence properties of a stochastic primal-dual splitting algorithm for solving structured monotone inclusions involving the sum of a cocoercive operator and a composite monotone operator. The proposed method is the stochastic extension to monotone inclusions of a proximal method studied in [26,35] for saddle point problems. It consists in a forward step determined by the stochastic evaluation of the cocoercive operator, a backward step in the dual variables involving the resolvent of the monotone operator, and an additional forward step using the stochastic evaluation of the cocoercive introduced in the first step. We prove weak almost sure convergence of the iterates by showing that the primal-dual sequence generated by the method is stochastic quasi Fejér-monotone with respect to the set of zeros of the considered primal and dual inclusions. Additional results on ergodic convergence in expectation are considered for the special case of saddle point models.

show abstract

“…Therefore, we restrict our attention to methods that activate C through direct evaluations, rather than through J γC . The monotone-Lipschitz operator C of Problem (1) arises as skew linear operators primal-dual optimization [13,14] and saddle point problems [15].…”

Section: Douglas-rachford (Dr) Splitting By Lions and Merciermentioning

confidence: 99%

Finding the Forward-Douglas–Rachford-Forward Method

Ryu

Vũ

2019

J Optim Theory Appl

View full text Add to dashboard Cite

We consider the monotone inclusion problem with a sum of 3 operators, in which 2 are monotone and 1 is monotone-Lipschitz. The classical Douglas-Rachford and Forward-backward-forward methods respectively solve the monotone inclusion problem with a sum of 2 monotone operators and a sum of 1 monotone and 1 monotone-Lipschitz operators. We first present a method that naturally combines Douglas-Rachford and Forward-backwardforward and show that it solves the 3 operator problem under further assumptions, but fails in general. We then present a method that naturally combines Douglas-Rachford and forward-reflected-backward, a recently proposed alternative to Forward-backward-forward by Malitsky and Tam [arXiv:1808.04162, 2018]. We show that this second method solves the 3 operator problem generally, without further assumptions. IntroductionWe consider the monotone inclusion problem of finding a zero of the sum of 2 maximal monotone and 1 monotone-Lipschitz operators. The classical Communicated by Jalal Fadili. Recently, there has been much work developing splitting methods combining and unifying classical ones. Another classical method is forward-backward (FB) splitting [3,4], which solves the problem with a sum of 1 monotone and 1 cocoercive operators. The effort of combining DR and FB was started by Raguet, Fadili, and Peyré [5,6], extended by Briceño-Arias [7], and completed by Davis and Yin [8] as they proved convergence for the sum of 2 monotone and 1 cocoercive operators. FB and FBF were combined by Briceño-Arias and Davis [9] as they proved convergence for 1 monotone, 1 cocoercive, and 1 monotone-Lipschitz operators. These combined splitting methods can efficiently solve monotone inclusion problems with more complex structure.On the other hand, DR and FBF have not been fully combined, to the best of our knowledge. Banert's relaxed forward backward (in the thesis [10]) and Briceño-Arias's forward-partial inverse-forward [11] combine DR and FBF in the setup where one operator is a normal cone operator with respect to a closed subspace. However, neither method applies to the general setup with 2 maximal monotone and 1 monotone-Lipschitz operators.In this work, we first present a method that naturally combines and unifies DR and FBF. We prove convergence under further assumptions, and we prove, through a counterexample, that convergence cannot be established in full generality. We then propose a second method that naturally combines and unifies DR and forward-reflected-backward (FRB), a recently proposed alternative to FBF by Malitsky and Tam [12]. We show that this combination of DR and FRB does converge in full generality.The paper is organized as follows. Section 2 states the problem formally. Section 3 reviews preliminary information and sets up the notation. Section 4 presents our first proposed method combining DR and FBF, proves convergence under certain further assumptions, and proves divergence in the fully general case. Section 5 presents our second proposed method combining DR and FRB and proves convergen...

show abstract

Systems of Structured Monotone Inclusions: Duality, Algorithms, and Applications

Cited by 64 publications

References 42 publications

Convergence Rate Analysis of Primal-Dual Splitting Schemes

Convergence Rate Analysis of Primal-Dual Splitting Schemes

A First-Order Stochastic Primal-Dual Algorithm with Correction Step

Finding the Forward-Douglas–Rachford-Forward Method

Contact Info

Product

Resources

About