Solving monotone inclusions involving parallel sums of linearly composed maximally monotone operators

Boţ, Radu Ioan; Hendrich, Christopher

doi:10.3934/ipi.2016014

Cited by 8 publications

(21 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Furthermore, the complexity of Problem 2 can be increased in various ways, e.g., by precomposing each of h i and l i with linear operators [3,10] or by solving systems of such inclusions [17,8]. We choose to discuss this relatively simple formulation for clarity of exposition.…”

Section: Damek Davismentioning

confidence: 99%

“…In this paper, we are mainly concerned with the line of work that began in [41,15,25] and the many generalizations and enhancements of the basic framework that followed [19,22,46,12,9,10,17,31,6,18]. Thus, we consider the following prototypical convex optimization problem as our guiding example: (1.1) where denotes the infimal convolution operation (see section 1.2), n ∈ N, n ≥ 1, H i are Hilbert spaces for i = 0, .…”

Section: Introductionmentioning

confidence: 99%

“…Our analysis not only deduces the convergence rates of a large class of primal-dual algorithms found in the literature, but it also serves as a resource for the analysis of future primal-dual algorithms that solve generalizations of problem 1.1, e.g., [3,10].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Convergence Rate Analysis of Primal-Dual Splitting Schemes

Davis

2015

SIAM J. Optim.

View full text Add to dashboard Cite

Primal-dual splitting schemes are a class of powerful algorithms that solve complicated monotone inclusions and convex optimization problems that are built from many simpler pieces. They decompose problems that are built from sums, linear compositions, and infimal convolutions of simple functions so that each simple term is processed individually via proximal mappings, gradient mappings, and multiplications by the linear maps. This leads to easily implementable and highly parallelizable or distributed algorithms, which often obtain nearly state-of-the-art performance. In this paper, we analyze a monotone inclusion problem that captures a large class of primal-dual splittings as a special case. We introduce a unifying scheme and use some abstract analysis of the algorithm to prove convergence rates of the proximal point algorithm, forward-backward splitting, Peaceman-Rachford splitting, and forward-backward-forward splitting applied to the model problem. Our ergodic convergence rates are deduced under variable metrics, stepsizes, and relaxation parameters. Our nonergodic convergence rates are the first shown in the literature. Finally, we apply our results to a large class of primal-dual algorithms that are a special case of our scheme and deduce their convergence rates.higher order regularizations of the components in each group, a task which might otherwise be intractable in large-scale applications.Finally, we note that the use of infimal convolutions in applications is not widespread, so we list a few instances where they may be useful: Infimal convolutions are used in image recovery [14, section 5] to remove staircasing effects in the total variation model. The infimal convolution of the indicator functions of two closed convex sets is the indicator function of their Minkowski sum, which has applications in motion planning for robotics [32, section 4.3.2]. In convex analysis, the Moreau envelope of a function arises as an infimal convolution with a multiple of the squared norm [2, section 12.4]. More generally, the infimal convolution of h i and l i can be interpreted as a regularization or smoothing of h i by l i and vice versa [2, section 18.3].1.1. Goals, challenges, and approaches. This work seeks to improve the theoretical understanding of the convergence rates of primal-dual splitting schemes. In this paper, we study primal-dual algorithms that are applications of standard operator splitting algorithms in product spaces consisting of primal and dual variables. Consequently, the convergence theory for these algorithms is well-developed, and they are known to converge (weakly) under mild conditions.Although we understand when these algorithms converge, relatively little is known about their rate of convergence. For convex optimization algorithms, the ergodic convergence rate of the primal-dual gap has been analyzed in a few cases [15,7,6,43]. However, even in cases where convergence rates are known, variable metrics and stepsizes, which can significantly improve practical performance of the algorithms [40...

show abstract

Section: Damek Davismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Convergence Rate Analysis of Primal-Dual Splitting Schemes

Davis

2015

SIAM J. Optim.

View full text Add to dashboard Cite

show abstract

“…A large variety of recent algorithms [12,26,37] and their generalizations and enhancements [4,7,6,8,16,17,18,20,28,39] are (skillful) applications of one of the following three operator-splitting schemes: (i) forwardbackward-forward splitting (FBFS) [38], (ii) forward-backward splitting (FBS) [36], and (iii) Douglas-Rachford splitting (DRS) [32], which all split the sum of two operators. (The recently introduced forward-Douglas-Rachford splitting (FDRS) turns out to be a special case of FBS applied to a suitable monotone inclusion [23,Section 7].)…”

Section: Existing Two-operator Splitting Schemesmentioning

confidence: 99%

A Three-Operator Splitting Scheme and its Optimization Applications

Davis

Yin

2017

Set-Valued Var. Anal

289

438

View full text Add to dashboard Cite

Operator splitting schemes have been successfully used in computational sciences to reduce complex problems into a series of simpler subproblems. Since 1950s, these schemes have been widely used to solve problems in PDE and control. Recently, large-scale optimization problems in machine learning, signal processing, and imaging have created a resurgence of interest in operator-splitting based algorithms because they often have simple descriptions, are easy to code, and have (nearly) state-of-the-art performance for large-scale optimization problems. Although operator splitting techniques were introduced over 60 years ago, their importance has significantly increased in the past decade. This paper introduces a new operator-splitting scheme for solving a variety of problems that are reduced to a monotone inclusion of three operators, one of which is cocoercive. Our scheme is very simple, and it does not reduce to any existing splitting schemes. Our scheme recovers the existing forward-backward, Douglas-Rachford, and forward-Douglas-Rachford splitting schemes as special cases.Our new splitting scheme leads to a set of new and simple algorithms for a variety of other problems, including the 3-set split feasibility problems, 3-objective minimization problems, and doubly and multiple regularization problems, as well as the simplest extension of the classic ADMM from 2 to 3 blocks of variables. In addition to the basic scheme, we introduce several modifications and enhancements that can improve the convergence rate in practice, including an acceleration that achieves the optimal rate of convergence for strongly monotone inclusions. Finally, we evaluate the algorithm on several applications.

show abstract

“…This splitting algorithm plays a role in solving a large class of composite monotone inclusions [3] and monotone inclusions involving the parallel sums [2,10,11,15] as well as applications to conposite convex optimization problem involving the infimal-convolutions [2][3][4]11,15]. However, these works are limitted to deterministic setting.…”

Section: Introductionmentioning

confidence: 99%

Almost sure convergence of the forward–backward–forward splitting algorithm

Vũ

2015

Optim Lett

View full text Add to dashboard Cite

In this paper, we propose a stochastic forward-backward-forward splitting algorithm and prove its almost sure weak convergence in real separable Hilbert spaces. Applications to composite monotone inclusion and minimization problems are demonstrated.Keywords Monotone inclusion · Monotone operator · Operator splitting · Lipschitzian operators · Forward-backward-forward algorithm · Composite operator · Duality · Primal-dual algorithm

show abstract

Solving monotone inclusions involving parallel sums of linearly composed maximally monotone operators

Cited by 8 publications

References 32 publications

Convergence Rate Analysis of Primal-Dual Splitting Schemes

Convergence Rate Analysis of Primal-Dual Splitting Schemes

A Three-Operator Splitting Scheme and its Optimization Applications

Almost sure convergence of the forward–backward–forward splitting algorithm

Contact Info

Product

Resources

About