A Variable Metric Extension of the Forward–Backward–Forward Algorithm for Monotone Operators

Vũ, Bằng Công

doi:10.1080/01630563.2013.763825

Cited by 32 publications

(36 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Remark 1. There is an asymmetry in our notation and the notation of [20,45,21,37]. In our analysis, the map U ∈ S ρ (H) induces a metric on H. In other papers, the maps U −1 induce a metric on H.…”

Section: Basic Properties Of Resolvents and Averaged Operatorsmentioning

confidence: 96%

Convergence Rate Analysis of Primal-Dual Splitting Schemes

Davis

2015

SIAM J. Optim.

View full text Add to dashboard Cite

Primal-dual splitting schemes are a class of powerful algorithms that solve complicated monotone inclusions and convex optimization problems that are built from many simpler pieces. They decompose problems that are built from sums, linear compositions, and infimal convolutions of simple functions so that each simple term is processed individually via proximal mappings, gradient mappings, and multiplications by the linear maps. This leads to easily implementable and highly parallelizable or distributed algorithms, which often obtain nearly state-of-the-art performance. In this paper, we analyze a monotone inclusion problem that captures a large class of primal-dual splittings as a special case. We introduce a unifying scheme and use some abstract analysis of the algorithm to prove convergence rates of the proximal point algorithm, forward-backward splitting, Peaceman-Rachford splitting, and forward-backward-forward splitting applied to the model problem. Our ergodic convergence rates are deduced under variable metrics, stepsizes, and relaxation parameters. Our nonergodic convergence rates are the first shown in the literature. Finally, we apply our results to a large class of primal-dual algorithms that are a special case of our scheme and deduce their convergence rates.higher order regularizations of the components in each group, a task which might otherwise be intractable in large-scale applications.Finally, we note that the use of infimal convolutions in applications is not widespread, so we list a few instances where they may be useful: Infimal convolutions are used in image recovery [14, section 5] to remove staircasing effects in the total variation model. The infimal convolution of the indicator functions of two closed convex sets is the indicator function of their Minkowski sum, which has applications in motion planning for robotics [32, section 4.3.2]. In convex analysis, the Moreau envelope of a function arises as an infimal convolution with a multiple of the squared norm [2, section 12.4]. More generally, the infimal convolution of h i and l i can be interpreted as a regularization or smoothing of h i by l i and vice versa [2, section 18.3].1.1. Goals, challenges, and approaches. This work seeks to improve the theoretical understanding of the convergence rates of primal-dual splitting schemes. In this paper, we study primal-dual algorithms that are applications of standard operator splitting algorithms in product spaces consisting of primal and dual variables. Consequently, the convergence theory for these algorithms is well-developed, and they are known to converge (weakly) under mild conditions.Although we understand when these algorithms converge, relatively little is known about their rate of convergence. For convex optimization algorithms, the ergodic convergence rate of the primal-dual gap has been analyzed in a few cases [15,7,6,43]. However, even in cases where convergence rates are known, variable metrics and stepsizes, which can significantly improve practical performance of the algorithms [40...

show abstract

Section: Basic Properties Of Resolvents and Averaged Operatorsmentioning

confidence: 96%

Convergence Rate Analysis of Primal-Dual Splitting Schemes

Davis

2015

SIAM J. Optim.

View full text Add to dashboard Cite

show abstract

“…Although they also compute dual solutions, for brevity, we present only the primal convergence result for the error-free, unrelaxed formulations of these algorithms. The first one is known as the primal-dual forward-backwardforward algorithm; see [32] for details and [61] for a variable metric version. for n = 0, 1, .…”

Section: Algorithm 35 (Douglas-rachford)mentioning

confidence: 99%

Proximal Activation of Smooth Functions in Splitting Algorithms for Convex Image Recovery

Combettes¹,

Glaudin²

2019

SIAM J. Imaging Sci.

View full text Add to dashboard Cite

Structured convex optimization problems typically involve a mix of smooth and nonsmooth functions. The common practice is to activate the smooth functions via their gradient and the nonsmooth ones via their proximity operator. We show that, although intuitively natural, this approach is not necessarily the most efficient numerically and that, in particular, activating all the functions proximally may be advantageous. To make this viewpoint viable computationally, we derive a number of new examples of proximity operators of smooth convex functions arising in applications. A novel variational model to relax inconsistent convex feasibility problems is also investigated within the proposed framework. Several numerical applications to image recovery are presented to compare the behavior of fully proximal versus mixed proximal/gradient implementations of several splitting algorithms.

show abstract

“…The general diagonal case was considered in several papers in the 1980s as a simple quasi-Newton method, but never widely adapted. Variable metric operator splitting methods have been designed to solve monotone inclusion problems and convex minimization problems, see for instance [22,72] in the maximal monotone case and [17] for the strongly monotone case. The convergence proofs rely on a variable metric extension of quasi-Fejér monotonicity [21].…”

Section: Relation To Prior Workmentioning

confidence: 99%

“…The right hand side uses the so-called proximal mapping, which is formally introduced in Definition 3.1. Standard results (see, e.g., [22,72]) show that, for a sequence (B k ) k∈N that varies moderately (in the Loewner partial ordering sense) such that inf k∈N B k = 1, convergence of the sequence (x k ) k∈N is expected when 0 < κ κ k κ < 2/L, where L is the Lipschitz constant of ∇f .…”

mentioning

confidence: 99%

On Quasi-Newton Forward-Backward Splitting: Proximal Calculus and Convergence

Becker¹,

Fadili²,

Ochs³

2019

SIAM J. Optim.

View full text Add to dashboard Cite

We introduce a framework for quasi-Newton forward-backward splitting algorithms (proximal quasi-Newton methods) with a metric induced by diagonal ± rank-r symmetric positive definite matrices. This special type of metric allows for a highly efficient evaluation of the proximal mapping. The key to this efficiency is a general proximal calculus in the new metric. By using duality, formulas are derived that relate the proximal mapping in a rank-r modified metric to the original metric. We also describe efficient implementations of the proximity calculation for a large class of functions; the implementations exploit the piece-wise linear nature of the dual problem. Then, we apply these results to acceleration of composite convex minimization problems, which leads to elegant quasi-Newton methods for which we prove convergence. The algorithm is tested on several numerical examples and compared to a comprehensive list of alternatives in the literature. Our quasi-Newton splitting algorithm with the prescribed metric compares favorably against state-of-the-art. The algorithm has extensive applications including signal processing, sparse recovery, machine learning and classification to name a few. Contributions.We introduce an general proximal calculus in a metric V = P ± Q ∈ S ++ (N ) given by P ∈ S ++ (N ) and a positive semi-definite rank-r matrix Q. This significantly extends the result in the preliminary version of this paper [7], where only V = P + Q with a rank-1 matrix Q is addressed. The general calculus is accompanied by several more concrete examples (see Section 3.3.4 for a non-exhaustive list), where, for example, the piecewise linear nature of certain dual problems is rigorously exploited.

show abstract

A Variable Metric Extension of the Forward–Backward–Forward Algorithm for Monotone Operators

Cited by 32 publications

References 18 publications

Convergence Rate Analysis of Primal-Dual Splitting Schemes

Convergence Rate Analysis of Primal-Dual Splitting Schemes

Proximal Activation of Smooth Functions in Splitting Algorithms for Convex Image Recovery

On Quasi-Newton Forward-Backward Splitting: Proximal Calculus and Convergence

Contact Info

Product

Resources

About