Acceleration of the PDHGM on Partially Strongly Convex Functions

Valkonen, Tuomo; Pock, Thomas

doi:10.1007/s10851-016-0692-2

Cited by 18 publications

(19 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Several first-order optimisation methods have been developed for (P 0 ) without blockseparable structure, typically for both G and F convex and K linear. Recently also some non-convexity and non-linearity has been introduced [4,21,23,37]. In applications in image processing and data science, one of G or F is typically non-smooth.…”

mentioning

confidence: 99%

“…Our present paper is based on [37] on the acceleration of the PDHGM when G is strongly convex only on a subspace: the deterministic two-block case m = 2 and n = 1 of (GF). Besides enabling (doubly-)stochastic updates and an arbitrary number of both primal and dual blocks, in the present work, we derive simplified step length rules through a more careful analysis.…”

mentioning

confidence: 99%

“…We use the notation u = (x, y) to combine the primal variable x and the dual variable y into a single variable u. Following [19,37], the primal-dual method of Chambolle and Pock [6] (PDHGM) may then be written in proximal point form as…”

mentioning

confidence: 99%

“…In [37], we extended the PDHGM to partially strongly convex problems: in (GF) this corresponded to the primal two-block and dual single-block case m = 2 and n = 1 with only G 1 assumed strongly convex. This extension was based on taking in (PP 0 ) the preconditioner…”

mentioning

confidence: 99%

See 3 more Smart Citations

Block-proximal methods with spatially adapted acceleration

Valkonen¹

2019

etna

Self Cite

View full text Add to dashboard Cite

We study and develop (stochastic) primal-dual block-coordinate descent methods for convex problems based on the method due to Chambolle and Pock. Our methods have known convergence rates for the iterates and the ergodic gap of O(1/N 2 ) if each block is strongly convex, O(1/N ) if no convexity is present, and more generally a mixed rate O(1/N 2 ) + O(1/N ) for strongly convex blocks if only some blocks are strongly convex. Additional novelties of our methods include blockwise-adapted step lengths and acceleration as well as the ability to update both the primal and dual variables randomly in blocks under a very light compatibility condition. In other words, these variants of our methods are doubly-stochastic. We test the proposed methods on various image processing problems, where we employ pixelwise-adapted acceleration.BLOCK-PROXIMAL METHODS WITH SPATIALLY ADAPTED ACCELERATION 17 conditions on the various step length and testing parameters. These conditions need to be verified through the development of explicit parameter update rules. We do this in Section 4 along with proving the claimed convergence rates (Theorem 4.5 and its corollaries). We also present there the final detailed versions of our proposed algorithms: Algorithm 1 (doubly stochastic) and Algorithm 2 (simplified). We finish with numerical experiments in Section 5.2. Background and overall structure of the algorithm. To make the notation definite, we write L(X; Y ) for the space of bounded linear operators between Hilbert spaces X and Y . The identity operator we denote by I. For T, S ∈ L(X; X), we use T ≥ S to indicate that T − S is positive semi-definite; in particular T ≥ 0 means that T is positive semi-definite. Also for possibly non-self-adjoint T , we introduce the inner product and norm-like notationsx, z T := T x, z andThe iteration index is off-by-one for σ ,i+1 and ψ ,i+1 for reasons of the historical development of the Chambolle-Pock method, when it was not written as a preconditioned proximal point method.

show abstract

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Block-proximal methods with spatially adapted acceleration

Valkonen¹

2019

etna

Self Cite

View full text Add to dashboard Cite

show abstract

“…The appropriate choice for µ > 0 is related to the constant of strong convexity of F * , and in the convex case yields the optimal convergence rate of O(1/k 2 ) for the functional values rather than the rate O(1/k) for the original version; see [3,4,21]. A similar acceleration is possible if G is strongly convex by swapping the roles of σ i and τ i in line 4; we will refer to both variants as Algorithm 2 in the following.…”

mentioning

confidence: 99%

Primal-Dual Extragradient Methods for Nonlinear Nonsmooth PDE-Constrained Optimization

Clason¹,

Valkonen²

2017

SIAM J. Optim.

Self Cite

View full text Add to dashboard Cite

Abstract. We study the extension of the Chambolle-Pock primal-dual algorithm to nonsmooth optimization problems involving nonlinear operators between function spaces. Local convergence is shown under technical conditions including metric regularity of the corresponding primal-dual optimality conditions. We also show convergence for a Nesterov-type accelerated variant, provided one part of the functional is strongly convex. We show the applicability of the accelerated algorithm to examples of inverse problems with L 1 and L ∞ fitting terms as well as of state-constrained optimal control problems, where convergence can be guaranteed after introducing an (arbitrarily small, still nonsmooth) Moreau-Yosida regularization. This is verified in numerical examples.

show abstract

First-Order Primal–Dual Methods for Nonsmooth Non-convex Optimisation

Valkonen

2021

Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging

View full text Add to dashboard Cite

Acceleration of the PDHGM on Partially Strongly Convex Functions

Cited by 18 publications

References 32 publications

Block-proximal methods with spatially adapted acceleration

Block-proximal methods with spatially adapted acceleration

Primal-Dual Extragradient Methods for Nonlinear Nonsmooth PDE-Constrained Optimization

First-Order Primal–Dual Methods for Nonsmooth Non-convex Optimisation

Contact Info

Product

Resources

About