Proximal Splitting Algorithms for Convex Optimization: A Tour of Recent Advances, with New Twists

Condat, Laurent; Kitahara, Daichi; Contreras, Andres; Hirabayashi, Akira

doi:10.48550/arxiv.1912.00137

Cited by 12 publications

(25 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For large-scale convex optimization problems like (1), primal-dual splitting algorithms [9,15,16,24,29,34,41] are well suited, as they are easy to implement and typically show state-of-the-art performance. The fully split algorithms do not require the ability to project onto the constraint space {x ∈ X : Kx = b}.…”

Section: Introductionmentioning

confidence: 99%

An Optimal Algorithm for Strongly Convex Minimization under Affine Constraints

Salim¹,

Condat²,

Kovalev³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Optimization problems under affine constraints appear in various areas of machine learning. We consider the task of minimizing a smooth strongly convex function F (x) under the affine constraint Kx = b, with an oracle providing evaluations of the gradient of F and matrix-vector multiplications by K and its transpose. We provide lower bounds on the number of gradient computations and matrix-vector multiplications to achieve a given accuracy. Then we propose an accelerated primal-dual algorithm achieving these lower bounds. Our algorithm is the first optimal algorithm for this class of problems.

show abstract

Section: Introductionmentioning

confidence: 99%

An Optimal Algorithm for Strongly Convex Minimization under Affine Constraints

Salim¹,

Condat²,

Kovalev³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, we can still obtain a linear convergence rate by carefully utilizing the L-smoothness property of the function F (x). Similar issue appeared in the design of algorithms for solving linearly-constrained minimizations problems, including the non-accelerated algorithms (Condat et al, 2019;Salim et al, 2020) and the optimal algorithm (Salim et al, 2021). However, the authors of these works considered a different problem reformulation from (11), and hence their results can not be applied here.…”

Section: Primal Algorithm Design and Convergencementioning

confidence: 99%

Lower Bounds and Optimal Algorithms for Smooth and Strongly Convex Decentralized Optimization Over Time-Varying Networks

Kovalev¹,

Gasanov²,

Richtárik³

et al. 2021

Preprint

View full text Add to dashboard Cite

We consider the task of minimizing the sum of smooth and strongly convex functions stored in a decentralized manner across the nodes of a communication network whose links are allowed to change in time. We solve two fundamental problems for this task. First, we establish the first lower bounds on the number of decentralized communication rounds and the number of local computations required to find an -accurate solution. Second, we design two optimal algorithms that attain these lower bounds: (i) a variant of the recently proposed algorithm ADOM (Kovalev et al., 2021) enhanced via a multi-consensus subroutine, which is optimal in the case when access to the dual gradients is assumed, and (ii) a novel algorithm, called ADOM+, which is optimal in the case when access to the primal gradients is assumed. We corroborate the theoretical efficiency of these algorithms by performing an experimental comparison with existing state-of-the-art methods.

show abstract

“…To solve the problem ( 21), a well suited algorithm is the Proximal Method of Multipliers [33], [34], which, initialized with some variables U (0) ∈ (C 3×3 ) |E| and s (0) ∈ C d , consists in the iteration: for i = 0, 1, . .…”

Section: Proposed Algorithmmentioning

confidence: 99%

“…where L * denotes the adjoint operator of L, f * denotes the convex conjugate of f [35], τ > 0 is a parameter, and we set σ = 1/( L 2 τ ), where the squared operator norm L 2 is twice the maximum number of edges per node. With this choice, the variable s (i) in the algorithm converges to a solution s ⋆ of (21) [34,Theorem 4.3]. In the algorithm, the proximity operator prox σf * maps each matrix Q n,n ′ , for (n, n ′ ) ∈ E, to the projection of Q n,n ′ + σId onto the cone of Hermitian negative semidefinite matrices; this is achieved by computing the eigendecomposition and setting the positive eigenvalues to zero.…”

Section: Proposed Algorithmmentioning

confidence: 99%

“…Thus, the proposed approach is best suited when the noise level is not too high. We solve the problem using the Chambolle-Pock algorithm [34], [37], which is similar to the algorithm shown in Section IV, with additional enforcement of x 1 = e −j , x 10 = e 2j at every iteration. Again, it turns out that the relaxation ( 24) is tight and we obtain the exact solution to (23).…”

Section: B Denoising Of a 2-d Imagementioning

confidence: 99%

See 1 more Smart Citation

Tikhonov Regularization of Circle-Valued Signals

Condat

2021

Preprint

Self Cite

View full text Add to dashboard Cite

It is common to have to process signals or images whose values are cyclic and can be represented as points on the complex circle, like wrapped phases, angles, orientations, or color hues. We consider a Tikhonov-type regularization model to smoothen or interpolate circle-valued signals defined on arbitrary graphs. We propose a convex relaxation of this nonconvex problem as a semidefinite program, and an efficient algorithm to solve it.

show abstract

Proximal Splitting Algorithms for Convex Optimization: A Tour of Recent Advances, with New Twists

Cited by 12 publications

References 0 publications

An Optimal Algorithm for Strongly Convex Minimization under Affine Constraints

An Optimal Algorithm for Strongly Convex Minimization under Affine Constraints

Lower Bounds and Optimal Algorithms for Smooth and Strongly Convex Decentralized Optimization Over Time-Varying Networks

Tikhonov Regularization of Circle-Valued Signals

Contact Info

Product

Resources

About