On the convergence of a linesearch based proximal-gradient method for nonconvex optimization

Bonettini, Silvia; Loris, Ignace; Porta, Federica; Prato, Marco; Rebegoldi, Simone

doi:10.1088/1361-6420/aa5bfd

Cited by 53 publications

(65 citation statements)

References 57 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The condition (3) is studied in [2,Section 4.1], which applies a primal-dual approach to (2) to satisfy it. In this connection, note that if we have access to a lower bound Q LB ≤ Q * (obtained by finding a feasible point for the dual of (2), or other means), then any d satisfying Q(d) ≤ (1 − η)Q LB also satisfies (3).…”

mentioning

confidence: 99%

See 1 more Smart Citation

Inexact Successive quadratic approximation for regularized optimization

Lee

Wright

2019

Comput Optim Appl

View full text Add to dashboard Cite

Successive quadratic approximations, or second-order proximal methods, are useful for minimizing functions that are a sum of a smooth part and a convex, possibly nonsmooth part that promotes regularization. Most analyses of iteration complexity focus on the special case of proximal gradient method, or accelerated variants thereof. There have been only a few studies of methods that use a second-order approximation to the smooth part, due in part to the difficulty of obtaining closed-form solutions to the subproblems at each iteration. In fact, iterative algorithms may need to be used to find inexact solutions to these subproblems. In this work, we present global analysis of the iteration complexity of inexact successive quadratic approximation methods, showing that an inexact solution of the subproblem that is within a fixed multiplicative precision of optimality suffices to guarantee the same order of convergence rate as the exact version, with complexity related in an intuitive way to the measure of inexactness. Our result allows flexible choices of the second-order term, including Newton and quasi-Newton choices, and does not necessarily require increasing precision of the subproblem solution on later iterations. For problems exhibiting a property related to strong convexity, the algorithms converge at global linear rates. For general convex problems, the convergence rate is linear in early stages, while the overall rate is O(1/k). For nonconvex problems, a first-order optimality criterion converges to zero at a rate of O(1/ √ k).

show abstract

mentioning

confidence: 99%

“…In practical situations, we need not enforce (3) explicitly for some chosen value of η. In fact, we do not necessarily require η to be known, or (3) to be checked at all. Rather, we can take advantage of the convergence rates of whatever solver is applied to (2) to ensure that (3) holds for some value of η ∈ (0, 1), possibly unknown.…”

mentioning

confidence: 99%

Inexact Successive quadratic approximation for regularized optimization

Lee

Wright

2019

Comput Optim Appl

View full text Add to dashboard Cite

show abstract

“…also. It follows from the continuity of the operations in the right hand sides of algorithm (14) that (u † , v † ) satisfies the equations (16), which characterize the minimizers of problem (4). One can then replace (û,v) by (u † , v † ) in inequality (27) to obtain…”

Section: Lemmamentioning

confidence: 99%

“…Proof:As f is strongly convex, problem (4) is guaranteed to have a (unique) solutionû. Hence, there existsv such that equations (16) are satisfied. We start from inequality (24) derived in the proof of Theorem 1.…”

Section: Lemmamentioning

confidence: 99%

On starting and stopping criteria for nested primal-dual iterations

Chen

Loris

2018

Numer Algor

View full text Add to dashboard Cite

The importance of an adequate inner loop starting point (as opposed to a sufficient inner loop stopping rule) is discussed in the context of an numerical optimization algorithm consisting of nested primaldual proximal-gradient iterations. While the number of inner iterations is fixed in advance, convergence of the whole algorithm is still guaranteed by virtue of a warm-start strategy for the inner loop, showing that inner loop "starting rules" can be just as effective as "stopping rules" for guaranteeing convergence. The algorithm itself is applicable to the numerical solution of convex optimization problems defined by the sum of a differentiable term and two possibly non-differentiable terms. One of the latter terms should take the form of the composition of a linear map and a proximable function, while the differentiable term needs an accessible gradient. The algorithm reduces to the classical proximal gradient algorithm in certain special cases and it also generalizes other existing algorithms. In addition, under some conditions of strong convexity, we show a linear rate of convergence.

show abstract

“…In addition, the convergence of Algorithm 1 can be asserted whenever the objective function J satisfies the KurdykaLojasiewicz (KL) property [31,32] at each point of its domain. More precisely, as shown in a number of recent papers [33,34,35], one can prove the convergence of a sequence {φ (n) } n∈N to a limit point (if any exists) which is stationary for J if the following three conditions are satisfied:…”

Section: Solve the Linear System Rmentioning

confidence: 99%

A comparison of edge-preserving approaches for differential interference contrast microscopy

et al. 2017

Self Cite

View full text Add to dashboard Cite

Abstract. In this paper we address the problem of estimating the phase from color images acquired with differential-interference-contrast microscopy. In particular, we consider the nonlinear and nonconvex optimization problem obtained by regularizing a least-squares-like discrepancy term with an edge-preserving functional, given by either the hypersurface potential or the total variation one. We investigate the analytical properties of the resulting objective functions, proving the existence of minimum points, and we propose effective optimization tools able to obtain in both the smooth and the nonsmooth case accurate reconstructions with a reduced computational demand.AMS classification scheme numbers: 65K05, 90C30, 90C90

show abstract

On the convergence of a linesearch based proximal-gradient method for nonconvex optimization

Cited by 53 publications

References 57 publications

Inexact Successive quadratic approximation for regularized optimization

Inexact Successive quadratic approximation for regularized optimization

On starting and stopping criteria for nested primal-dual iterations

A comparison of edge-preserving approaches for differential interference contrast microscopy

Contact Info

Product

Resources

About