First-order optimization algorithms via inertial systems with Hessian driven damping

Attouch, Hédy; Chbani, Zaki; Fadili, Jalal M.; Riahi, Hassan

doi:10.1007/s10107-020-01591-1

Cited by 83 publications

(89 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this section we will prove the existence and uniqueness of a global C 2 -solution of the dynamical system (5). The proof of the existence and uniqueness theorem is based on the idea to reformulate (5) as a particular first order dynamical system in a suitably chosen product space (see also [11]).…”

Section: Existence and Uniquenessmentioning

confidence: 99%

“…Assume now that β > 0. We notice that x : [t 0 , +∞) −→ H is a solution of the dynamical system (5), that is…”

Section: Theorem 21 For Every Initial Valuementioning

confidence: 99%

“…In this section we will show to which extent different assumptions we impose to the Tikhonov parametrization t → (t) influence the asymptotic behaviour of the trajectory x generated by the dynamical system (5). In particular, we are looking at the convergence of the function g along the trajectory and the weak convergence of the trajectory.…”

Section: Asymptotic Analysismentioning

confidence: 99%

“…We start with a result which provides a setting that guarantees the convergence of g(x(t)) to min g as t → +∞. Theorem 3.1 Let x be the unique global C 2 -solution of (5). Assume that one of the following conditions is fulfilled:…”

Section: Asymptotic Analysismentioning

confidence: 99%

“…Fast convergence rates for the values and the gradient of the objective function along the trajectories are obtained and the weak convergence of the trajectories to a minimizer of g is shown. We would also like to mention that iterative schemes which result via (symplectic) discretizations of dynamical systems with Hessian driven damping terms have been recently formulated and investigated from the point of view of their convergence properties in [5,18,19]. Another development having as a starting point (1) is the investigation of dynamical systems involving a Tikhonov regularization term.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Tikhonov regularization of a second order dynamical system with Hessian driven damping

2020

View full text Add to dashboard Cite

We investigate the asymptotic properties of the trajectories generated by a second-order dynamical system with Hessian driven damping and a Tikhonov regularization term in connection with the minimization of a smooth convex function in Hilbert spaces. We obtain fast convergence results for the function values along the trajectories. The Tikhonov regularization term enables the derivation of strong convergence results of the trajectory to the minimizer of the objective function of minimum norm. Keywords Second order dynamical system • Convex optimization • Tikhonov regularization • Fast convergence methods • Hessian-driven damping Mathematics Subject Classification 34G25 • 47J25 • 47H05 • 90C26 • 90C30 • 65K10 The paper is dedicated to Prof. Marco A. López on the occasion of his 70th birthday.

show abstract

Section: Existence and Uniquenessmentioning

confidence: 99%

“…Assume now that β > 0. We notice that x : [t 0 , +∞) −→ H is a solution of the dynamical system (5), that is…”

Section: Theorem 21 For Every Initial Valuementioning

confidence: 99%

Section: Asymptotic Analysismentioning

confidence: 99%

Section: Asymptotic Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Tikhonov regularization of a second order dynamical system with Hessian driven damping

2020

View full text Add to dashboard Cite

show abstract

Splitting Methods for Nonconvex and Nonsmooth Optimization

Li,

2022

Encyclopedia of Optimization

View full text Add to dashboard Cite

Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution

Attouch,

László

2024

Math Meth Oper Res

View full text Add to dashboard Cite

In a Hilbertian framework, for the minimization of a general convex differentiable function f, we introduce new inertial dynamics and algorithms that generate trajectories and iterates that converge fastly towards the minimizer of f with minimum norm. Our study is based on the non-autonomous version of the Polyak heavy ball method, which, at time t, is associated with the strongly convex function obtained by adding to f a Tikhonov regularization term with vanishing coefficient $$\varepsilon (t)$$ ε ( t ) . In this dynamic, the damping coefficient is proportional to the square root of the Tikhonov regularization parameter $$\varepsilon (t)$$ ε ( t ) . By adjusting the speed of convergence of $$\varepsilon (t)$$ ε ( t ) towards zero, we will obtain both rapid convergence towards the infimal value of f, and the strong convergence of the trajectories towards the element of minimum norm of the set of minimizers of f. In particular, we obtain an improved version of the dynamic of Su-Boyd-Candès for the accelerated gradient method of Nesterov. This study naturally leads to corresponding first-order algorithms obtained by temporal discretization. In the case of a proper lower semicontinuous and convex function f, we study the proximal algorithms in detail, and show that they benefit from similar properties.

show abstract

First-order optimization algorithms via inertial systems with Hessian driven damping

Cited by 83 publications

References 35 publications

Tikhonov regularization of a second order dynamical system with Hessian driven damping

Tikhonov regularization of a second order dynamical system with Hessian driven damping

Splitting Methods for Nonconvex and Nonsmooth Optimization

Convex optimization via inertial algorithms with vanishing Tikhonov regularization: fast convergence to the minimum norm solution

Contact Info

Product

Resources

About