In a real Hilbert space H, we study the fast convergence properties as t → +∞ of the trajectories of the second-order evolution equationẍ (t) + α tẋ (t) + ∇Φ(x(t)) = 0,where ∇Φ is the gradient of a convex continuously differentiable function Φ : H → R, and α is a positive parameter. In this inertial system, the viscous damping coefficient α t vanishes asymptotically in a moderate way. For α > 3, we show that any trajectory converges weakly to a minimizer of Φ, just assuming that argmin Φ = ∅. The strong convergence is established in various practical situations. These results complement the O(t −2 ) rate of convergence for the values obtained by Su, Boyd and Candès. Time discretization of this system, and some of its variants, provides new fast converging algorithms, expanding the field of rapid methods for structured convex minimization introduced by Nesterov, and further developed by Beck and Teboulle. This study also complements recent advances due to Chambolle and Dossal.3. Relationship with fast numerical optimization methods: As pointed out in [38, Section 2], for α = 3, (1) can be seen as a continuous version of the fast convergent method of Nesterov (see [29,30,31,32]), and its widely used successors, such as the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA), studied in [19]. These methods have a convergence rate of Φ(x k ) − min Φ = O(k −2 ), where k is the number of iterations. As for the continuous-time system (1), convergence of the sequences generated by FISTA and related methods has not been established so far. This is a central and long-standing question in the study of numerical optimization methods.The purpose of this research is to establish the convergence of the trajectories satisfying (1), as well as the sequences generated by the corresponding numerical methods with Nesterov-type acceleration. We also complete the study with several stability properties concerning both the continuous-time system and the algorithms.

Abstract. In a Hilbert space setting H, given Φ : H → R a convex continuously differentiable function, and α a positive parameter, we consider the inertial system with Asymptotic Vanishing Damping (AVD) αẍ (t) + α tẋ (t) + ∇Φ(x(t)) = 0.Depending on the value of α with respect to 3, we give a complete picture of the convergence properties as t → +∞ of the trajectories generated by (AVD) α , as well as iterations of the corresponding algorithms. Indeed, as shown by Su-Boyd-Candès, the case α = 3 corresponds to a continuous version of the accelerated gradient method of Nesterov, with the rate of convergence Φ(x(t)) − min Φ = O(t −2 ) for α ≥ 3. Our main result concerns the subcritical case α ≤ 3, where we show that Φ(). This overall picture shows a continuous variation of the rate of convergence of the values Φ(x(t)) − min H Φ = O(t −p(α) ) with respect to α > 0: the coefficient p(α) increases linearly up to 2 when α goes from 0 to 3, then displays a plateau. Then we examine the convergence of trajectories to optimal solutions. When α > 3, we obtain the weak convergence of the trajectories, and so find the recent results by May and Attouch-Chbani-Peypouquet-Redont. As a new result, in the one-dimensional framework, for the critical value α = 3, we prove the convergence of the trajectories without any restrictive hypothesis on the convex function Φ. In the second part of this paper, we study the convergence properties of the associated forward-backward inertial algorithms. They aim to solve structured convex minimization problems of the form min {Θ := Φ + Ψ}, with Φ smooth and Ψ nonsmooth. The continuous dynamics serves as a guideline for this study, and is very useful for suggesting Lyapunov functions. We obtain a similar rate of convergence for the sequence of iterates (x k ): for α ≤ 3 we have Θ(, and for α > 3 Θ(x k ) − min Θ = o(k −2 ) . We conclude this study by showing that the results are robust with respect to external perturbations.

In a Hilbert space setting, for convex optimization, we show the convergence of the iterates to optimal solutions for a class of accelerated first-order algorithms. They can be interpreted as discrete temporal versions of an inertial dynamic involving both viscous damping and Hessian-driven damping. The asymptotically vanishing viscous damping is linked to the accelerated gradient method of Nesterov while the Hessian driven damping makes it possible to significantly attenuate the oscillations. By treating the Hessian-driven damping as the time derivative of the gradient term, this gives, in discretized form, first-order algorithms. These results complement the previous work of the authors where it was shown the fast convergence of the values, and the fast convergence towards zero of the gradients.

In a Hilbert setting, we consider a class of inertial proximal algorithms for nonsmooth convex optimization, with fast convergence properties. They can be obtained by time discretization of inertial gradient dynamics which have been rescaled in time. We will rely specifically on the recent developement linking Nesterov's accelerated method with vanishing damping inertial dynamics. Doing so, we somehow improve and obtain a dynamical interpretation of the seminal papers of Güler on the convergence rate of the proximal methods for convex optimization.

No abstract