Fast Proximal Methods via Time Scaling of Damped Inertial Dynamics

Abstract

In a Hilbert setting, we consider a class of inertial proximal algorithms for nonsmooth convex optimization, with fast convergence properties. They can be obtained by time discretization of inertial gradient dynamics which have been rescaled in time. We will rely specifically on the recent developement linking Nesterov's accelerated method with vanishing damping inertial dynamics. Doing so, we somehow improve and obtain a dynamical interpretation of the seminal papers of Güler on the convergence rate of the proximal methods for convex optimization.

Fast convex optimization via inertial dynamics combining viscous and Hessian-driven damping with time rescaling

“…3. For a general λ k , Attouch, Chbani, and Riahi first considered in [12] the case α k = 1 − α k . They proved that under a growth condition on λ k , we have the estimate f…”

“…Our work is part of the active research stream that studies the close link between continuous dissipative dynamical systems and optimization algorithms. In general, the implicit temporal discretization of continuous gradient-based dynamics provides proximal algorithms that benefit from similar asymptotic convergence properties, see [29] for a systematic study in the case of first-order evolution systems, and [5,6,8,12,11,19,20,21] for some recent results concerning second-order evolution equations. The main object of our study is the second-order in time differential equation (IGS) γ,β,bẍ (t) + γ(t)ẋ(t) + β(t)∇ 2 f (x(t))ẋ(t) + b(t)∇f (x(t)) = 0,…”

“…This is where the coefficient b(t) comes in as a factor of ∇f (x(t)). In [12] [11], in the case of general coefficients γ(•) and b(•) without the Hessian damping, the authors made in-depth study. In the case γ(t) = α t , they proved that under appropriate conditions on α and…”

Convergence Rate of Inertial Proximal Algorithms with General Extrapolation and Proximal Coefficients

“…This makes the link with the historical aspects and the recent developments based on Nesterov acceleration method described in the previous section. For recent developments concerning the rich relations between inertial gradient systems with time-dependent friction and the inertial proximal-based algorithms, one can consult [2], [5], [7], [8], [14], [17], [37], [36].…”

“…It was obtained by the authors in [7, Theorem 7.1]. Theorem 1.2 (Attouch-Chbani-Riahi [7]). Take α ≥ 1.…”