Dynamic smoothness parameter for fast gradient methods

Abstract

We present and computationally evaluate a variant of the fast gradient method by Nesterov that is capable of exploiting information, even if approximate, about the optimal value of the problem. This information is available in some applications, among which the computation of bounds for hard integer programs. We show that dynamically changing the smoothness parameter of the algorithm using this information results in a better convergence profile of the algorithm in practice.

Essentials of numerical nonsmooth optimization

“…As far as the classic approach is concerned, reference books, whose reading is strongly suggested, are Shor (1985), Polyak (1987). In more recent years, the interest in subgradient-type methods was renewed, thanks to the Mirror Descent Algorithm introduced by Nemirowski and Yudin (see also Beck and Teboulle 2003), and to some papers by Nesterov (2005Nesterov ( , 2009a (see also the variant Frangioni et al 2018). Very recent developments are in Dvurechensky et al (2020).…”

“…for some σ > 0. Minimization of the smooth function f μ (x) is then pursued via a gradienttype method (see also Frangioni et al 2018 for a discussion on tuning of the smoothing parameter μ. )…”