Time-adaptive Lagrangian Variational Integrators for Accelerated Optimization on Manifolds

Abstract

A variational framework for accelerated optimization was recently introduced on normed vector spaces and Riemannian manifolds in Wibisono et al. [38] and Duruisseaux and Leok [8]. It was observed that a careful combination of time-adaptivity and symplecticity in the numerical integration can result in a significant gain in computational efficiency. It is however well known that symplectic integrators lose their near energy preservation properties when variable time-steps are used. The most common approach to circumvent this problem involves the Poincaré transformation on the Hamiltonian side, and was used in Duruisseaux et al. [9] to construct efficient explicit algorithms for symplectic accelerated optimization. However, the current formulations of Hamiltonian variational integrators do not make intrinsic sense on more general spaces such as Riemannian manifolds and Lie groups. In contrast, Lagrangian variational integrators are well-defined on manifolds, so we develop here a framework for time-adaptivity in Lagrangian variational integrators and use the resulting geometric integrators to solve optimization problems on normed vector spaces and Lie groups.

Bregman dynamics, contact transformations and convex optimization

“…Moreover, as commented in the text, it will be interesting to study several optimization dynamics through the lens of Herglotz's variational principle. Finally, all the results presented here have a natural generalization to the case of general differentiable manifolds, and therefore we can extend our analysis with methods similar to those employed in [39][40][41].…”

“…Now we come to the main point of our work: in order to further illustrate the utility of contact transformations in a case of great current interest in the literature (see e.g. [37][38][39][40][41]), in the remainder of this section we focus on the Bregman dynamics and show how to use time-dependent contact transformation in order to re-write the Bregman dynamics in its most general form in such a way that it is clear that it is always derived from a separable Hamiltonian, and is thus amenable of simple, geometric and explicit discretizations by direct splitting.…”