Accelerated Optimization on Riemannian Manifolds via Projected Variational Integrators

Abstract

A variational formulation of accelerated optimization on normed spaces was recently introduced by considering a specific family of time-dependent Bregman Lagrangian and Hamiltonian systems whose corresponding trajectories converge to the minimizer of the given convex function at an arbitrary accelerated rate of O(1 t p ). This framework has been exploited using time-adaptive geometric integrators to design efficient explicit algorithms for symplectic accelerated optimization. It was observed that geometric discretizations were substantially less prone to stability issues, and were therefore more robust, reliable, and computationally efficient. More recently, this variational framework has been extended to the Riemannian manifold setting by considering a more general family of time-dependent Bregman Lagrangian and Hamiltonian systems on Riemannian manifolds. It is thus natural to develop time-adaptive Hamiltonian variational integrators for accelerated optimization on Riemannian manifolds. In the past, Hamiltonian variational integrators have been constructed with holonomic constraints, but the resulting algorithms were implicit in nature, which significantly increased their cost per iteration. In this paper, we will test the performance of explicit methods based on Hamiltonian variational integrators combined with projections that constrain the numerical solution to remain on the constraint manifold.

Nesterov Acceleration for Riemannian Optimization

“…For both g-convex and g-strongly convex cases, [1] proposed ODEs that can model accelerated methods on Riemannian manifolds given K min and D. [26] extended this result and developed a variational framework. Time-discretization methods for such ODEs on Riemannian manifolds have recently been of considerable interest as well [27][28][29].…”