Fenchel Duality Theory and a Primal-Dual Algorithm on Riemannian Manifolds

Abstract

This paper introduces a new notion of a Fenchel conjugate, which generalizes the classical Fenchel conjugation to functions defined on Riemannian manifolds. We investigate its properties, e.g., the Fenchel–Young inequality and the characterization of the convex subdifferential using the analogue of the Fenchel–Moreau Theorem. These properties of the Fenchel conjugate are employed to derive a Riemannian primal-dual optimization algorithm and to prove its convergence for the case of Hadamard manifolds under appropriate assumptions. Numerical results illustrate the performance of the algorithm, which competes with the recently derived Douglas–Rachford algorithm on manifolds of nonpositive curvature. Furthermore, we show numerically that our novel algorithm may even converge on manifolds of positive curvature.

The difference of convex algorithm on Riemannian manifolds

“…Thus, there exists η ∈ T * p M such that h * (p, η) < +∞ and g * (p, η) = +∞. By using (3) we have h * (p, η) − g * (p, η) = h * (p, η) − (+∞) = −∞, which contradicts the equality in (9) and the first statement is proved. Since dom(h * (p, •)) ⊆ dom(g * (p, •)), it follows from (8) that g * (p, y) < +∞.…”

“…In addition to the theoretical issues addressed, which have an interest of their own, the Riemannian machinery provides support to design efficient algorithms to solve optimization problem in this setting; papers on this subject include [1,20,26,[36][37][38]41,49,57,62] and references therein. In this sense, the concept of conjugate of a convex function was recently presented in the Riemannian setting, which is an important tool in convex analysis and play an important role in the theory of duality on Riemannian manifolds, see [8,9]. In particular, this definition enables us to propose a Riemannian version of DCA.…”

Manopt.jl: Optimization on Manifolds in Julia

“…The Riemannian Chambolle-Pock algorithm presented in Bergmann et al (2021) was developed using Manopt.jl. Based on this theory and algorithm, a higher-order algorithm was introduced in Diepeveen & Lellmann (2021).…”

“…Algorithms include the derivative-free Particle Swarm and Nelder-Mead algorithms, as well as classical gradient, conjugate gradient and stochastic gradient descent. Furthermore, quasi-Newton methods like a Riemannian L-BFGS (Huang et al, 2015) and nonsmooth optimization algorithms like a Cyclic Proximal Point Algorithm (Bačák, 2014), a (parallel) Douglas-Rachford algorithm and a Chambolle-Pock algorithm (Bergmann et al, 2021) are provided, together with several basic cost functions, gradients and proximal maps as well as debug and record capabilities.…”

“…Alternating Gradient Descent (alternating_gradient_descent) • Chambolle-Pock (ChambollePock)(Bergmann et al, 2021) • Conjugate Gradient Descent (conjugate_gradient_descent), which includes eight direction update rules using the coefficient keyword: SteepestDirectionUpdateR ule, ConjugateDescentCoefficient. DaiYuanCoefficient, FletcherReevesCoe fficient, HagerZhangCoefficient, HeestenesStiefelCoefficient, LiuStorey Coefficient, and PolakRibiereCoefficient• Cyclic Proximal Point (cyclic_proximal_point)(Bačák, 2014) • (parallel) Douglas-Rachford (DouglasRachford) • Gradient Descent (gradient_descent), including direction update rules (IdentityU pdateRule for the classical gradient descent) to perform MomentumGradient, Averag eGradient, and Nesterov types • Nelder-Mead (NelderMead) • Particle-Swarm Optimization (particle_swarm)(Borckmans et al, 2010) • Quasi-Newton (quasi_Newton), with BFGS, DFP, Broyden and a symmetric rank 1 (SR1) update, their inverse updates as well as a limited memory variant of (inverse) BFGS (using the memory keyword)(Huang et al, 2015) • Stochastic Gradient Descent (stochastic_gradient_descent) • Subgradient Method (subgradient_method) • Trust Regions (trust_regions), with inner Steihaug-Toint (truncated_conjugate_ gradient_descent) solver(Absil et al, 2006) …”