A unified differential equation solver approach for separable convex optimization: splitting, acceleration and nonergodic rate

Abstract

This paper provides a self-contained ordinary differential equation solver approach for linearly constrained separable convex optimization problems and presents several classes of new accelerated primal-dual splitting methods. A novel dynamical system with built-in time rescaling factors is first introduced and the exponential decay of a tailored Lyapunov function is established. Numerical discretizations of the continuous time model are considered to construct new algorithms for the original optimization problem and also analyzed via a unified discrete Lyapunov function. Specifically, those schemes lead to new proximal alternating direction methods of multipliers with linearizations and extrapolations, and accelerated nonergodic rates are proved respectively for convex and partially strong convex objectives.

Faster augmented Lagrangian method for solving convex optimization problems with linear constraints

“…The author provided O(1/k), O(1/k 2 ), and linear non-ergodic convergence rates for primal-dual gap under convexity, f or g * being strongly convex, and both f and g * being strongly convex, respectively. Luo [29] also obtained the same non-ergodic rate as in [28] using continuous dynamical system approaches when solving (P2).…”

An efficient semismooth Newton-AMG-based inexact primal-dual algorithm for generalized transport problems

“…By the celebrated stability result of linear inequality system [68,76], we have global linear convergence for ADMM [33]. For general convex objectives, the provable nonergodic rate of many accelerated variants of (linearized) ALM and ADMM is O(1/k); see [54,63,64,91]. The method [91, Algorithm 1] possesses a faster sublinear rate O(1/k 2 ) but involves a large scale quadratic programming (of dimension n 2 ) of the primal variable.…”

“…In the sequel, we investigate the performance of our IPD-SsN-AMG method on specific problems including optimal transport, Birkhoff projection and partial optimal transport. Also, comparisons with the semismooth Newton-based augmented Lagrangian methods proposed in [55,56] and the accelerated ADMM method in [64] will be presented, under the same stopping condition (7.2) with KKT Tol = 10 −6 .…”

“…For convenience, we abbreviate these two methods simply as ALM-SsN-PCG. The method in [64], denoted shortly by Acc-ADMM, possesses sublinear rates O(1/k) and O(1/k 2 ) respectively for convex and partially strongly convex objectives.…”