Improved convergence rates and trajectory convergence for primal-dual dynamical systems with vanishing damping

Boţ, Radu Ioan; Nguyen, Dang-Khoa

doi:10.1016/j.jde.2021.09.021

Cited by 16 publications

(24 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is the first result which provides the convergence of the sequence of iterates generated by a fast algorithm for linearly constrained convex optimization problems without additional assumptions such as strong convexity. All convergence and convergence rate results of this paper are compatible with the ones obtained in [8] in the continuous setting.…”

Section: Our Contributionssupporting

confidence: 85%

“…We consider as starting point a second-order dynamical system with asymptotic vanishing damping term associated with the optimization problem (1.1). This dynamical system is formulated in terms of the augmented Lagrangian and it has been studied in [8]. By an appropriate time discretization this system gives rise to an inertial primaldual numerical algorithm, which allows a flexible choice of the inertial parameters.…”

Section: Our Contributionsmentioning

confidence: 99%

“…Zeng, Lei, and Chen (in [30]) and He, Hu, and Fang (in [16]) have investigated a dynamical system with asymptotic vanishing damping attached to (1.1), and have shown a convergence rate of order O 1/t 2 for the primal-dual gap, while Attouch, Chbani, Fadili and Riahi have considered in [2] a more general dynamical system with time rescaling. More recently, for a primal-dual dynamical system formulated in the spirit of [2,16,30], Boţ and Nguyen have obtained in [8] fast convergence rates for the primal-dual gap, the feasibility measure and the objective function value along the generated trajectory, and, additionally, have proved asymptotic convergence guarantees for the primal-dual trajectory to a primal-dual optimal solution.…”

mentioning

confidence: 99%

“…The aim of this paper is to propose a numerical algorithm for solving (1.1), which results from the discretization of the dynamical system in [8], exhibits fast convergence rates for the primal-dual gap, the feasibility measure, and the objective function value as well as convergence of the sequence of iterates without additional assumptions such as strong convexity. Although there is an obvious interplay between continuous time dissipative dynamical systems and their discrete counterparts, one cannot directly and straightforwardly transfer asymptotic results from the continuous setting to numerical algorithms, thus, a separate analysis is needed for the latter.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Fast Augmented Lagrangian Method in the convex regime with convergence guarantees for the iterates

2022

Self Cite

View full text Add to dashboard Cite

This work aims to minimize a continuously differentiable convex function with Lipschitz continuous gradient under linear equality constraints. The proposed inertial algorithm results from the discretization of the second-order primal-dual dynamical system with asymptotically vanishing damping term addressed by Boţ and Nguyen (J. Differential Equations 303:369–406, 2021), and it is formulated in terms of the Augmented Lagrangian associated with the minimization problem. The general setting we consider for the inertial parameters covers the three classical rules by Nesterov, Chambolle–Dossal and Attouch–Cabot used in the literature to formulate fast gradient methods. For these rules, we obtain in the convex regime convergence rates of order $${\mathcal {O}}\left( 1/k^{2} \right) $$ O 1 / k 2 for the primal-dual gap, the feasibility measure, and the objective function value. In addition, we prove that the generated sequence of primal-dual iterates converges to a primal-dual solution in a general setting that covers the two latter rules. This is the first result which provides the convergence of the sequence of iterates generated by a fast algorithm for linearly constrained convex optimization problems without additional assumptions such as strong convexity. We also emphasize that all convergence results of this paper are compatible with the ones obtained in Boţ and Nguyen (J. Differential Equations 303:369–406, 2021) in the continuous setting.

show abstract

Section: Our Contributionssupporting

confidence: 85%

Section: Our Contributionsmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Fast Augmented Lagrangian Method in the convex regime with convergence guarantees for the iterates

2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, more continuous time models are proposed for the separable case (1); see [3,10,11,12,40,41,50,59,62], where the solution existence, uniqueness and perturbation effect have been considered. However, none of these works proposed new primal-dual splitting algorithms with provable nonergodic convergence rates for solving the original optimization problem (1).…”

Section: Dynamical System Approachmentioning

confidence: 99%

A unified differential equation solver approach for separable convex optimization: splitting, acceleration and nonergodic rate

Luo¹,

Zhang²

2021

Preprint

View full text Add to dashboard Cite

This paper provides a self-contained ordinary differential equation solver approach for linearly constrained separable convex optimization problems and presents several classes of new accelerated primal-dual splitting methods. A novel dynamical system with built-in time rescaling factors is first introduced and the exponential decay of a tailored Lyapunov function is established. Numerical discretizations of the continuous time model are considered to construct new algorithms for the original optimization problem and also analyzed via a unified discrete Lyapunov function. Specifically, those schemes lead to new proximal alternating direction methods of multipliers with linearizations and extrapolations, and accelerated nonergodic rates are proved respectively for convex and partially strong convex objectives.

show abstract

Time Rescaling of a Primal-Dual Dynamical System with Asymptotically Vanishing Damping

Hulett

Nguyen

2023

Appl Math Optim

Self Cite

View full text Add to dashboard Cite

In this work, we approach the minimization of a continuously differentiable convex function under linear equality constraints by a second-order dynamical system with an asymptotically vanishing damping term. The system under consideration is a time rescaled version of another system previously found in the literature. We show fast convergence of the primal-dual gap, the feasibility measure, and the objective function value along the generated trajectories. These convergence rates now depend on the rescaling parameter, and thus can be improved by choosing said parameter appropriately. When the objective function has a Lipschitz continuous gradient, we show that the primal-dual trajectory asymptotically converges weakly to a primal-dual optimal solution to the underlying minimization problem. We also exhibit improved rates of convergence of the gradient along the primal trajectories and of the adjoint of the corresponding linear operator along the dual trajectories. We illustrate the theoretical outcomes and also carry out a comparison with other classes of dynamical systems through numerical experiments.

show abstract

Improved convergence rates and trajectory convergence for primal-dual dynamical systems with vanishing damping

Cited by 16 publications

References 41 publications

Fast Augmented Lagrangian Method in the convex regime with convergence guarantees for the iterates

Fast Augmented Lagrangian Method in the convex regime with convergence guarantees for the iterates

A unified differential equation solver approach for separable convex optimization: splitting, acceleration and nonergodic rate

Time Rescaling of a Primal-Dual Dynamical System with Asymptotically Vanishing Damping

Contact Info

Product

Resources

About