Robust Convergence Analysis of Three-Operator Splitting

Abstract

Operator splitting methods solve composite optimization problems by breaking them into smaller sub-problems that can be solved sequentially or in parallel. In this paper, we propose a unified framework for certifying both linear and sublinear convergence rates for three-operator splitting (TOS) method under a variety of assumptions about the objective function. By viewing the algorithm as a dynamical system with feedback uncertainty (the oracle model), we leverage robust control theory to analyze the worst-case performance of the algorithm using matrix inequalities. We then show how these matrix inequalities can be used to verify sublinear/linear convergence of the TOS algorithm and guide the search for selecting the parameters of the algorithm (both symbolically and numerically) for optimal worst-case performance. We illustrate our results numerically by solving an input-constrained optimal control problem.

Convergence Analyses of Davis-Yin Splitting via Scaled Relative Graphs

“…Aragón-Artacho and Torregrosa-Belén proved convergence of the DYS iteration over a wider range of stepsizes [1], Dao and Phan analyzed a modified version of DYS iterations [12], and Pedregosa analyzed sublinear convergence rates of DYS iterations [25], but they did not analyze linear rates of convergence. Ryu, Taylor, Bergeling, and Giselsson [30] and Wang, Fazlyab, Chen, and Preciado [35] formulated the problem of computing tight contraction factors as SDPs using the performance estimation problem (PEP) and incremental quadratic constraint (IQC) approaches, but these are numerical results, not analytic convergence proofs in the classical sense. The only linear convergence rates of DYS iterations are those presented in the arXiv version of the original DYS paper [13].…”