A Unified Contraction Analysis of a Class of Distributed Algorithms for Composite Optimization

Abstract

We study distributed composite optimization over networks: agents minimize the sum of a smooth (strongly) convex functionthe agents' sum-utility-plus a nonsmooth (extended-valued) convex one. We propose a general algorithmic framework for such a class of problems and provide a unified convergence analysis leveraging the theory of operator splitting. Our results unify several approaches proposed in the literature of distributed optimization for special instances of our formulation. Distinguishing features of our scheme are: (i) when the agents' functions are strongly convex, the algorithm converges at a linear rate, whose dependencies on the agents' functions and the network topology are decoupled, matching the typical rates of centralized optimization; (ii) the step-size does not depend on the network parameters but only on the optimization ones; and (iii) the algorithm can adjust the ratio between the number of communications and computations to achieve the same rate of the centralized proximal gradient scheme (in terms of computations). This is the first time that a distributed algorithm applicable to composite optimization enjoys such properties.

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

“…Note that to be used, A must be returned as nonsingular. More details of Chebyshev acceleration applied to the ABC-Algorithm along with some numerical results can be found in [1], [2].…”

“…The results of this work have been partially presented in [1]. While preparing the final version of this manuscript, we noticed the arxiv submission [26], which is an independent and parallel work (cf.…”

Distributed algorithms for scalable proximity operator computation and application to video denoising

“…The aforementioned algorithms were originally made available with singlenode implementations, which may be suboptimal or even unsuitable, when dealing with massive datasets. Therefore, various asynchronous or distributed extensions have been proposed [16,27,18,28,29], where each term is handled independently by a processing unit and the convergence toward an aggregate solution to the optimization problem is ensured via a suitable communication strategy between those processing units. However, the convergence analysis of primal-dual distributed algorithms is usually based on fixed-point theory, that require specific probabilistic assumptions on the block update rule.…”

Block Distributed 3MG Algorithm and its Application to 3D Image Restoration

“…This raises challenging questions, in terms of convergence analysis, as the communication delays may introduce instabilities. A plethora of recent works have focused on proposing distributed optimization algorithms with assessed convergence, based on stochastic proximal primal [13,14] or primal-dual [15,16,17,18,19,20] techniques. However, as they rely on the formulation of dual instances of a stochastic coordinate descent strategy, those algorithms are limited to convex (sometimes even strongly convex) optimization and often require specific probabilistic assumptions on the block update rule difficult to meet in practice.…”