Asynchronous distributed optimization using a randomized alternating direction method of multipliers

Abstract

Consider a set of networked agents endowed with private cost functions and seeking to find a consensus on the minimizer of the aggregate cost. A new class of random asynchronous distributed optimization methods is introduced. The methods generalize the standard Alternating Direction Method of Multipliers (ADMM) to an asynchronous setting where isolated components of the network are activated in an uncoordinated fashion. The algorithms rely on the introduction of randomized Gauss-Seidel iterations of a Douglas-Rachford operator for finding zeros of a sum of two monotone operators. Convergence to the sought minimizers is provided under mild connectivity conditions. Numerical results sustain our claims.

Using big steps in coordinate descent primal-dual algorithms

“…However, in the approach of [4], [5], [6] the convergence conditions are identical to the ones of the brute method, the one without coordinate descent. These conditions involve the global Lipschitz constant of the gradient the differentiable term instead than its coordinate-wise Lipschitz constants.…”

“…The key idea behind the convergence proof of [4] is to establish the so-called stochastic Fejér monotonicity of the sequence of iterates as noted by [5]. In a more general setting than [4], Combettes et al in [5] and Bianchi et al [6] extend the proof to the so-called α-averaged operators, which include FNE operators as a special case. This generalization allows to apply the coordinate descent principle to a broader class of primal-dual algorithms which is no longer restricted to the ADMM or the Douglas Rachford algorithm.…”

“…It is known that the ADMM can be written as a fixed point algorithm of a FNE operator, which led the authors of [4] to propose a coordinate descent version of ADMM with application to distributed optimization. The key idea behind the convergence proof of [4] is to establish the so-called stochastic Fejér monotonicity of the sequence of iterates as noted by [5]. In a more general setting than [4], Combettes et al in [5] and Bianchi et al [6] extend the proof to the so-called α-averaged operators, which include FNE operators as a special case.…”

“…In the case where the optimization problem contains a separable and a strongly convex function, Zhang and Xiao [3] introduce a stochastic CD primal-dual algorithm and analyze its convergence rate. In 2013, Iutzeler et al [4] proved that random coordinate descent can be successfully applied to fixed point iterations of firmly non-expansive (FNE) operators. It is known that the ADMM can be written as a fixed point algorithm of a FNE operator, which led the authors of [4] to propose a coordinate descent version of ADMM with application to distributed optimization.…”

“…In 2013, Iutzeler et al [4] proved that random coordinate descent can be successfully applied to fixed point iterations of firmly non-expansive (FNE) operators. It is known that the ADMM can be written as a fixed point algorithm of a FNE operator, which led the authors of [4] to propose a coordinate descent version of ADMM with application to distributed optimization. The key idea behind the convergence proof of [4] is to establish the so-called stochastic Fejér monotonicity of the sequence of iterates as noted by [5].…”