Quantized consensus ADMM for multi-agent distributed optimization

Abstract

Multi-agent distributed optimization over a network minimizes a global objective formed by a sum of local convex functions using only local computation and communication. We develop and analyze a quantized distributed algorithm based on the alternating direction method of multipliers (ADMM) when inter-agent communications are subject to finite capacity and other practical constraints. While existing quantized ADMM approaches only work for quadratic local objectives, the proposed algorithm can deal with more general objective functions (possibly non-smooth) including the LASSO. Under certain convexity assumptions, our algorithm converges to a consensus within log 1+η Ω iterations, where η > 0 depends on the local objectives and the network topology, and Ω is a polynomial determined by the quantization resolution, the distance between initial and optimal variable values, the local objective functions and the network topology. A tight upper bound on the consensus error is also obtained which does not depend on the size of the network. Index TermsMulti-agent distributed optimization, quantization, alternating direction method of multipliers (ADMM), linear convergence.

Convergence Bounds for Compressed Gradient Methods with Memory Based Error Compensation

“…This rounding quantizer is a -compressor with = d · ∆ 2 /4, [27], [28], [29], [30]. In addition, if gradients are bounded, the sign compressor [4], the K-greedy quantizer [6] and the dynamic gradient quantizer [2], [6] are all -compressors.…”

Distributed Discrete-Time Optimization in Multiagent Networks Using Only Sign of Relative State

“…Many existing algorithms to solve distributed optimization in multi-agent networks generally comprise two parts, see e.g. [6]- [13] and the references therein. One is to drive all agents to reach a consensus, and the other is to push the consensus value toward an optimal solution of the optimization problem.…”

“…In this case, each agent can only access one bit of relative state information from each of its neighbors. Clearly, this is very different from the quantized settings in [10]- [13], which use the quantized version of the absolute state, and dynamic quantizers are essential for computing an exact optimal solution [10]. With static quantizers, each node can only find a sub-optimal solution [11]- [13].…”

“…We show in this work that knowing only the sign of relative state (which is essentially 1 one bit information for each neighbor) is sufficient to obtain an exact optimal solution. Other distributed optimization algorithms include the ADMM-based methods [13], [16], [17], and proximal gradient methods [18]. Note that these algorithms need much more than one bit of information per time from its neighbors.…”

Optimized Quantization in Distributed Graph Signal Processing

“…However, they do not specifically consider distributed processing of graph signals with limited communication between nodes. There are also many works that focused on solving consensus problems in a network subject to quantized communication [11,12,13,14,15,16], but they merely focus on average computation, and not more general processing tasks.…”