Adaptive Diffusion with Compressed Communication

Carpentiero, Marco; Matta, Vincenzo; Sayed, Ali H.

doi:10.1109/icassp43922.2022.9747154

Cited by 10 publications

(13 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Randomized quantizers Q(•) 1 will be employed instead of deterministic quantizers. As explained in [30], deterministic quantizers can lead to severe estimation biases in inference problems. To overcome this issue, randomized quantizers Q(•) are commonly used to compensate for the bias (on average, over time) [12], [29]- [33].…”

Section: Randomized Quantizersmentioning

confidence: 99%

“…quantization is used in (28b) to leverage possible correlation between subsequent iterates [30]. In this case, instead of communicating compressed versions of the estimates ψ k,i , the prediction error ψ k,i − φ k,i−1 is quantized at agent k and then transmitted [12], [29]- [33].…”

Section: A Differential Randomized Quantization Approachmentioning

confidence: 99%

Section: A Differential Randomized Quantization Approachmentioning

confidence: 99%

“…where 1 N is the N × 1 vector of all ones, then solving problem (3) will be equivalent to solving the well-studied consensus problem (30). Different algorithms for solving (30) over strongly-connected networks have been proposed [8]- [10], [13]- [15]. By choosing an N × N doubly-stochastic matrix A = [a k ] satisfying: the diffusion strategy for instance would take the form [8]- [10]:…”

Section: A Differential Randomized Quantization Approachmentioning

confidence: 99%

“…) with a set of combination coefficients {a k } satisfying (31), we obtain a new quantized diffusion-type approach for solving the consensus optimization problem (30).…”

Section: A Differential Randomized Quantization Approachmentioning

confidence: 99%

See 4 more Smart Citations

Quantization for decentralized learning under subspace constraints

Nassif¹,

Vlaski²,

Carpentiero³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In this paper, we consider decentralized optimization problems where agents have individual cost functions to minimize subject to subspace constraints that require the minimizers across the network to lie in low-dimensional subspaces. This constrained formulation includes consensus or single-task optimization as special cases, and allows for more general task relatedness models such as multitask smoothness and coupled optimization. In order to cope with communication constraints, we propose and study an adaptive decentralized strategy where the agents employ differential randomized quantizers to compress their estimates before communicating with their neighbors. The analysis shows that, under some general conditions on the quantization noise, and for sufficiently small step-sizes µ, the strategy is stable both in terms of mean-square error and average bit rate: by reducing µ, it is possible to keep the estimation errors small (on the order of µ) without increasing indefinitely the bit rate as µ → 0. Simulations illustrate the theoretical findings and the effectiveness of the proposed approach, revealing that decentralized learning is achievable at the expense of only a few bits.

show abstract

Section: Randomized Quantizersmentioning

confidence: 99%

Section: A Differential Randomized Quantization Approachmentioning

confidence: 99%

Section: A Differential Randomized Quantization Approachmentioning

confidence: 99%

Section: A Differential Randomized Quantization Approachmentioning

confidence: 99%

“…) with a set of combination coefficients {a k } satisfying (31), we obtain a new quantized diffusion-type approach for solving the consensus optimization problem (30).…”

Section: A Differential Randomized Quantization Approachmentioning

confidence: 99%

See 3 more Smart Citations