Distributed iterative thresholding for ℓ₀/ℓ₁-regularized linear inverse problems

Abstract

The 0 / 1 -regularized least-squares approach is used to deal with linear inverse problems under sparsity constraints, which arise in mathematical and engineering fields. In particular, multiagent models have recently emerged in this context to describe diverse kinds of networked systems, ranging from medical databases to wireless sensor networks. In this paper, we study methods for solving 0 / 1 -regularized leastsquares problems in such multiagent systems. We propose a novel class of distributed protocols based on iterative thresholding and input driven consensus techniques, which are well-suited to work in-network when the communication to a central processing unit is not allowed. Estimation is performed by the agents themselves, which typically consist of devices with limited computational capabilities. This motivates us to develop low-complexity and low-memory algorithms that are feasible in real applications. Our main result is a rigorous proof of the convergence of these methods in regular networks. We introduce a suitable distributed, regularized, least-squares functional, and we prove that our algorithms reach their minima using results from dynamical systems theory. Furthermore, we propose numerical comparisons with the alternating direction method of multipliers and the distributed subgradient methods, in terms of performance, complexity, and memory usage. We conclude that our techniques are preferable for their good memory-accuracy tradeoff.Index Terms-Distributed optimization, input driven consensus algorithms, multi-agent systems, regularized linear inverse problems, sparse estimation. Chiara Ravazzi received the B.Sc.

A Proximal Gradient Algorithm for Decentralized Composite Optimization

“…In PG-EXTRA, we use the Metropolis constant edge weight for W andW = I+W 2 , and a constant step size α. We compare PG-EXTRA with the recent work DISTA [25], [26], which has two free parameters: temperature parameter q ∈ (0, 1) and τ ≤ M (i) −2 2 , ∀i. We have hand optimized q and show the effect of τ in our experiment.…”

An overview of multi-processor approximate message passing

“…The partitioning style depends on data availability, computational considerations, and privacy concerns. Both types of partitioning result in reduced storage requirements per node and faster computation [8][9][10][11][12][13][14][15][16].…”

Wireless Sensor Networks

“…A secure average consensus-based time synchronization protocol is proposed in [28]. Consensus-based D-LS Using ADMM [29] DiCE: introducing new consensus constraints to reduce exchanged message [30] Fast-DiCE: fast convergence by using Nesterov's optimal gradient descend method [31] Consensus-based D-LS with quantization and communication noise [32] Consensus-based D-TLS [34] IPI-based D-TLS: reduced computational complexity [35] Two stage consensus-based solution for L norm regularization [36] Three stage solution with low complexity and memory requirement [37] PSSE: iteratively exclude abnormal values [38] Consensus-based framework from both attacker and defender aspects [39] …”

“…A model-robust adaptation is also adopted to control the approximation error caused by spatial quantization. An iterative thresholding and input driven consensus-based three-step method appears in [37] with low complexity and memory requirement. In order to obtain robust power state estimation, [38] proposes a distributed PSSE estimator based on ADMM to iteratively exclude the abnormal values.…”