Randomized gossip algorithms

Abstract

Abstract-Motivated by applications to sensor, peer-to-peer, and ad hoc networks, we study distributed algorithms, also known as gossip algorithms, for exchanging information and for computing in an arbitrarily connected network of nodes. The topology of such networks changes continuously as new nodes join and old nodes leave the network. Algorithms for such networks need to be robust against changes in topology. Additionally, nodes in sensor networks operate under limited computational, communication, and energy resources. These constraints have motivated the design of "gossip" algorithms: schemes which distribute the computational burden and in which a node communicates with a randomly chosen neighbor.We analyze the averaging problem under the gossip constraint for an arbitrary network graph, and find that the averaging time of a gossip algorithm depends on the second largest eigenvalue of a doubly stochastic matrix characterizing the algorithm. Designing the fastest gossip algorithm corresponds to minimizing this eigenvalue, which is a semidefinite program (SDP). In general, SDPs cannot be solved in a distributed fashion; however, exploiting problem structure, we propose a distributed subgradient method that solves the optimization problem over the network.The relation of averaging time to the second largest eigenvalue naturally relates it to the mixing time of a random walk with transition probabilities derived from the gossip algorithm. We use this connection to study the performance and scaling of gossip algorithms on two popular networks: Wireless Sensor Networks, which are modeled as Geometric Random Graphs, and the Internet graph under the so-called Preferential Connectivity (PC) model.

Fast Decentralized Averaging via Multi-scale Gossip

“…Throughout this paper when we refer to a random geometric graph, we mean one with the connectivity r con (n). Boyd, Ghosh, Prabhakar, and Shah [3] studied scaling laws for pairwise randomized gossip on random geometric graphs and found that communication cost scales as Θ( n 2 log n log −1 ) messages even if the algorithm is optimized with respect to the topology.…”

“…As explained in [3] this facilitates the analysis of gossip algorithms but does not force the actual implementation to be sequential. Multiple communication events happen in parallel.…”

“…At each gossip iteration, a random connected subset S(t) ⊂ V of nodes exchange their current estimates, x i (t), and locally compute the update x i (t + 1) = 1 |S(t)| j∈S(t) x j (t). In the original algorithm, described in [1] and revisited in [2] and [3], |S(t)| = 2 at every iteration, and the pair of nodes that update are neighbors in G.…”

Distributed parameter estimation in unreliable sensor networks via broadcast gossip algorithms

“…A convenient way when dealing with agreement in unreliable networks is to exchange messages by an asynchronous framework. The best known one is gossip-based algorithms which have attracted considerable recent attention [5]- [10]. In addition, limitations on the sensor cost, bandwidth, and energy budget dictate that information transmitted between sensors has to be quantized in practice [11]- [14].…”

Distributed Particle Filter for Target Tracking in Sensor Networks

“…Since each sensor independently processes its own particles, we need to fuse the sensors' information. To achieve distributed information fusion, a consensus algorithm [4] and a gossip method [14,15] are adopted. The consensus algorithm is synchronous, that is, all sensors activate and communicate with their neighbors at each iteration.…”