Distributed Variational Bayesian Algorithms Over Sensor Networks

Abstract

Distributed inference/estimation in Bayesian framework in the context of sensor networks has recently received much attention due to its broad applicability. The variational Bayesian (VB) algorithm is a technique for approximating intractable integrals arising in Bayesian inference. In this paper, we propose two novel distributed VB algorithms for general Bayesian inference problem, which can be applied to a very general class of conjugate-exponential models. In the first approach, the global natural parameters at each node are optimized using a stochastic natural gradient that utilizes the Riemannian geometry of the approximation space, followed by an information diffusion step for cooperation with the neighbors. In the second method, a constrained optimization formulation for distributed estimation is established in natural parameter space and solved by alternating direction method of multipliers (ADMM). An application of the distributed inference/estimation of a Bayesian Gaussian mixture model is then presented, to evaluate the effectiveness of the proposed algorithms. Simulations on both synthetic and real datasets demonstrate that the proposed algorithms have excellent performance, which are almost as good as the corresponding centralized VB algorithm relying on all data available in a fusion center.

The Algorithms of Distributed Learning and Distributed Estimation about Intelligent Wireless Sensor Network

“…For the Bayesian learning problem in wireless sensor networks, references [10,13,19] systematically studied how to solve the Bayesian learning problem by using the variational Bayes method in a distributed environment. For the problem of Bayesian inference and estimation on the network, reference [10] proposed a general framework of distributed variational seeing algorithm for conjugate exponential family models. For the joint sparse signal recovery problem in sensor networks, a distributed variational Bayesian algorithm based on quantized communication and inaccurate ADM is proposed in [19].…”

“…Therefore, the real-time online information processing, data analysis, and dynamic optimization of agents are accomplished [8,9]. Intelligent network is a highly intelligent distributed network system, in which each agent is able to self-recognition, self-judgment, and self-adjusting by network topology [10]. At individual level, intelligent network requires each agent not only can perceive its environment information, but also can perform network communication through its connection to other agents to realize ongoing reinforcement learning [11].…”

Levenberg-marquardt algorithm to identify the fault analysis for industrial applications

“…A utilization of the dispersed inference/estimation of a Bayesian Gaussian mixture method is then displayed, to assess the effectiveness of the aimed algorithms. Simulations on both real datasets and synthetic exhibit that the aimed algorithms have brilliant execution, which is nearly in the same class as the relating centralized VB algorithm depending on all information accessible in a fusion center [5]. The issue of robust transmission of detected information through an immense field of little and vulnerable sensors towards sink nodes.…”

Distributed Boosting Variational Inference Algorithm Over Multi-Agent Networks

“…Some distributed variational inference (DVI) algorithms have been proposed to estimate the posterior over networks [18]- [20]. DVI with Gaussian mixture model (DVI-GMM) [18], [20] is proposed to approximate the multi-modality posterior by distributed optimization methods. Although DVI-GMM can successfully capture the multi-modality posterior over networks, it still faces some challenges from both the computational complexity and theory.…”