On Variational Message Passing on Factor Graphs

Abstract

In this paper, it is shown how (naive and structured) variational algorithms may be derived from a factor graph by mechanically applying generic message computation rules; in this way, one can bypass error-prone variational calculus. In prior work by Bishop et al., Xing et al., and Geiger, directed and undirected graphical models have been used for this purpose. The factor graph notation amounts to simpler generic variational message computation rules; by means of factor graphs, variational methods can straightforwardly be compared to and combined with various other message-passing inference algorithms, e.g., Kalman filters and smoothers, iterated conditional modes, expectation maximization (EM), gradient methods, and particle filters. Some of those combinations have been explored in the literature, others seem to be new. Generic message computation rules for such combinations are formulated.

Variational stabilized linear forgetting in state-space models

“…This paper follows the variational message passing (VMP) formulation from [15], which defines a recipe for iterative message updates, such that the variational approximation is guaranteed to converge to a local minimum of the KL divergence. Fig.…”

“…3. Variational message updates are computed in accordance with [15], with shorthand notations for the mean 1 Derivations are available at http://biaslab.github.io/pdf/slf/supplement.pdf f (·) = E q(·) [f (·)], covariance Cov[f (·)] = Cov q(·) [f (·)], and average energy U[f (·)] = −E q(·) [log f (·)]. Table I summarizes the variational message updates for the Bernoulli node, with switch z ∈ {0, 1} and parametrized by π ∈ [0, 1] through node function f (z, π) = π z (1 − π) 1−z .…”

“…This is important, because signal modeling often involves an iterative search process for the best model. This paper develops a variational message passing (VMP) approach to SLF that is based on the variational Bayes method [14], [15], which is a well-known and principled technique [16] for approximate inference.…”

A variational message passing algorithm for sensor self-localization in wireless networks

“…The position posterior pdf p (x r |D) of any mobile sensor r ∈ V M can now be estimated via message passing methods [7]- [14]. Two common message passing methods adapted to graphs as depicted in Figure 2 are displayed in Figure 3: the SP algorithm, which implements BP [9], and VMP, which implements the variational Bayesian method [12].…”

“…For continuous hidden variables, evaluation of (6), (8) and (9) (see Figure 3a) can become arbitrarily complex [3], [7], [12], [13]. A way to control this is to restrict the messages passed between the nodes to be Gaussian [7].…”