Belief Propagation for Linear Programming

Abstract

Belief Propagation (BP) is a popular, distributed heuristic for performing MAP computations in Graphical Models. BP can be interpreted, from a variational perspective, as minimizing the Bethe Free Energy (BFE). BP can also be used to solve a special class of Linear Programming (LP) problems. For this class of problems, MAP inference can be stated as an integer LP with an LP relaxation that coincides with minimization of the BFE at "zero temperature". We generalize these prior results and establish a tight characterization of the LP problems that can be formulated as an equivalent LP relaxation of MAP inference. Moreover, we suggest an efficient, iterative annealing BP algorithm for solving this broader class of LP problems. We demonstrate the algorithm's performance on a set of weighted matching problems by using it as a cutting plane method to solve a sequence of LPs tightened by adding "blossom" inequalities.

Loop calculus and bootstrap-belief propagation for perfect matchings on arbitrary graphs

“…This was shown for PMs on bipartite graphs in [7,8,9] and extended to general weighted matchings in [11]. And in [20], the MWPM was shown to be an example of a problem where the LP optima and iterative BP solution are equivalent (when the latter converges).…”

“…In such cases, the LC can be used to improve upon the BP estimate for the PF (when the BP estimate is not tight) by re-summing a number of important LC terms [15,16]. As suggested in [17,18,19,20], the BP approximation to the ML problem can be improved by analyzing higher order terms of the LC (sometimes re-summing as in [20]) and using this information to modify the original GM. The hope is that when BP is run on the new GM it will be exact or, at least, improve upon its initial approximation.…”

The Use of Maximum Edge Weight Clique and Markov Random Field Problem Formulation to Discover Motifs

“…While there exists literature on the connection between the approaches used to solve the MEWCP and the MAP-MRF, the connection between the MEWCP and the MAP-MRF in particular seems not to have been explicitly stated. For instance, belief propagation has been used as a heuristic for certain classes of ILPs in the past (Gelfand et al, 2013), but often maximum edge weight cliques and Markov random fields are not mentioned as they are a special subcategory of a more general connection. We make the assertion here that the MEWCP and a MAP-MRF are equivalent, and in making this assertion, we are able to relate a collection of approaches commonly used for MAP-MRF to a collection of approaches used by the MEWCP.…”

“…which is equivalent to the formulation for a MEWCP for a complete multipartite graph (chapter 13 of Koller and Friedman ( 2009)) 1 , seen in Section 3.2.1. There seems to exist a further deeper connection between belief propagation and LP through the weighted matching graph problem (which tries to find a set of edges so that each vertex is matched with another vertex and the weights of the chosen edges are maximized) and all instances of being able to use belief propagation as a heuristic to LPs (Gelfand et al, 2013). However, in this case either the MAP goes unmentioned, or the belief propagation finds the marginal inference rather than the MAP of a system of random variables.…”

“…We could also try to use the well-known Markov chain Monte Carlo approach that is used to solve Markov random fields as a heuristic for an ILP formulation. Since there exists previous research exploring the connection between ILP and belief propagation through the weighted matching problem (Gelfand et al, 2013), we could further explore the connection between MEWCP and MAP-MRF to see if there exists an even broader graph problem connection that encompasses the Markov random field.…”