A Family of CCCP Algorithms Which Minimize the TRW Free Energy

Abstract

We propose a family of convergent double-loop algorithms which minimize the TRW free energy. These algorithms are based on the concave convex procedure (CCCP) so we call them TRW-CCCP. Our formulation includes many free parameters which specify an infinite number of decompositions of the TRW free energy into convex and concave parts. TRW-CCCP is guaranteed to converge to the global minima for any settings of these free parameters, including adaptive settings if they satisfy conditions defined in this paper. We show that the values of these free parameters control the speed of convergence of the the inner and outer loops in TRW-CCCP. We performed experiments on a two-dimensional Ising model observing that TRW-CCCP converges to the global minimum of the TRW free energy and that the convergence rate depends on the parameter settings. We compare with the original message passing algorithm (TRW-BP) by varying the difficulty of the problem (by adjusting the energy function) and the number of iterations in the inner loop of TRW-CCCP. We show that on difficult problems TRW-CCCP converges faster than TRW-BP (in terms of total number of iterations) if few inner loop iterations are used.