Bregman Forward-Backward Operator Splitting

Abstract

We propose an iterative method for finding a zero of the sum of two maximally monotone operators in reflexive Banach spaces. One of the operators is single-valued, and the method alternates an explicit step on this operator and an implicit step on the other one. Both steps involve the gradient of a convex function that is free to vary over the iterations. The convergence of the resulting forward-backward splitting method is analyzed using the theory of Legendre functions, under a novel assumption on the single-valued operator that captures various existing properties. When applied to minimization problems, rates are obtained for the objective values. The proposed framework unifies and extends several iterative methods which have thus far not been brought together, and it is also new in Euclidean spaces.

Optimal Complexity and Certification of Bregman First-Order Methods

“…On the way, we develop PEP techniques for Bregman settings, and extend the analysis of [34] for handling classes of differentiable and strictly convex functions. While we present the analysis on the basic NoLips algorithm for readability purposes, our results and methodology can be applied to various Bregman methods, such as inertial variants [1], or the Bregman proximal point scheme for convex minimization and monotone inclusions [18,10].…”

Forward-Backward Splitting for Optimal Transport Based Problems

“…We propose to solve problem (1) via a forward-backward splitting algorithm based on Bregman distances [24,25]. To this end, we remark that the Problem (1) can be generically formulated as…”

“…The above problem fits nicely into the forward-backward splitting framework of [24,25], which allows us to solve (2) through the following iterative algorithm 2…”

“…Require: Function J : R n×m → R with β-Lipschitz continuous gradient Require: Cost C ∈ R n×m , marginals µ (1) ∈ M n + , and (1) , µ (2) ) 4: return γ ∞ whose solution can be efficiently computed with the Sinkhorn algorithm [4]. According to the iterations in (2), replacing Σ with ∇H(γ k ) − α∇J(γ k ) leads to Algorithm 1, which is guaranteed to converge to a solution to Problem (2) by adequately setting the step-size α, as discussed in [24,25].…”

“…Indeed, they both consist of a sequential application of the Sinkhorn algorithm to an initial coupling, until it converges to a solution to the regularized problem. However, the CGS method performs a line search at each iteration to ensure the convergence, whereas Algorithm 1 simply works with a constant step-size, leading to an optimization method that is both more efficient and much easier to implement [24,25].…”