On the convergence rate improvement of a primal-dual splitting algorithm for solving monotone inclusion problems

Abstract

We present two modified versions of the primal-dual splitting algorithm relying on forward-backward splitting proposed in [21] for solving monotone inclusion problems. Under strong monotonicity assumptions for some of the operators involved we obtain for the sequences of iterates that approach the solution orders of convergence of O( 1 n ) and O(ω n ), for ω ∈ (0, 1), respectively. The investigated primal-dual algorithms are fully decomposable, in the sense that the operators are processed individually at each iteration. We also discuss the modified algorithms in the context of convex optimization problems and present numerical experiments in image processing and support vector machines classification.

Recent Developments on Primal–Dual Splitting Methods with Applications to Convex Minimization

“…n / n 0 converges. In the light of (13) increasing and unbounded sequence, by applying the Stolz-Cesàro Theorem, it yields (see [15]) …”

“…For particular instances of this iterative scheme in the context of monotone inclusion problems we refer the reader to [13] and in the context of convex optimization problems to [18,23]. Further, we discuss two accelerated versions of it proposed in [15], for which an evaluation of the convergence behavior of the sequences of primal and dual iterates, respectively, is possible. In Sect.…”

Phase retrieval problems in x-ray physics

“…Before we come to the generalization of CP to Banach spaces we also like to refer to other generalizations of this method: In addition to the above-mentioned preconditioned versions, there exist extended variants for solving monotone inclusion problems ( [14,78]) and also to the case of nonlinear operators T ( [80]). Recently, Lorenz and Pock ( [55]) proposed a quite general forward-backward algorithm for monotone inclusion problems with CP as a special case.…”

“…Now, in particular if the corresponding Tikhonov-type functional which has to be minimized is nonsmooth the proposed generalization CP-BS is an attractive method for this purpose. We also refer to [14,55,62,78,80] for other generalizations and extensions of CP .…”

Employing different loss functions for the classification of images via supervised learning

“…They can be seen as a theoretically well-funded alternative to some early algorithms with a consistent heuristical component (see, for instance, [8]). In this context we mention the computational approaches relying on the alternating direction method of multipliers (ADMM) (see, for instance, [5,11]), on the use of convex smoothing techniques (see [18,31]) and on some primal-dual proximal splitting methods (see [3]), all of them relying on quite intricate implementations. The approach introduced in this paper relies on the idea that for the most popular loss functions used in supervised classification one only needs to solve the dual conjugate problem, assuming the minimization of a convex differentiable function over the nonnegative orthant, and which can be done by standard optimization routines.…”