Abstract. In this work we propose a new splitting technique, namely Asymmetric Forward-Backward-Adjoint splitting, for solving monotone inclusions involving three terms, a maximally monotone, a cocoercive and a bounded linear operator. Classical operator splitting methods, like Douglas-Rachford and Forward-Backward splitting are special cases of our new algorithm. Asymmetric Forward-Backward-Adjoint splitting unifies, extends and sheds light on the connections between many seemingly unrelated primal-dual algorithms for solving structured convex optimization problems proposed in recent years. More importantly, it greatly extends the scope and applicability of splitting techniques to a wider variety of problems. One important special case leads to a Douglas-Rachford type scheme that includes a third cocoercive operator.
This paper proposes TriPD, a new primal-dual algorithm for minimizing the sum of a Lipschitz-differentiable convex function and two possibly nonsmooth convex functions, one of which is composed with a linear mapping. We devise a randomized block-coordinate version of the algorithm which converges under the same stepsize conditions as the full algorithm. It is shown that both the original as well as the block-coordinate scheme feature linear convergence rate when the functions involved are either piecewise linear-quadratic, or when they satisfy a certain quadratic growth condition (which is weaker than strong convexity). Moreover, we apply the developed algorithms to the problem of multi-agent optimization on a graph, thus obtaining novel synchronous and asynchronous distributed methods. The proposed algorithms are fully distributed in the sense that the updates and the stepsizes of each agent only depend on local information. In fact, no prior global coordination is required. Finally, we showcase an application of our algorithm in distributed formation control.
This paper analyzes block-coordinate proximal gradient methods for minimizing the sum of a separable smooth function and a (nonseparable) nonsmooth function, both of which are allowed to be nonconvex. The main tool in our analysis is the forwardbackward envelope (FBE), which serves as a particularly suitable continuous and realvalued Lyapunov function. Global and linear convergence results are established when the cost function satisfies the Kurdyka-Łojasiewicz property without imposing convexity requirements on the smooth function. Two prominent special cases of the investigated setting are regularized finite sum minimization and the sharing problem; in particular, an immediate byproduct of our analysis leads to novel convergence results and rates for the popular Finito/MISO algorithm in the nonsmooth and nonconvex setting with very general sampling strategies.
Abstract-We consider a network of agents, each with its own private cost consisting of the sum of two possibly nonsmooth convex functions, one of which is composed with a linear operator. At every iteration each agent performs local calculations and can only communicate with its neighbors. The goal is to minimize the aggregate of the private cost functions and reach a consensus over a graph. We propose a primal-dual algorithm based on Asymmetric Forward-Backward-Adjoint (AFBA), a new operator splitting technique introduced recently by two of the authors. Our algorithm includes the method of Chambolle and Pock as a special case and has linear convergence rate when the cost functions are piecewise linear-quadratic. We show that our distributed algorithm is easy to implement without the need to perform matrix inversions or inner loops. We demonstrate through computational experiments how selecting the parameter of our algorithm can lead to larger step sizes and yield better performance.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.