Derivation and Analysis of the Primal-Dual Method of Multipliers Based on Monotone Operator Theory

Abstract

In this paper we present a novel derivation for an existing node-based algorithm for distributed optimisation termed the primal-dual method of multipliers (PDMM). In contrast to its initial derivation, in this work monotone operator theory is used to connect PDMM with other first-order methods such as Douglas-Rachford splitting and the alternating direction method of multipliers thus providing insight to the operation of the scheme. In particular, we show how PDMM combines a lifted dual form in conjunction with Peaceman-Rachford splitting to remove the need for collaboration between nodes per iteration. We demonstrate sufficient conditions for strong primal convergence for a general class of functions while under the assumption of strong convexity and functional smoothness, we also introduce a primal geometric convergence bound. Finally we introduce a distributed method of parameter selection in the geometric convergent case, requiring only finite transmissions to implement regardless of network topology.Index Terms-Primal-Dual method of multipliers (PDMM), alternating direction method of multipliers (ADMM), distributed optimisation, monotone operator, optimisation over networks.

Convex Optimisation-Based Privacy-Preserving Distributed Average Consensus in Wireless Sensor Networks

“…As shown in [34], the primal variable x (k) will converge geometrically to x * for arbitrary initialisation x (0) and λ (0) , thereby proving the correctness of the algorithm.…”

“…The proposed approach is based on the primal-dual method of multipliers (PDMM), an instance of Peaceman-Rachford splitting of the extended dual problem (see [34] for details). PDMM can, like ADMM, be used for iteratively solving constrained convex optimisation problems.…”

“…To address these limitations, we propose a convex optimisation-based method. To explain the basic concept, we show how it can be applied in the primal-dual method of multipliers (PDMM) [10,34] which is an iterative algorithm for solving constrained convex optimisation problems. The concept can, however, also be applied to other convex optimisation methods, for example ADMM-based algorithms.…”

Edge Consensus Computing for Heterogeneous Data Sets

“…Several algorithms address the edge consensus computing problem for the case that the cost function is convex, including the distributed alternating direction method of multipliers (ADMM) [3,4,5,6,7] and PDMM [8,9,10]. In distributed ADMM, the primal variables, which are explicit in the cost function, are exchanged among the nodes.…”

“…[11,12]), which is used to control differences among nodes with respect to the variables. However, the convergence rate is often relatively slow because it is based on Douglas-Rachford splitting [13,14] as remarked in [10]. In contrast, PDMM facilitates fast convergence because the constraints on the dual variables are represented by convex form and it is based on Peaceman-Rachford splitting [15,14,16] as remarked in [10].…”