Proximal ADMM for Nonconvex and Nonsmooth Optimization

Abstract

By enabling the nodes or agents to solve small-sized subproblems to achieve coordination, distributed algorithms are favored by many networked systems for efficient and scalable computation. While for convex problems, substantial distributed algorithms are available, the results for the more broad nonconvex counterparts are extremely lacking. This paper develops a distributed algorithm for a class of nonconvex and nonsmooth problems featured by i) a nonconvex objective formed by both separate and composite objective components regarding the decision components of interconnected agents, ii) local bounded convex constraints, and iii) coupled linear constraints. This problem is directly originated from smart buildings and is also broad in other domains. To provide a distributed algorithm with convergence guarantee, we revise the powerful tool of alternating direction method of multiplier (ADMM) and proposed a proximal ADMM. Specifically, noting that the main difficulty to establish the convergence for the nonconvex and nonsmooth optimization within the ADMM framework is to assume the boundness of dual updates, we propose to update the dual variables in a discounted manner. This leads to the establishment of a so-called sufficiently decreasing and lower bounded Lyapunov function, which is critical to establish the convergence. We prove that the method converges to some approximate stationary points. We besides showcase the efficacy and performance of the method by a numerical example and the concrete application to multi-zone heating, ventilation, and air-conditioning (HVAC) control in smart buildings.

A Survey of ADMM Variants for Distributed Optimization: Problems, Algorithms and Features

“…When problem (P2) is nonconvex (i.e., 𝑓 𝑖 are nonconvex), the other multi-block ADMM variants are not applicable in general. To address a nonconvex multiblock (P2), [110] proposed a proximal ADMM with global convergence guarantee. Proximal ADMM takes the iterative scheme Proximal ADMM: Primal update:…”

“…The major difference lies in the dual update where a discounted factor (1 − 𝜏) is imposed. As argued in [110], the discounted dual update scheme is critical to ensure the convergence of the method for solving nonconvex (P2). Specifically, it ensures the boundness of Lagrangian multipliers and makes it possible to identify a proper sufficiently decreasing and lower bounded Lyapunov function, which is a general key step to establish convergence of ADMM and its variants in nonconvex setting.…”

“…Specifically, it ensures the boundness of Lagrangian multipliers and makes it possible to identify a proper sufficiently decreasing and lower bounded Lyapunov function, which is a general key step to establish convergence of ADMM and its variants in nonconvex setting. The convergence of the method towards approximate stationary points was established for Lipschitz differentiable objective functions 𝑓 𝑖 [110].…”

“…Proximal ADMM is able to provide a global convergence guarantee but only ensures approximate stationary points. It is nontrivial to develop an ADMM variant for solving (P2) in nonconvex setting as discussed in [110]. This is attributed to the entirely nonsmooth structure of problem (P2) caused by the local bounded convex constraints imposed on each decision block.…”

“…Nonconvex. [110] has been the main solution [111,112]. Majorized ADMM is a conceptual algorithmic framework and its practical implementation requires to determine a proper majorized surrogate φ(x, y).…”