A Smooth Double Proximal Primal-Dual Algorithm for a Class of Distributed Nonsmooth Optimization Problems

Abstract

This technical note studies a class of distributed nonsmooth convex consensus optimization problem. The cost function is a summation of local cost functions which are convex but nonsmooth. Each of the local cost functions consists of a twice differentiable (smooth) convex function and two lower semi-continuous (nonsmooth) convex functions. We call this problem as single-smooth plus double-nonsmooth (SSDN) problem. Under mild conditions, we propose a distributed double proximal primal-dual optimization algorithm. The double proximal operator is designed to deal with the difficulty caused by the unproximable property of the summation of those two nonsmooth functions. Besides, it can also guarantee that the proposed algorithm is locally Lipschitz continuous. An auxiliary variable in the double proximal operator is introduced to estimate the subgradient of the second nonsmooth function. Theoretically, we conduct the convergence analysis by employing Lyapunov stability theory. It shows that the proposed algorithm can make the states achieve consensus at the optimal point. In the end, nontrivial simulations are presented and the results demonstrate the effectiveness of the proposed algorithm.

Solving Nonsmooth Resource Allocation Problems with Feasibility Constraints through Novel Distributed Algorithms

“…For the nonsmooth resource allocation problem (4), consider the case where the communication topology is a strongly connected and weight-balanced digraphs. If Assumptions 1 and 2 hold, with the initial condition satisfying N i=1 w i (0) = 0 n , then (y * , x * , s * , w * ) is an equilibrium point of (10), if and only if y * is an optimal solution of (4).…”

Distributed proximal gradient algorithm for non-smooth non-convex optimization over time-varying networks

“…Let g k = ∇g(x k ) + e k+1 . It follows from (8) and the definition of inexact proximal operator (3) that…”

“…One fundamental model for distributed non-smooth non-convex optimization, arising from optimization problems such as Lasso [1] , SVM [2] , and optimizing neural networks [3] , is that each local objective function of a node is the summation of a (non-convex) differentiable function and a non-smooth convex function (l 1 norm or indicator function). Although the research on distributed optimization has made significant progress on non-smooth convex problems [4]- [8] , distributed non-smooth non-convex optimization is still challenging.…”

Distributed proximal‐gradient algorithms for nonsmooth convex optimization of second‐order multiagent systems

“…Definition 2 ( -Lipschitz 36,37 ). A function f (⋅) ∶ S → R n is Lipschitz with constant > 0, or simply -Lipschitz over the set S if…”

“…f i (x i ) and g i (x i ) represent the quadratic objective and the l 1 penalty with the anchor 0 for each agent i, respectively. 36 Obviously, g i (x) (i = 1, 2, … , 6) is nondifferentiable. f i (x) (i = 1, 2, … , 6) is m i -strongly convex and its gradient satisfies i -Lipschitz condition.…”