Perturbed proximal primal–dual algorithm for nonconvex nonsmooth optimization

A Proximal Zeroth-Order Algorithm for Nonconvex Nonsmooth Problems

“…Recently, a proximal algorithm (PG-EXTRA) is proposed in [3], which uses constant stepsize and achieves o(1/R) rate for nondifferentiable but convex optimization. A recent research attention for solving problem (1) in [4] has been devoted to the so-called perturbed proximal primal-dual algorithm (PProx-PDA) in which a primal gradient descent step is performed followed by an approximate dual gradient ascent step. Our algorithm is closely related to PProx-PDA, which achieves a sublinear convergence to only an εstationary point.…”

“…In fact, matrix B is often used to eliminate the nonconvexity in the augmented Lagrangian, in order for the obtained subproblem to be strongly convex, or even to provide a closed-form solution through choosing matrix B with A T A + B T B I N . Although parameters ρ, γ and β are fixed for all r, it could be shown that adapting the parameters can accelerate the convergence of the algorithm [4]. Finally, we note that step 2 in Algorithm 1 is decomposable over the variables, therefore they are wellsituated to be implemented in a distributed manner.…”

A Feedback-Based Regularized Primal-Dual Gradient Method for Time-Varying Nonconvex Optimization

“…(i) The paper utilizes a regularized primal-dual method, where the regularization comes in the form of a strongly concave term in the dual vector variable that is added to the Lagrangian function [26]; see also [35] and [24,21]. The strongly concave regularization term plays a critical role in establishing Q-linear convergence of the proposed algorithm.…”

“…The strongly concave regularization term plays a critical role in establishing Q-linear convergence of the proposed algorithm. However, as an artifact of this regularization, existing works for time-invariant convex programs [26], time-varying convex programs [6], and for static nonconvex problems [21] could prove that gradient-based iterative methods approach an approximate KKT point [2]. On the other hand, the present paper provides analytical results in terms of tracking a KKT point (as opposed to an approximate KKT point) of the nonconvex problem (1.3) and provides bounds for the distance of the algorithmic iterates from a KKT trajectory.…”

“…Examples of works dealing with gradient-based or proximal-based methods for (static) non-convex problems include, for example, [10,21,23,32,41,51,52] (see also references therein); in particular, [21] considers a perturbed primal-dual method applied to a problem with nonconvex cost and (linear) consensus constraints. The present paper considers a more general case, investigates running algorithms for timevarying settings, and it analyzes both discrete-time and continuous-time algorithms.…”