Convergence Rate Analysis of a Sequential Convex Programming Method with Line Search for a Class of Constrained Difference-of-Convex Optimization Problems

Abstract

In this paper, we study the sequential convex programming method with monotone line search (SCP ls ) in [46] for a class of difference-of-convex (DC) optimization problems with multiple smooth inequality constraints. The SCP ls is a representative variant of moving-ball-approximationtype algorithms [6,10,13,54] for constrained optimization problems. We analyze the convergence rate of the sequence generated by SCP ls in both nonconvex and convex settings by imposing suitable Kurdyka-Lojasiewicz (KL) assumptions. Specifically, in the nonconvex settings, we assume that a special potential function related to the objective and the constraints is a KL function, while in the convex settings we impose KL assumptions directly on the extended objective function (i.e., sum of the objective and the indicator function of the constraint set). A relationship between these two different KL assumptions is established in the convex settings under additional differentiability assumptions. We also discuss how to deduce the KL exponent of the extended objective function from its Lagrangian in the convex settings, under additional assumptions on the constraint functions. Thanks to this result, the extended objectives of some constrained optimization models such as minimizing 1 subject to logistic/Poisson loss are found to be KL functions with exponent 1 2 under mild assumptions. To illustrate how our results can be applied, we consider SCP ls for minimizing 1−2 [60] subject to residual error measured by 2 norm/Lorentzian norm [21]. We first discuss how the various conditions required in our analysis can be verified, and then perform numerical experiments to illustrate the convergence behaviors of SCP ls .

Retraction-based first-order feasible methods for difference-of-convex programs with smooth inequality and simple geometric constraints

“…Specifically, we show that the extended objective 1 of a large class of Euclidean norm (and more generally, group LASSO penalty) regularized convex optimization problems is a KL function with exponent 1 2 . This complements the recent study in this direction [29,39,41,43], and, in particular, gives rise to linear convergence of our method in solving these structured optimization models.…”

“…The next corollary deals with (5.1) with m = 1. Its proof follows a similar line of arguments as the proof of [41,Corollary 4.3]. We omit the proof for brevity.…”

“…The desired results now follow from the same arguments as in the proof of [41, Theorem 3.12], starting from [41,Eq. (3.16)].…”

“…The proof of the following theorem is similar to that of [41,Theorem 4.2]. We omit its proof for brevity.…”

Analysis and algorithms for some compressed sensing models based on L1/L2 minimization

“…So this problem corresponds to (1.2) with P 1 and P 2 as in (1.6) and q = P 1 − P 2 . In the literature, algorithms for solving (1.3) with 1 norm or p quasi-norm in place of the quotient of the 1 and 2 norms have been discussed in [5,18,33], and [41] discussed an algorithm for solving (1.4) with 1 norm in place of the quotient of the 1 and 2 norms. These existing algorithms, however, are not directly applicable for solving (1.2) due to the fractional objective and the possibly nonsmooth continuous function q in the constraint.…”