Convergence rate analysis of a sequential convex programming method with line search for a class of constrained difference-of-convex optimization problems

Abstract

In this paper, we study the sequential convex programming method with monotone line search (SCP ls ) in [34] for a class of difference-of-convex (DC) optimization problems with multiple smooth inequality constraints. The SCP ls is a representative variant of moving-ball-approximationtype algorithms [4,8,11,39] for constrained optimization problems. We analyze the convergence rate of the sequence generated by SCP ls in both nonconvex and convex settings by imposing suitable Kurdyka-Lojasiewicz (KL) assumptions. Specifically, in the nonconvex settings, we assume that a special potential function related to the objective and the constraints is a KL function, while in the convex settings we impose KL assumptions directly on the extended objective function (i.e., sum of the objective and the indicator function of the constraint set). A relationship between these two different KL assumptions is established in the convex settings under additional differentiability assumptions. We also discuss how to deduce the KL exponent of the extended objective function from its Lagrangian in the convex settings, under additional assumptions on the constraint functions. Thanks to this result, the extended objectives of some constrained optimization models such as minimizing 1 subject to logistic/Poisson loss are found to be KL functions with exponent 1 2 under mild assumptions. To illustrate how our results can be applied, we consider SCP ls for minimizing 1−2 [43] subject to residual error measured by 2 norm/Lorentzian norm [19]. We first discuss how the various conditions required in our analysis can be verified, and then perform numerical experiments to illustrate the convergence behaviors of SCP ls .

The exact modulus of the generalized concave Kurdyka-Łojasiewicz property

“…Hence one has for q ∈ N p+q k=pz k+1 − z k ≤ C • φ Ψ(z p ) − Ψ(z * ) + z p − z p−1 ,which proves(16). By taking q → ∞, one has∞ k=1 z k+1 − z k ≤ p−k+1 − z k + C • φ Ψ(z p ) − Ψ(z * ) + z p − z p−1 ,from which(17) readily follows by setting p = l + 1. Now let p ≥ l + 1, where l is the index given in assertion (i), and let q ∈ N. Thenz p+q − z p ≤ p+q−1 k=p z k+1 − z k ≤ p+q k=p z k+1 − z k .…”

“…We will show that (z k ) k∈N converges to z * , and along the way we also establish ( 16) and (17). We proceed by considering two cases.…”

“…Furthermore, the trajectory of iterates has finite length property, i.e., ∞ k=1 z k+1 − z k < ∞, where (z k ) k∈N is a sequence generated by PALM. This pleasant convergence mechanism owing to the KL property has gained increasing attention, see, e.g., [1,15,17,9,14,7] and the references therein. These results, despite devoting to different algorithms, share a common theme: Employing the KL property to ensure that the sequence generated by the respective algorithm has finite length property, see, e.g., [9,Lemma 3.5], [15,Theorem 3.1] and [1,Theorem 1].…”

Calculus rules of the generalized concave Kurdyka-Łojasiewicz property

“…Notice that A is surjective. Hence [15,Exercise 1.53] implies that A * y ≥ r y , which means that F (x) = Ax − b satisfies (23) for every x ∈ R n with α = 1/r and ε 1 = ∞. Then applying a similar argument in Theorem 3.11 completes the proof.…”

“…The concave KL property is instrumental in the convergence analysis of many proximaltype algorithms, see, e.g., [1,2,3,4,7,8,16,20,22,23] and the references therein; see also [6,10,14] for seminal theoretical work on this area. Convergence rates of such algorithms are usually determined by the KL exponent θ ∈ [0, 1) when desingularizing functions have the Lojasiewicz form ϕ(t) = c • t 1−θ for some c > 0.…”