Kurdyka-Łojasiewicz exponent via inf-projection

Abstract

Kurdyka-Lojasiewicz (KL) exponent plays an important role in estimating the convergence rate of many contemporary first-order methods. In particular, a KL exponent of 1 2 is related to local linear convergence. Nevertheless, KL exponent is in general extremely hard to estimate. In this paper, we show under mild assumptions that KL exponent is preserved via inf-projection. Inf-projection is a fundamental operation that is ubiquitous when reformulating optimization problems via the lift-and-project approach. By studying its operation on KL exponent, we show that the KL exponent is 1 2 for several important convex optimization models, including some semidefinite-programming-representable functions and functions that involve C 2 -cone reducible structures, under conditions such as strict complementarity. Our results are applicable to concrete optimization models such as group fused Lasso and overlapping group Lasso. In addition, for nonconvex models, we show that the KL exponent of many difference-of-convex functions can be derived from that of their natural majorant functions, and the KL exponent of the Bregman envelope of a function is the same as that of the function itself. Finally, we estimate the KL exponent of the sum of the least squares function and the indicator function of the set of matrices of rank at most k.

A Bregman forward-backward linesearch algorithm for nonconvex composite optimization: superlinear convergence to nonisolated local minima

“…Whenever the desingularizing function can be taken of the form ψ(s) = ̺s ϑ with ̺ > 0 an ϑ ∈ (0, 1), it is usually referred to as Łojasiewicz function (with exponent 1 − ϑ). It has been shown in [71,Thm. 5.2] that whenever the kernel function h is twice continuously differentiable and (locally) strongly convex, the function ϕ admits a Łojasiewicz desingularizing function with exponent ϑ ≥ 1 /2 iff so does the Bregman envelope ϕ h , in which case the exponent is preserved.…”

“…1) Bregman forward-backward envelope: a new key tool. We introduce an envelope function for forward-backward splitting using Bregman distance, the Bregman forward-backward envelope (BFBE), which is a generalization of its Euclidean counterpart introduced in [52] and later further analyzed in [42,57,64,66,71]. A local equivalence of the BFBE and its Euclidean version allows to provide first-and second-order differential properties of the BFBE based on the known Euclidean properties of prox-regularity and epi-differentiability (Theorems 4.11 and 4.13).…”

The exact modulus of the generalized concave Kurdyka-Łojasiewicz property

“…• One way to estimate the exact modulus is applying calculus rules of the generalized KL property. Li and Pong [6] and Yu et al [16] developed several calculus rules of the KL property, in the case where desingularizing functions take the specific form ϕ(t) = c • t 1−θ , where c > 0 and θ ∈ [0, 1). However, the exact modulus has various forms depending on the given function, which requires us to obtain general calculus rules without assuming that desingularizing functions have the specific form.…”

Calculus rules of the generalized concave Kurdyka-Łojasiewicz property

“…The concave KL property is instrumental in the convergence analysis of many proximaltype algorithms, see, e.g., [1,2,3,4,7,8,16,20,22,23] and the references therein; see also [6,10,14] for seminal theoretical work on this area. Convergence rates of such algorithms are usually determined by the KL exponent θ ∈ [0, 1) when desingularizing functions have the Lojasiewicz form ϕ(t) = c • t 1−θ for some c > 0.…”

“…Li and Pong [12] recently developed several important calculus rules for the KL exponent, which facilitates estimating the KL exponent for some structured optimization problems. Under suitable assumptions, Wu, Pan and Bi [21] studied the KL exponent for problems involving sums of the zero norm and nonconvex functions, see also [13,22] for other pleasing progress on this line of research.…”