Convergence of the Iterates of Descent Methods for Analytic Cost Functions

Absil, Pierre-Antoine; Mahony, Robert; Andrews, Ben

doi:10.1137/040605266

Cited by 256 publications

(419 citation statements)

References 26 publications

Supporting

Mentioning

401

Contrasting

Unclassified

Order By: Relevance

“…This condition has been considered in [1] for nonconvex functions defined in finitedimensional spaces. It is easily seen that (53) …”

Section: Applying Theorem 20 [(I)⇔(vi)] We Obtainmentioning

confidence: 99%

“…The introduction of this class, as well as the employed terminology (desingularization) are justified by (1). Indeed, composing a function f , which typically has 0 as a critical value, with an appropriate element ϕ of K(0,r) consists in relabeling its level sets in order to "straighten up all derivatives" of f in [0 < f <r].…”

Section: Notation (Multivalued Mappings)mentioning

confidence: 99%

“…Condition (53) is fulfilled by several explicit gradient-like methods, including trust region methods, line-search gradient methods and some Riemannian variants; see [1] for examples and references.…”

Section: Applying Theorem 20 [(I)⇔(vi)] We Obtainmentioning

confidence: 99%

“…This was one of the motivations for the nonsmooth K L-inequalities developed in [8,9]. Due to its considerable impact on several fields of applied mathematics: minimization and algorithms [1,5,8,39], asymptotic theory of differential inclusions [48], neural networks [28], complexity theory [47] (see [47,Definition 3], where functions satisfying a K L-type inequality are called gradient dominated functions), and partial differential equations [51,35,30,31], we hereby tackle the problem of characterizing such inequalities in a nonsmooth infinite-dimensional setting and provide further clarifications for several application aspects. Our framework is rather broad (infinite dimensions, nonsmooth functions); nevertheless, to the best of our knowledge, most of the present results are also new in a smooth finite-dimensional framework: readers who feel unfamiliar with notions of nonsmooth and variational analysis may, as a first stage, consider that all functions involved are differentiable, replace subdifferentials by usual derivatives and subgradient systems by smooth ones.…”

Section: Introductionmentioning

confidence: 99%

“…In the framework of a C 1 function f defined on a real Hilbert space [H, ·, · ], and assuming for simplicity thatf = 0 is a critical value, this generalized inequality (that we hereby call the Kurdyka-Lojasiewicz inequality, or in short, the K L-inequality) states that (1) ||∇(ϕ • f )(x)|| ≥ 1, for some continuous function ϕ : [0, r) → R, C 1 on (0, r) with ϕ > 0 and all x in [0 < f < r] := {y ∈ H : 0 < f(y) < r}. The class of such functions ϕ will be further denoted by K(0,r); see (8).…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Characterizations of Łojasiewicz inequalities: Subgradient flows, talweg, convexity

Bolte

Daniilidis

Ley

et al. 2009

Trans. Amer. Math. Soc.

268

307

View full text Add to dashboard Cite

Abstract. The classical Lojasiewicz inequality and its extensions for partial differential equation problems (Simon) and to o-minimal structures (Kurdyka) have a considerable impact on the analysis of gradient-like methods and related problems: minimization methods, complexity theory, asymptotic analysis of dissipative partial differential equations, and tame geometry. This paper provides alternative characterizations of this type of inequality for nonsmooth lower semicontinuous functions defined on a metric or a real Hilbert space. In the framework of metric spaces, we show that a generalized form of the Lojasiewicz inequality (hereby called the Kurdyka-Lojasiewicz inequality) is related to metric regularity and to the Lipschitz continuity of the sublevel mapping, yielding applications to discrete methods (strong convergence of the proximal algorithm). In a Hilbert setting we further establish that asymptotic properties of the semiflow generated by −∂f are strongly linked to this inequality. This is done by introducing the notion of a piecewise subgradient curve: such curves have uniformly bounded lengths if and only if the KurdykaLojasiewicz inequality is satisfied. Further characterizations in terms of talweg lines -a concept linked to the location of the less steepest points at the level sets of f -and integrability conditions are given. In the convex case these results are significantly reinforced, allowing us in particular to establish a kind of asymptotic equivalence for discrete gradient methods and continuous gradient curves. On the other hand, a counterexample of a convex C 2 function in R 2 is constructed to illustrate the fact that, contrary to our intuition, and unless a specific growth condition is satisfied, convex functions may fail to fulfill the Kurdyka-Lojasiewicz inequality.

show abstract

“…This condition has been considered in [1] for nonconvex functions defined in finitedimensional spaces. It is easily seen that (53) …”

Section: Applying Theorem 20 [(I)⇔(vi)] We Obtainmentioning

confidence: 99%

Section: Notation (Multivalued Mappings)mentioning

confidence: 99%

Section: Applying Theorem 20 [(I)⇔(vi)] We Obtainmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Characterizations of Łojasiewicz inequalities: Subgradient flows, talweg, convexity

Bolte

Daniilidis

Ley

et al. 2009

Trans. Amer. Math. Soc.

268

307

View full text Add to dashboard Cite

show abstract

Łojasiewicz inequality and exponential convergence of the full‐range model of CNNs

Marco

Forti

Grazzini

et al. 2012

Circuit Theory & Apps

View full text Add to dashboard Cite

SUMMARYThis paper considers the full-range (FR) model of Cellular Neural Networks (CNNs) in the case where the signal range is delimited by an ideal hard-limiter nonlinearity with two vertical segments in the i −v characteristic. A ⁄ Lojasiewicz inequality around any equilibrium point, for a FRCNN with a symmetric interconnection matrix, is proved. It is also shown that the ⁄ Lojasiewicz exponent is equal to 1 2 . The main consequence is that any forward solution of a symmetric FRCNN has finite length and is exponentially convergent toward an equilibrium point, even in degenerate situations where the FRCNN possesses nonisolated equilibrium points. The obtained results are shown to improve the previous results in literature on convergence or almost convergence of symmetric FRCNNs.

show abstract

Pointwise convergence of gradient‐like systems

Lageman

2007

Mathematische Nachrichten

View full text Add to dashboard Cite

Key words Gradient-like systems, ω-limit set, convergence, o-minimal structures, analytic-geometric categories MSC (2000) 37C10, 37B25, 32P20, 34A34S. Łojasiewicz has shown that the ω-limit sets of the trajectories of analytic gradient systems consist of at most one point. We extend this result to the larger class of gradient-like vector fields satisfying an angle condition. In particular, this includes gradient systems, defined by arbitrary C 1 functions from an analytic-geometric category. Corresponding pointwise convergence results are shown for discrete gradient-like algorithms on a Riemannian manifold. This generalizes recent results by Absil, Mahony, and Andrews to the Riemannian geometry setting.

show abstract

Convergence of the Iterates of Descent Methods for Analytic Cost Functions

Cited by 256 publications

References 26 publications

Characterizations of Łojasiewicz inequalities: Subgradient flows, talweg, convexity

Characterizations of Łojasiewicz inequalities: Subgradient flows, talweg, convexity

Łojasiewicz inequality and exponential convergence of the full‐range model of CNNs

Pointwise convergence of gradient‐like systems

Contact Info

Product

Resources

About