Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models

Birgin, Ernesto G.; Gardenghi, John Lenon Cardoso; Martínez, José Mário; Santos, Sandra A.; Toint, Philippe L.

doi:10.1007/s10107-016-1065-8

Cited by 132 publications

(268 citation statements)

References 16 publications

Supporting

Mentioning

253

Contrasting

Unclassified

Order By: Relevance

“…From the complexity point of view, it is known that the complexity of obtaining -approximate first-order criticality for unconstrained and convexly constrained problem can be reduced to O( −( p+1)/ p ) if one is ready to define the step by using a regularization model of order p ≥ 1. In the unconstrained case, this was shown for p = 2 in [16,47] and for general p ≥ 1 in [9], while the convexly constrained case was analysed (for p = 2) in [17]. The question of whether this methodology and the associated improvements in evaluation complexity bounds can be extended to order above one also remains open at this stage.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Second-Order Optimality and Beyond: Characterization and Evaluation Complexity in Convexly Constrained Nonlinear Optimization

2017

View full text Add to dashboard Cite

High-order optimality conditions for convexly constrained nonlinear optimization problems are analysed. A corresponding (expensive) measure of criticality for arbitrary order is proposed and extended to define high-order -approximate critical points. This new measure is then used within a conceptual trust-region algorithm to show that if derivatives of the objective function up to order q ≥ 1 can be evaluated and are Lipschitz continuous, then this algorithm applied to the convexly constrained problem needs at most O( −(q+1) ) evaluations of f and its derivatives to compute an -approximate qth-order critical point. This provides the first evaluation complexity result for critical points of arbitrary order in nonlinear optimization. An example is discussed, showing that the obtained evaluation complexity bounds are essentially sharp.

show abstract

Section: Discussionmentioning

confidence: 99%

“…The latter provide the first evaluation complexity bounds for general criticality order q. Note that, if q = 1, bounds of the type O( −( p+1)/ p ) exist if one is ready to minimize models of degree p > q (see [9]). Whether similar improvements can be obtained for q > 1 remains an open question at this stage.…”

Section: Lemma 43mentioning

confidence: 99%

Second-Order Optimality and Beyond: Characterization and Evaluation Complexity in Convexly Constrained Nonlinear Optimization

2017

View full text Add to dashboard Cite

show abstract

“…xt,p,Mt (y), with α given by (3.1). Similarly to [3], the trial point x + t must satisfies the following conditions:…”

Section: Notations and Generalitiesmentioning

confidence: 99%

Regularized Newton Methods for Minimizing Functions with Hölder Continuous Hessians

Grapiglia¹,

Nesterov

2017

SIAM J. Optim.

View full text Add to dashboard Cite

In this paper we study p-order methods for unconstrained minimization of convex functions that are p-times differentiable with ν-Hölder continuous pth derivatives. We propose tensor schemes with and without acceleration. For the schemes without acceleration, we establish iteration complexity bounds of O ǫ −1/(p+ν−1) for reducing the functional residual below a given ǫ ∈ (0, 1). Assuming that ν is know, we obtain an improved complexity bound of O ǫ −1/(p+ν) for the corresponding accelerated scheme. For the case in which ν is unknown, we present a universal accelerated tensor scheme with iteration complexity of O ǫ −p/[(p+1)(p+ν−1)] . A lower complexity bound of O ǫ −2/[3(p+ν)−2] is also obtained for this problem class.

show abstract

“…. Problems of the form (2.2) appear as auxiliary problems in p-order tensor methods for convex and nonconvex unconstrained optimization (see, e.g., [3,15,5,10,11]). In these methods, only approximate stationary points of Ω (ν)…”

Section: Notations and Generalitiesmentioning

confidence: 99%

“…Recently, two important works have pointed new ways towards practical tensor methods. In the context of nonconvex optimization, Birgin et al [3] presented a p-order tensor method that can findx with ∇f (x) * ≤ ǫ in at most O(ǫ − p+1 p ) iterations, generalizing the bound of O(ǫ − 3 2 ) proved in [17] for the CNM (case p = 2). The method is based on the same regularized models used in [1], but allows the trial points to be only approximate stationary points of the tensor models.…”

Section: Introductionmentioning

confidence: 99%

On inexact solution of auxiliary problems in tensor methods for convex optimization

Grapiglia

Nesterov

2020

Optimization Methods and Software

View full text Add to dashboard Cite

In this paper we study the auxiliary problems that appear in p-order tensor methods for unconstrained minimization of convex functions with ν-Hölder continuous pth derivatives. This type of auxiliary problems corresponds to the minimization of a (p+ν)-order regularization of the pth order Taylor approximation of the objective. For the case p = 3, we consider the use of a Gradient Methods with Bregman distance. When the regularization parameter is sufficiently large, we prove that the referred methods take at most O(log(ǫ −1 )) iterations to find either a suitable approximate stationary point of the tensor model or an ǫ-approximate stationary point of the original objective function.

show abstract

Worst-case evaluation complexity for unconstrained nonlinear optimization using high-order regularized models

Cited by 132 publications

References 16 publications

Second-Order Optimality and Beyond: Characterization and Evaluation Complexity in Convexly Constrained Nonlinear Optimization

Second-Order Optimality and Beyond: Characterization and Evaluation Complexity in Convexly Constrained Nonlinear Optimization

Regularized Newton Methods for Minimizing Functions with Hölder Continuous Hessians

On inexact solution of auxiliary problems in tensor methods for convex optimization

Contact Info

Product

Resources

About