Combining and scaling descent and negative curvature directions

Avelino, Catarina P.; Moguerza, Javier M.; Olivares, Alberto; Prieto, Francisco J.

doi:10.1007/s10107-009-0305-6

Cited by 6 publications

(8 citation statements)

References 18 publications

(24 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In order to illustrate the effect of our approach, we selected a benchmark of 61 smooth problems from the CUTEst collection [16] for which first and second-order derivatives were provided and negative curvature was detected, inspired by the benchmark of Avelino et al [2] 2 . Table 1 list the problems and their dimensions.…”

Section: Numerical Illustrationsmentioning

confidence: 99%

“…A partial explanation may be found in the theory established for second-order convergent (derivative-based) line-search methods [2,15,21,23]. In such studies, it has been identified that an algorithm exploiting both directions of descent and of negative curvature should not necessarily associate those with the same step length.…”

mentioning

confidence: 99%

“…More precisely, of the CUTEst problems tested in[2], we considered the 60 problems for which negative curvature was detected during a run of one of the second-order methods tested in[2], and for which dimension could be chosen to be less than 50. We additionally included problem SPARSINE, not tested in[2], but for which we observed negative curvature at the default initial point given in CUTEst.…”

mentioning

confidence: 99%

See 2 more Smart Citations

A decoupled first/second-order steps technique for nonconvex nonlinear unconstrained optimization with improved complexity bounds

2018

View full text Add to dashboard Cite

In order to be provably convergent towards a second-order stationary point, optimization methods applied to nonconvex problems must necessarily exploit both first and second-order information. However, as revealed by recent complexity analyses of some of these methods, the overall effort to reach second-order points is significantly larger when compared to the one of approaching first-order ones. On the other hand, there are other algorithmic schemes, initially designed with first-order convergence in mind, that do not appear to maintain the same first-order performance when modified to take second-order information into account.In this paper, we propose a technique that separately computes first and second-order steps, and that globally converges to second-order stationary points: it consists in better connecting the steps to be taken and the stationarity criteria, potentially guaranteeing larger steps and decreases in the objective. Our approach is shown to lead to an improvement of the corresponding complexity bound with respect to the first-order optimality tolerance, while having a positive impact on the practical behavior. Although the applicability of our ideas is wider, we focus the presentation on trust-region methods with and without derivatives, and motivate in both cases the interest of our strategy.

show abstract

Section: Numerical Illustrationsmentioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 1 more Smart Citation

A decoupled first/second-order steps technique for nonconvex nonlinear unconstrained optimization with improved complexity bounds

2018

View full text Add to dashboard Cite

show abstract

“…We compare implementations of Algorithms 2.1 and 3.1 on less pathological cases than the one mentioned above. The test functions are taken from the CUTEst package [24], and have been identified as presenting negative curvature at some points by Avelino et al [5,Table 6].…”

Section: Numerical Observationsmentioning

confidence: 99%

“…This represents a total of 60 problems out of the 119 problems tested in [5]. For all of those problems, we used the smallest dimension available in the SIF files, resulting in 36 problems with dimensions less than 10, 22 problems having dimensions between 10 and 15, 1 problem with dimension 28, and 1 problem with dimension 50.…”

Section: Numerical Observationsmentioning

confidence: 99%

A second-order globally convergent direct-search method and its worst-case complexity

2015

View full text Add to dashboard Cite

Direct-search algorithms form one of the main classes of algorithms for smooth unconstrained derivative-free optimization, due to their simplicity and their well-established convergence results. They proceed by iteratively looking for improvement along some vectors or directions. In the presence of smoothness, first-order global convergence comes from the ability of the vectors to approximate the steepest descent direction, which can be quantified by a first-order criticality (cosine) measure. The use of a set of vectors with a positive cosine measure together with the imposition of a sufficient decrease condition to accept new iterates leads to a convergence result as well as a worst-case complexity bound. In this paper, we present a second-order study of a general class of direct-search methods. We start by proving a weak second-order convergence result related to a criticality measure defined along the directions used throughout the iterations. Extensions of this result to obtain a true second-order optimality one are discussed, one possibility being a method using approximate Hessian eigenvectors as directions (which is proved to be truly secondorder globally convergent). Numerically guaranteeing such a convergence can be rather expensive to ensure, as it is indicated by the worst-case complexity analysis provided in this paper, but turns out to be appropriate for some pathological examples.

show abstract

On the Use of the SYMMBK Algorithm for Computing Negative Curvature Directions Within Newton–Krylov Methods

Fasano,

Piermarini,

Roma

2024

AIRO Springer Series

View full text Add to dashboard Cite

In this paper, we consider the issue of computing negative curvature directions, for nonconvex functions, within Newton-Krylov methods for large scale unconstrained optimization. This issue has been widely investigated in the literature, and different approaches have been proposed. We focus on the well known SYMMBK method proposed for solving large scale symmetric possibly indefinite linear systems [3,5,7,20], and show how to exploit it to yield an effective negative curvature direction. The distinguishing feature of our proposal is that the computation of such negative curvature direction is iteratively carried out, without storing no more than a couple of additional vectors. The results of a preliminary numerical experience are reported showing the reliability of the novel approach we propose.

show abstract

Combining and scaling descent and negative curvature directions

Cited by 6 publications

References 18 publications

A decoupled first/second-order steps technique for nonconvex nonlinear unconstrained optimization with improved complexity bounds

A decoupled first/second-order steps technique for nonconvex nonlinear unconstrained optimization with improved complexity bounds

A second-order globally convergent direct-search method and its worst-case complexity

On the Use of the SYMMBK Algorithm for Computing Negative Curvature Directions Within Newton–Krylov Methods

Contact Info

Product

Resources

About