Exploiting negative curvature directions in linesearch methods for unconstrained optimization

Abstract

In this paper we propose efficient new linesearch algorithms for solving large scale unconstrained optimization problems which exploit any local nonconvexity of the objective function. Current algorithms in this class typically compute a pair of search directions at every iteration: a Newton-type direction, which ensures both global and fast asymptotic convergence, and a negative curvature direction, which enables the iterates to escape from the region of local non-convexity. A new point is generated by performing a search along a line or a curve obtained by combining these two directions. However, in almost all if these algorithms, the relative scaling of the directions is not taken into account.We propose a new algorithm which accounts for the relative scaling of the two directions. To do this, only the most promising of the two directions is selected at any given iteration, and a linesearch is performed along the chosen direction. The appropriate direction is selected by estimating the rate of decrease of the quadratic model of the objective function in both candidate directions. We prove global convergence to second-order critical points for the new algorithm, and report some preliminary numerical results.

A curvilinear method based on minimal-memory BFGS updates

“…Moreover, in [14] an Armijo-type rule was used, to ensure convergence to a second-order critical point, where the Hessian matrix is positive semi-definite. In [9] the alternative use of a negative curvature direction and a Newton-type direction was proposed, within an appropriate linesearch procedure. In the work of Olivares et al [17] the authors established criteria such that at each iteration, either a linesearch procedure using one direction (a Newton-type or a negative curvature direction) or a curvilinear search combining both directions, is performed.…”

“…However, the storage of a matrix is required and only a few Lanczos vectors are stored. Other works also use directions of negative curvature produced by preconditioned conjugate gradients and Lanczos methods [9,12]. In these methods it is necessary to repeat the recurrence in order to regenerate the Lanczos vectors whenever they are needed.…”

Nonconvex optimization using negative curvature within a modified linesearch

“…This approach is similar to the one discussed in Gould et al [14], with the main difference that, while they only consider two alternatives, we consider the three following possibilities:…”

“…Gould et al [14] study both directions in the descent pair ðs k ; d k Þ, and select the one that provides the largest descent ratio, measured against the quadratic model. Thus, the Newton direction s k is chosen whenever the condition…”

“…Depending on the results from the application of this criterion, the algorithm will apply either a curvilinear search similar to that proposed by Moré and Sorensen [21], combining both search directions, or a standard linesearch similar to that proposed by Gould et al [14], using just one of the two directions.…”