We consider minimization of a smooth nonconvex objective function using an iterative algorithm based on Newton's method and the linear conjugate gradient algorithm, with explicit detection and use of negative curvature directions for the Hessian of the objective function. The algorithm tracks Newton-conjugate gradient procedures developed in the 1980s closely, but includes enhancements that allow worst-case complexity results to be proved for convergence to points that satisfy approximate first-order and second-order optimality conditions. The complexity results match the best known results in the literature for second-order methods.
We describe an algorithm based on a logarithmic barrier function, Newton's method, and linear conjugate gradients that seeks an approximate minimizer of a smooth function over the nonnegative orthant. We develop a bound on the complexity of the approach, stated in terms of the required accuracy and the cost of a single gradient evaluation of the objective function and/or a matrix-vector multiplication involving the Hessian of the objective. The approach can be implemented without explicit calculation or storage of the Hessian.
We examine the behavior of accelerated gradient methods in smooth nonconvex unconstrained optimization, focusing in particular on their behavior near strict saddle points. Accelerated methods are iterative methods that typically step along a direction that is a linear combination of the previous step and the gradient of the function evaluated at a point at or near the current iterate. (The previous step encodes gradient information from earlier stages in the iterative process.) We show by means of the stable manifold theorem that the heavy-ball method is unlikely to converge to strict saddle points, which are points at which the gradient of the objective is zero but the Hessian has at least one negative eigenvalue. We then examine the behavior of the heavy-ball method and other accelerated gradient methods in the vicinity of a strict saddle point of a nonconvex quadratic function, showing that both methods can diverge from this point more rapidly than the steepest-descent method.
Populations of Idotea pelagica and Idotea granulosa were studied monthly at Carnsore Point, County Wexford, from July 1978 to October 1979. The density of I. pelagica in mussel beds varied between 26 and 180/100 cm, reaching a peak in June. Numbers of I. granulosa in samples of Gigartina stellata fluctuated from month to month and averaged 200/100 g dry weight of algae. Peak densities were recorded in September of both years. Both species exhibited continuous growth, and breeding occurred throughout the year but with a main breeding period starting in winter. I. pelagica grew more rapidly and had an extensive breeding period lasting from December to August with two generations, some members of both generations producing second broods. For I. granulosa, the main breeding period was January to April, after which most adults died. Some of their offspring reached maturity in late summer giving rise to a lesser breeding peak in late summer and autumn but most did not mature until winter. Summer breeders of both species were smaller than those breeding in winter. The breeding cycles are compared with those of other populations in Britain and the Baltic and the possible reasons for the differences discussed.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.