Recursive Trust-Region Methods for Multiscale Nonlinear Optimization

Abstract

Abstract. A class of trust-region methods is presented for solving unconstrained nonlinear and possibly nonconvex discretized optimization problems, like those arising in systems governed by partial differential equations. The algorithms in this class make use of the discretization level as a mean of speeding up the computation of the step. This use is recursive, leading to true multilevel/multiscale optimization methods reminiscent of multigrid methods in linear algebra and the solution of partial-differential equations. A simple algorithm of the class is then described and its numerical performance is shown to be numerically promising. This observation then motivates a proof of global convergence to first-order stationary points on the fine grid that is valid for all algorithms in the class.Keywords: nonlinear optimization, multiscale problems, simplified models, recursive algorithms, convergence theory.AMS subject classifications. 90C30, 65K05, 90C26, 90C061. Introduction. Large-scale finite-dimensional optimization problems often arise from the discretization of infinite-dimensional problems, a primary example being optimal-control problems defined in terms of either ordinary or partial differential equations. While the direct solution of such problems for a discretization level is often possible using existing packages for large-scale numerical optimization, this technique typically does make very little use of the fact that the underlying infinite-dimensional problem may be described at several discretization levels; the approach thus rapidly becomes cumbersome. Motivated by this observation, we explore here a class of algorithms which makes explicit use of this fact in the hope of improving efficiency and, possibly, enhancing reliability.Using the different levels of discretization for an infinite-dimensional problem is not a new idea. A simple first approach is to use coarser grids in order to compute approximate solutions which can then be used as starting points for the optimization problem on a finer grid (see [5,6,7,22], for instance). However, potentially more efficient techniques are inspired from the multigrid paradigm in the solution of partial differential equations and associated systems of linear algebraic equations (see, for example, [10,11,12,23,40,42], for descriptions and references). The work presented here was in particular motivated by the paper by Gelman and Mandel [16], the "generalized truncated Newton algorithm" presented in Fisher [15], a talk by Moré [27] and the contributions by Nash and co-authors [25,26,29]. These latter three papers present the description of MG/OPT, a linesearch-based recursive algorithm, an outline of its convergence properties and impressive numerical results. The generalized truncated Newton algorithm and MG/OPT are very similar and, like many linesearch methods, naturally suited to convex problems, although their generalization to the nonconvex case is possible. Further motivation is also provided by the computational success of the low/high-fidelity mo...

Worst case complexity of direct search

“…Similar bounds have been proved by Gratton, Toint, and co-authors [8,9] for trust-region methods and by Cartis, Gould, and Toint [3] for adaptive cubic overestimation methods, when these algorithms are based on a Cauchy decrease condition. Cartis, Gould, and Toint [4] have shown that such a bound is tight for the steepest descent method by presenting an example where the number of iterations is arbitrarily close to it (these authors have also shown that Newton's method can also take a number of iterations arbitrarily close to O( −2 )).…”

Worst case complexity of direct search under convexity

“…Such a bound has been proved sharp or tight by Cartis, Gould, and Toint [1]. There has been quite an amount of research on WCC bounds for several other classes of algorithms in the non-convex case (see, e.g., [7,9,14]). …”

Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization

“…The author expects that many of them can be adapted to the more general context discussed here. In particular, one may think of extensions in the spirit of the retrospective trust-region algorithm by Bastin, Malmedy, Mouffe, Toint and Tomanos (2009), or to multilevel frameworks presented by Gratton, Sartenaer and Toint (2008). Which of these will be most useful is a topic for further research.…”