On the Convergence of Grid-Based Methods for Unconstrained Optimization

Abstract

Abstract. The convergence of direct search methods for unconstrained minimization is examined in the case where the underlying method can be interpreted as a grid or pattern search over successively refined meshes. An important aspect of the main convergence result is that translation, rotation, scaling and shearing of the successive grids are allowed.Key words. Grid-based optimization, derivative free optimization, positive basis methods, convergence analysis, multidirectional search. 9]). Most of the current derivative-free algorithms for which convergence results have been established belong to one or more of three categories: line search methods, trust region methods or grid-based methods. In this paper, the convergence of derivative-free methods for unconstrained minimization is examined in the case where the underlying method can be interpreted as a grid or pattern search over successively refined meshes. Therefore, the methods discussed here are similar to those studied in [6], [8], [9], but permit greater freedom in the orientation and scaling of successive grids. Alternative approaches based on trust-regions or line searches can be found in [1], [7] and the references therein. AMSThe properties of grid-based methods are explored and it is shown that convergence can be achieved for a quite general class of algorithm. An important aspect of the main convergence result is that successive grids may be arbitrarily translated, rotated, and sheared relative to one another, and each grid axis may be re-scaled independently of the others. This flexibility allows second-order information to be incorporated into the shape of successive grids, for example by aligning grid axes along conjugate directions or the principal axes of an approximating quadratic. The hope is to construct non-derivative algorithms that possess useful properties of conjugate direction or quasi-Newton algorithms, thus exploiting curvature information without assuming the existence of second derivatives or the availability of first derivatives.We present two optimization frameworks for unconstrained optimization of continuously differentiable functions that are bounded below. For the first framework, in which finite searches are conducted along grid directions of descent, we establish convergence of a subsequence of iterates to a stationary point of the objective function. For the second framework, under the stronger assumption that the algorithm searches the grid direction of locally greatest descent at every iterate, we show that the entire sequence of iterates converges to a stationary point.The restrictions on the grids in our framework are much less severe than for the pattern search methods of [6,9] where a single set of grid axes is used, only rational scalings of grids are permitted, and arbitrary translations are not allowed.

New horizons in sphere-packing theory, part II: lattice-based derivative-free optimization via global surrogates

“…To accomplish this, again taking the q presepecified repulsive particles as fixed, we initialize n + m − q free repulsive particles randomly on n + m − q nearest-neighbor lattice points around the CMP and then, at each iteration, move the two or three 4 free particles that are furthest from equilibrium in the force-based model described above (that is, those free particles which have the highest force component projected onto the surface of the sphere) into new positions selected from the available locations in such a way as to minimize the maximum force (projected onto the sphere) over the entire set of (fixed and free) particles. Though each iteration of this algorithm involves an exhaustive search for placing the two or three free particles in question, it converges quickly when τ is O(100) or less.…”

AcMus: an open, integrated platform for room acoustics research

“…This approach has great relevance since the objective functions involved have too many local minima with values that are very far from the true optimal value. Two optimization algorithms have been implemented: the Density Clustering method [26] and a Grid-based local search [19]. Density Clustering is a stochastic optimization heuristic that has two phases: the global phase generates random feasible points using a uniform distribution on the feasible set, and evaluates the objective function on these points; the local phase applies a local optimization procedure on a subset of the feasible points, corresponding to those with better objective function values.…”

Extended Global Convergence Framework for Unconstrained Optimization

“…The proof of finiteness for step 2 can be found in [9] and follows the same argument as in [5] and [6].…”

“…According to [9] the function must be continuously differentiable and the sequence of iterates bounded.…”

“…The ideas from [4] and [5] seeded the definition and the convergence theory for the class of pattern search methods (PS) [6]. Two more general convergence theories for unconstrained optimization followed: global convergence framework for derivative free optimization methods (GCF) by Lucidi and Sciandrone [7], which emerged as an attempt to unify convergence theories for pattern search and line search (LS) [8], and grid-based search (GBS) [9] as an extension of PS.…”