Grid Restrained Nelder-Mead Algorithm

Abstract

Probably the most popular algorithm for unconstrained minimization for problems of moderate dimension is the Nelder-Mead algorithm published in 1965. Despite its age only limited convergence results exist. Several counterexamples can be found in the literature for which the algorithm performs badly or even fails. A convergent variant derived from the original Nelder-Mead algorithm is presented. The proposed algorithm's convergence is based on the principle of grid restrainment and therefore does not require sufficient descent as the recent convergent variant proposed by Price, Coope, and Byatt. Convergence properties of the proposed grid-restrained algorithm are analysed. Results of numerical testing are also included and compared to the results of the algorithm proposed by Price et al. The results clearly demonstrate that the proposed grid-restrained algorithm is an efficient direct search method.

Unconstrained derivative-free optimization by successive approximation

“…(10) guarantees the existence of limit pointB ∞ and assures us that no limit point contains zero vectors. The latter is a consequence of (8) and does not require an additional lower bound λ on b as in [4]. From (11) and (12) it follows that˜ k > α/(2Λ − α) > 0.…”

“…The proof again goes along the lines of [4], except that the grid-restrainment error must be applied twice.…”

“…(D) > 0 implies that D positively spans R n (for proof see [4]). Now suppose that the setD is obtained from D by restraining the members of D to grid G using partitioning P. Let d min denote the shortest member of D. Then (provided that no member ofD has zero length)…”

Optimization in Non-Standard Problems. An Application to the Provision of Public Inputs

“…According to Burmen et al (2005), the performance of this approach could be shortly summarized as follows.…”

“…First-best levels are always higher than the second-best ones in line with the mainstream of previous literature on 32 Variants of the original Nelder-Mead method are focused on avoiding problems related to this one. For further details, see Burmen et al (2005). 33 The differences in favour of RIM were initially more significant when the problems were straightforward implemented.…”