A Revised Simplex Search Procedure for Stochastic Simulation Response Surface Optimization

Abstract

We develop a variant of the Nelder-Mead (NM) simplex search procedure for stochastic simulation optimization that is designed to avoid many of the weaknesses encumbering similar direct-search methods-in particular, excessive sensitivity to starting values, premature termination at a local optimum, lack of robustness against noisy responses, and computational inefficiency. The Revised Simplex Search (RSS) procedure consists of a three-phase application of the NM method in which: (a) the ending values for one phase become the starting values for the next phase; (b) the step size for the initial simplex (respectively, the shrink coefficient) decreases geometrically (respectively, increases linearly) over successive phases; and (c) the final estimated optimum is the best of the ending values for the three phases. To compare RSS versus NM and procedure RS؉S9 due to Barton and Ivey, we summarize a simulation study based on four selected performance measures computed for six test problems that include additive white-noise error, with three levels of problem dimensionality and noise variability used in each problem. In the selected test problems, RSS yielded significantly more accurate estimates of the optimum than NM or RS؉S9, and both RSS and RS؉S9 required roughly four times as many function evaluations as NM.

A benchmark of kriging-based infill criteria for noisy optimization

“…The full factorial structure of the benchmark allows the effects of each parameter to be estimated in an unbiased way, i. e., the change of performance obtained by changing a parameter can be assessed over all combinations of the remaining parameters. To do so, we postulate, for each test function, a linear model with second order interactions for the performance response, in the fashion of [16]. Each factor (noise level, budget, initial DOE size, covariance, LHS instance, criterion) is treated as categorical, and the model is generated using deviation coding (see [13], chap.…”

Simulation optimization: a review of algorithms and applications

“…There is some classical as well as relatively recent work done on investigating both pattern search methods [196,6,131] and Nelder-Mead simplex algorithms [144,19,94,31] and their convergence in the context of simulation optimization.…”

Quantile-Based Optimization of Noisy Computer Experiments With Tunable Precision

“…The strategy proposed in Section 6.2 is compared to the AEI method as proposed in Huang et al (2006) for the optimization of homogeneously noisy experiments, which has already been found to be very competitive compared to other local or global optimizers such as the revised simplex search (Humphrey and Wilson 2000) or DIRECT (Gablonsky and Kelley 2001). Both EQI and AEI heuristics are compared to the classical EI, using a noisy kriging model (as in Section 2.4) and with the minimal value of the observations replaced by the minimum of the kriging mean at the observations, which can be considered as the baseline approach.…”