SOP: parallel surrogate global optimization with Pareto center selection for computationally expensive single objective problems

Abstract

This paper presents a parallel surrogate-based global optimization method for computationally expensive objective functions that is more effective for larger numbers of processors. To reach this goal, we integrated concepts from multi-objective optimization and tabu search into, single objective, surrogate optimization. Our proposed derivative-free algorithm, called SOP, uses non-dominated sorting of points for which the expensive function has been previously evaluated. The two objectives are the expensive function value of the point and the minimum distance of the point to previously evaluated points. Based on the results of non-dominated sorting, P points from the sorted fronts are selected as centers from which many candidate points are generated by random perturbations. Based on surrogate approximation, the best candidate point is subsequently selected for expensive evaluation for each of the P centers, with simultaneous computation on P processors. Centers that previously did not generate good solutions are tabu with a given tenure. We show almost sure convergence of this algorithm under some conditions. The performance of SOP is compared with two RBF based methods. The test results show that SOP is an efficient method that can reduce time required to find a good near optimal solution. In a number of cases the efficiency Electronic supplementary material The online version of this article (123 J Glob Optim of SOP is so good that SOP with 8 processors found an accurate answer in less wall-clock time than the other algorithms did with 32 processors.

GOPS: efficient RBF surrogate global optimization algorithm with high dimensions and many parallel processors including application to multimodal water quality PDE model calibration

“…The proposed SOP algorithm showed good speed up with up to 64 processors per iteration. In previous studies before their SOP study (Krityakierne et al 2016), the maximum number of processors in parallel with global surrogate optimization was not larger than 10. Given P processors, SOP selects P evaluation points for the next iteration from candidate points generated around P center points.…”

“…The value of the weight between the surrogate estimation and distance criteria is varied to select as many evaluation points as there are processors. Krityakierne et al (2016) proposed the SOP algorithm for parallel computation and reported that there are a few studies on parallel surrogate global optimization that scaled up to many processors. The proposed SOP algorithm showed good speed up with up to 64 processors per iteration.…”

“…GOPS uses some features from the earlier parallel algorithm SOP (Krityakierne et al 2016) and adds important new features to improve performance quite significantly, especially with many processors and decision vectors of high dimensions. GOPS has new strategies to dynamically change the diversity of candidate points generated by multiple sampling centers.…”

“…Centers in each iteration are selected based on the non-dominated sorting techniques (Krityakierne et al 2016). In each iteration, all the evaluated points in S (n) are ranked based on two objectives, (1) the objective function value f ( i ) and (2) the negative of minimum distance from i to all other evaluated points S (n) � i (denoted as (n)…”

“…GOPS uses some features of the earlier SOP algorithm (Krityakierne et al 2016) but improves on earlier algorithms by (a) new centers are selected based on bivariate non-dominated sorting of previously evaluated points with additional constraints to ensure the objective value is below a target percentile and (b) as iterations increase, the number of centers decreases and the number of evaluation points per center increases. These features in GOPS are not present in earlier algorithms, which makes GOPS more robust and faster to converge.…”