Stopping criteria, initialization, and implementations of BFGS and their effect on the BBOB test suite

Abstract

Benchmarking algorithms is a crucial task to understand them and to make recommendations for which algorithms to use in practice. However, one has to keep in mind that we typically compare only algorithm implementations and that care must be taken when making general statements about an algorithm while implementation details and parameter settings might have a strong impact on the performance. In this paper, we investigate those impacts of initialization, internal parameter setting, and algorithm implementation over different languages for the well-known BFGS algorithm. We must conclude that even in the default setting, the BFGS algorithms in Python's scipy library and in Matlab's fminunc differ widely-with the latter even changing significantly over time.

A global surrogate assisted CMA-ES

“…Figure 1 shows empirical runtime distributions (in number of function evaluations) of BFGS (as "BFGS ros" [29] and BFGS-P-St [5]), NEWUOA [27,30], MCS [18], the Simplex Downhill method [25] enhanced with restarts [6] denoted as NELDERD, and SLSQP in dimension 20 on all functions, five subgroups of functions and the single functions 1, 6, 7, 10, 13, and 18.…”

Benchmarking multivariate solvers of SciPy on the noiseless testbed

“…Furthermore, the effect of decreasing the step length for the finite difference approximation of the gradient was investigated to some extent: decreasing the default value of this parameter (10 −8 ) can improve the performance on particular functions, such as the Ellipsoid, while it shows worse success ratio on others. A more detailed study was presented in [2] and in the following comparison it is set to its default value.…”