Log-log Convergence for Noisy Optimization

Abstract

Abstract. We consider noisy optimization problems, without the assumption of variance vanishing in the neighborhood of the optimum. We show mathematically that simple rules with exponential number of resamplings lead to a log-log convergence rate. In particular, in this case the log of the distance to the optimum is linear on the log of the number of resamplings. As well as with number of resamplings polynomial in the inverse step-size. We show empirically that this convergence rate is obtained also with polynomial number of resamplings. In this polynomial resampling setting, using classical evolution strategies and an ad hoc choice of the number of resamplings, we seemingly get the same rate as those obtained with specific Estimation of Distribution Algorithms designed for noisy setting. We also experiment non-adaptive polynomial resamplings. Compared to the state of the art, our results provide (i) proofs of log-log convergence for evolution strategies (which were not covered by existing results) in the case of objective functions with quadratic expectations and constant noise, (ii) log-log rates also for objective functions with expectation E[f (x)] = ||x − x * || p , where x * represents the optimum (iii) experiments with different parametrizations than those considered in the proof. These results propose some simple revaluation schemes. This paper extends [1].

Analysis of runtime of optimization algorithms for noisy functions over discrete codomains

“…The impact of resampling on the convergence rate has been empirically or theoretically investigated in the references [23,1,12,22,21]. We here focus on the adaptation of resampling works from continuous codomains [2] to discrete ones and we cover a broad class of optimizers stated in the next subsection.…”

A mathematically derived number of resamplings for noisy optimization

“…[3] proved mathematically that (i) a non-adaptive rule with exponential number of resamplings and (ii) an adaptive number of resamplings depending on the step-size can lead to log-log convergence. [3] has also shown experimentally that (iii) a non-adaptive rule with polynomial number of resamplings can lead to the log-log convergence, i.e. log ||xm|| 2 log m ∼ A < 0, where xm is the recommendation after m evaluations.…”

“…A large number of resamplings per individual may introduce a dissipation of budget (evaluations), hence the choice of the resampling number is important. The number of resamplings can be chosen by adaptive rules, such as estimating the noise level,possibly using Bernstein races [7], using the step-size [3,4], or in a non-adaptive manner [3,4]. [3] proved mathematically that (i) a non-adaptive rule with exponential number of resamplings and (ii) an adaptive number of resamplings depending on the step-size can lead to log-log convergence.…”

“…The number of resamplings can be chosen by adaptive rules, such as estimating the noise level,possibly using Bernstein races [7], using the step-size [3,4], or in a non-adaptive manner [3,4]. [3] proved mathematically that (i) a non-adaptive rule with exponential number of resamplings and (ii) an adaptive number of resamplings depending on the step-size can lead to log-log convergence. [3] has also shown experimentally that (iii) a non-adaptive rule with polynomial number of resamplings can lead to the log-log convergence, i.e.…”

“…the logarithm of the distance to the optimum typically scales linearly with the logarithm of number of evaluations) [9,3].…”