We introduce a new benchmark problem called Deceptive Leading Blocks (DLB) to rigorously study the runtime of the Univariate Marginal Distribution Algorithm (UMDA) in the presence of epistasis and deception. We show that simple Evolutionary Algorithms (EAs) outperform the UMDA unless the selective pressure µ/λ is extremely high, where µ and λ are the parent and offspring population sizes, respectively. More precisely, we show that the UMDA with a parent population size of µ = Ω(log n) has an expected runtime of e Ω(µ) on the DLB problem assuming any selective pressure µ λ ≥ 14 1000 , as opposed to the expected runtime of O nλ log λ + n 3 for the non-elitist (µ, λ) EA with µ/λ ≤ 1/e. These results illustrate inherent limitations of univariate EDAs against deception and epistasis, which are common characteristics of real-world problems. In contrast, empirical evidence reveals the efficiency of the bi-variate MIMIC algorithm on the DLB problem. Our results suggest that one should consider EDAs with more complex probabilistic models when optimising problems with some degree of epistasis and deception. * Preliminary version of this work will appear in the Proceedings of 15th ACM/SIGEVO Workshop on FoundationsEstimation of distribution algorithms (EDAs) [42,43,33] are a class of randomised search heuristics with many real-world applications (see [27] and references therein). Unlike traditional EAs, which define implicit models of promising solutions via genetic operations such as crossover and mutation, EDAs optimise objective functions by constructing and sampling explicit probabilistic models to generate offspring for the next iteration. The workflow of EDAs is an iterative process, where the initial model is a uniform distribution over the search space. The starting population consists of λ individuals sampled from the uniform distribution. A fitness function then scores each individual, and the algorithm selects the µ fittest individuals to update the model (where µ < λ). The procedure is repeated until some termination condition is fulfilled, which is usually a threshold on the number of iterations or on the quality of the fittest offspring [27, 19]. Many variants of EDAs have been proposed over the last decades. They differ in the way their models are represented, updated as well as sampled over iterations. In general, EDAs are categorised into two main classes: univariate and multivariate. Univariate EDAs take advantage of first-order statistics (i.e. the mean) to build a probability vector-based model and assume independence between decision variables. The probabilistic model is represented as an n-vector, where each component is called a marginal (also frequency) and n is the problem instance size. Typical univariate EDAs are compact Genetic Algorithm (cGA [25]), Univariate Marginal Distribution Algorithm (UMDA [42]) and Population-Based Incremental Learning (PBIL [3]). In contrast, multivariate EDAs apply higher-order statistics to model the correlations between decision variables of the addressed problems.Th...

Estimation of Distribution Algorithms (EDAs) are stochastic heuristics that search for optimal solutions by learning and sampling from probabilistic models. Despite their popularity in real-world applications, there is little rigorous understanding of their performance. Even for the Univariate Marginal Distribution Algorithm (UMDA) -a simple population-based EDA assuming independence between decision variables -the optimisation time on the linear problem OneMax was until recently undetermined. The incomplete theoretical understanding of EDAs is mainly due to lack of appropriate analytical tools.We show that the recently developed level-based theorem for non-elitist populations combined with anticoncentration results yield upper bounds on the expected optimisation time of the UMDA. This approach results in the bound O nλ log λ + n 2 on two problems, LeadingOnes and BinVal, for population sizes λ > µ = Ω(log n), where µ and λ are parameters of the algorithm. We also prove that the UMDA with population sizes µ ∈ O ( √ n) ∩ Ω(log n) optimises OneMax in expected time O (λn), and for larger population sizes µ = Ω( √ n log n), in expected time O (λ √ n). The facility and generality of our arguments suggest that this is a promising approach to derive bounds on the expected optimisation time of EDAs.

Unlike traditional evolutionary algorithms which produce offspring via genetic operators, Estimation of Distribution Algorithms (EDAs) sample solutions from probabilistic models which are learned from selected individuals. It is hoped that EDAs may improve optimisation performance on epistatic fitness landscapes by learning variable interactions. However, hardly any rigorous results are available to support claims about the performance of EDAs, even for fitness functions without epistasis. The expected runtime of the Univariate Marginal Distribution Algorithm (UMDA) on OneMax was recently shown to be in O (nλ log λ) [8]. Later, Krejca and Witt [15] proved the lower bound Ω (λ √ n + n log n) via an involved drift analysis . We prove a O (nλ) bound, given some restrictions on the population size. This implies the tight bound Θ (n log n) when λ = O (log n), matching the runtime of classical EAs. Our analysis uses the levelbased theorem and anti-concentration properties of the Poisson-binomial distribution. We expect that these generic methods will facilitate further analysis of EDAs.

The Population-Based Incremental Learning (PBIL) algorithm uses a convex combination of the current model and the empirical model to construct the next model, which is then sampled to generate offspring. The Univariate Marginal Distribution Algorithm (UMDA) is a special case of the PBIL, where the current model is ignored. Dang and Lehre (GECCO 2015) showed that UMDA can optimise LeadingOnes efficiently. The question still remained open if the PBIL performs equally well. Here, by applying the level-based theorem in addition to Dvoretzky-Kiefer-Wolfowitz inequality, we show that the PBIL optimises function LeadingOnes in expected time O nλ log λ + n 2 for a population size λ = Ω(log n), which matches the bound of the UMDA. Finally, we show that the result carries over to BinVal, giving the fist runtime result for the PBIL on the BinVal problem. the UMDA using truncation selection and derived the first upper bounds of O (nλ log λ) and O nλ log λ + n 2 on the expected optimisation times of the UMDA on OneMax and LeadingOnes, respectively, where the population size is λ = Ω(log n). These results were obtained using a relatively new technique called level-based analysis [3]. Very recently, Witt [13] proved that the UMDA optimises OneMax within O (µn) and O (µ √ n) when µ ≥ c log n and µ ≥ c ′ √ n log n for some constants c, c ′ > 0, respectively. However, these bounds only hold when λ = (1 + Θ(1))µ. This constraint on λ and µ was relaxed by Lehre and Nguyen [8], where the upper bound O (λn) holds for λ = Ω(µ) and c log n ≤ µ = O ( √ n)for some constant c > 0. The first rigorous runtime analysis of the PBIL [1], was presented very recently by Wu et al. [14]. In this work, the PBIL was referred to as a cross entropy algorithm.The study proved an upper bound O n 2+ε of the PBIL with margins [1/n, 1 − 1/n] on LeadingOnes, where λ = n 1+ε , µ = O(n ε/2 ), η ∈ Ω (1) and ε ∈ (0, 1). Until now, the known runtime bounds for the PBIL were significantly higher than those for the UMDA. Thus, it is of interest to determine whether the PBIL is less efficient than the UMDA, or whether the bounds derived in the early works were too loose. This paper makes two contributions. First, we address the question above by deriving a tighter bound O nλ log λ + n 2 on the expected optimisation time of the PBIL onLeadingOnes. The bound holds for population sizes λ = Ω (log n), which is a much weaker assumption than λ = ω(n) as required in [14]. Our proof is more straightforward than that in [14] because much of the complexities of the analysis are already handled by the level-based method [3]. The second contribution is the first runtime bound of the PBIL on BinVal. This

We perform a rigorous runtime analysis for the Univariate Marginal Distribution Algorithm on the LeadingOnes function, a well-known benchmark function in the theory community of evolutionary computation with a high correlation between decision variables. For a problem instance of size n, the currently best known upper bound on the expected runtime is O nλ log λ + n 2 (Dang and Lehre, GECCO 2015), while a lower bound necessary to understand how the algorithm copes with variable dependencies is still missing. Motivated by this, we show that the algorithm requires a e Ω(µ) runtime with high probability and in expectation if the selective pressure is low; otherwise, we obtain a lower bound of Ω( nλ log(λ−µ) ) on the expected runtime. Furthermore, we for the first time consider the algorithm on the function under a prior noise model and obtain an O n 2 expected runtime for the optimal parameter settings. In the end, our theoretical results are accompanied by empirical findings, not only matching with rigorous analyses but also providing new insights into the behaviour of the algorithm.