Exponential natural evolution strategies

Abstract

The family of natural evolution strategies (NES) offers a principled approach to real-valued evolutionary optimization by following the natural gradient of the expected fitness. Like the well-known CMA-ES, the most competitive algorithm in the field, NES comes with important invariance properties. In this paper, we introduce a number of elegant and efficient improvements of the basic NES algorithm. First, we propose to parameterize the positive definite covariance matrix using the exponential map, which allows the covariance matrix to be updated in a vector space. This new technique makes the algorithm completely invariant under linear transformations of the underlying search space, which was previously achieved only in the limit of small step sizes. Second, we compute all updates in the natural coordinate system, such that the natural gradient coincides with the vanilla gradient. This way we avoid the computation of the inverse Fisher information matrix, which is the main computational bottleneck of the original NES algorithm. Our new algorithm, exponential NES (xNES), is significantly simpler than its predecessors. We show that the various update rules in CMA-ES are closely related to the natural gradient updates of xNES. However, xNES is more principled than CMA-ES, as all the update rules needed for covariance matrix adaptation are derived from a single principle. We empirically assess the performance of the new algorithm on standard benchmark functions.

A Natural Evolution Strategy for Multi-objective Optimization

“…Natural evolution strategies (NES) [3,[8][9][10][11] are a class of evolutionary algorithms for real-valued optimization. They maintain a Gaussian search distribution with fully adaptive covariance matrix.…”

“…In this paper we build upon the most recent NES variant, exponential NES (xNES), first presented in [3]. We start with stating its working principles, which are needed later on to cleanly derive its hillclimber variant.…”

“…The exponential map M → exp(M ) = ∞ n=0 1 n! M n for symmetric matrices is used to encode the covariance matrix, resulting in a multiplicative form of the covariance matrix update (see [3] for details).…”

“…This amounts to a (non-redundant) representation similar to CMA-ES, that is, we split off a global step size variable from the covariance matrix in the form A = σ · B, with det(B) = 1. We obtain the corresponding gradient components [3] for the full derivation). Let η µ , η σ , and η B denote learning rates for the different parameter components.…”

“…The recently introduced family of natural evolution strategies (NES [3,[8][9][10][11]), consists in an optimization method that follows a sampled natural gradient of the expected fitness, and as such, provides a more principled alternative to CMA-ES. In this paper we combine the well-founded framework of NES with the proven approach of tackling MOO using evolution strategies.…”