A Tutorial on the Cross-Entropy Method

Abstract

Abstract:The cross-entropy method is a recent versatile Monte Carlo technique. This article provides a brief introduction to the cross-entropy method and discusses how it can be used for rare-event probability estimation and for solving combinatorial, continuous, constrained and noisy optimization problems. A comprehensive list of references on cross-entropy methods and applications is included.Keywords: cross-entropy, Kullback-Leibler divergence, rare events, importance sampling, stochastic search.The cross-entropy (CE) method is a recent generic Monte Carlo technique for solving complicated simulation and optimization problems. The approach was introduced by R.Y. Rubinstein in [41,42], extending his earlier work on variance minimization methods for rare-event probability estimation [40].The CE method can be applied to two types of problem:, where X is a random variable or vector taking values in some set X and H is function on X . An important special case is the estimation of a probability = P(S(X) γ), where S is another function on X .2. Optimization: Optimize (that is, maximize or minimize) S(x) over all x ∈ X , where S is some objective function on X . S can be either a known or a noisy function. In the latter case the objective function needs to be estimated, e.g., via simulation.In the estimation setting, the CE method can be viewed as an adaptive importance sampling procedure that uses the cross-entropy or Kullback-Leibler divergence as a measure of closeness between two sampling distributions, as is explained further in Section 1. In the optimization setting, the optimization problem is first translated into a rare-event estimation problem, and then the CE method for estimation is used as an adaptive algorithm to locate the optimum, as is explained further in Section 2.An easy tutorial on the CE method is given in [15]. A more comprehensive treatment can be found in [45]; see also [46, Chapter 8]. The CE method homepage can be found at www.cemethod.org .The CE method has been successfully applied to a diverse range of estimation and optimization problems, including buffer allocation [1], queueing models of telecommunication systems [14,16], optimal control of HIV/AIDS spread [48,49], signal detection [30], combinatorial auctions [9], DNA sequence alignment [24,38], scheduling and vehicle routing [3,8,11,20,23,53], neural and reinforcement learning [31,32,34,52,54], project management [12], rare-event simulation with light-and heavy-tail distributions [2,10,21,28], clustering analysis [4,5,29]. Applications to classical combinatorial optimization problems including the max-cut, traveling salesman, and Hamiltonian cycle 1

Controlling the number of HIV infectives in a mobile population

“…A gentle tutorial can be found in [8]. The Cross-Entropy (CE) method is an iterative method that involves the following two phases:…”

Cumulative weighting optimization

“…We use the same approach suggested by Rubinstein [22], and De Boer et al [23] for solving these problems. Each distance matrix D is given an initial state probability transition matrix, whose (i, j)th element specifies the probability of transitioning from city i to city j.…”

“…Without loss of generality, we will assume the samples are sorted according to their values (i.e., H x i k < H x j k if and only if i < j). A detailed discussion of the admissible tour generation process can be found in de Boer et al [23]. The CWO algorithm differs from other algorithms in how it updates its transition matrix.…”

Adaptation of Cross Entropy optimisation to a dynamic Bridge WIM calibration problem

“…Like the Genetic Algorithm [34] CE is a population based method of optimization. It can be described as an iterative procedure composed of two steps [35]; firstly a large portion of the population of solutions with high objective function values is culled; secondly, the remaining solutions are used to generate a new population. The CE method is used here to determine those global stiffness and mass matrices of the FE plate model that give a best fit of theoretical to measured strains for all runs of the pre-weighed calibration truck(s).…”