Heavy tails with parameter adaptation in random projection based continuous EDA

Sanyang, Momodou L.; Kabán, Ata

doi:10.1109/cec.2015.7257140

Cited by 7 publications

(12 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In the rightmost plot we show the evolution of the best fitness. Superimposed, we also show the trajectories of competing state of the art methods: EDA-MCC [5], RP-EDA [9], and tRP-EDA [12]. All use the same budget and same population size.…”

Section: Resultsmentioning

confidence: 99%

“…There are also methods that apply dimensionality reduction techniques to reduce the dimension of the problems in order to avail EDA the opportunity to demonstrate its capabilities. An example of this type of techniques are random projections [9], [12] and [14]. However, none of these methods have been designed to take advantage of the intrinsic structure of the problems as our REMEDA approach does.…”

Section: Related Workmentioning

confidence: 99%

“…In this paper we further develop the random embedding technique, and introduce it to evolutionary search, by employing it to scale up Estimation of Distribution Algorithms (EDA) for problems with low intrinsic dimension. Although the underlying theoretical considerations are applicable to any optimisation method, our focus on EDA is due to it being one of the most successful methods in low dimensional problems [11] and most unsuccessful or expensive in high dimensions [5,9,12].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

REMEDA: Random Embedding EDA for Optimising Functions with Intrinsic Dimension

Sanyang

Kabán

2016

Parallel Problem Solving From Nature – PPSN XIV

Self Cite

View full text Add to dashboard Cite

Where a licence is displayed above, please note the terms and conditions of the licence govern your use of this document. When citing, please reference the published version. Take down policy While the University of Birmingham exercises care and attention in making items available there are rare occasions when an item has been uploaded in error or has been deemed to be commercially or otherwise sensitive.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

REMEDA: Random Embedding EDA for Optimising Functions with Intrinsic Dimension

Sanyang

Kabán

2016

Parallel Problem Solving From Nature – PPSN XIV

Self Cite

View full text Add to dashboard Cite

show abstract

“…This allows us to vary the problem size and observe the trends in performance comparatively for Gaussian and Cauchy search distributions. Among alternatives that could be used, the random projection ensemble based EDAs [9], [17] were specifically designed for high dimensional problems. Since testing in low dimensional regimes (e.g.…”

Section: Presentation Of the Algorithm Used In This Workmentioning

confidence: 99%

How effective is Cauchy-EDA in high dimensions?

Sanyang

Durrant

Kabán

2016

2016 IEEE Congress on Evolutionary Computation (CEC)

Self Cite

View full text Add to dashboard Cite

We consider the problem of high dimensional blackbox optimisation via Estimation of Distribution Algorithms (EDA) and the use of heavy-tailed search distributions in this setting. Some authors have suggested that employing a heavy tailed search distribution, such as a Cauchy, may make EDA better explore a high dimensional search space. However, other authors have found Cauchy search distributions are less effective than Gaussian search distributions in high dimensional problems. In this paper, we set out to resolve this controversy. To achieve this we run extensive experiments on a battery of high-dimensional test functions, and develop some theory which shows that small search steps are always more likely to move the search distribution towards the global optimum than large ones and, in particular, large search steps in high-dimensional spaces nearly always do badly in this respect. We hypothesise that, since exploration by large steps is mostly counterproductive in high dimensions, and since the fraction of good directions decays exponentially fast with increasing dimension, instead one should focus mainly on finding the right direction in which to move the search distribution. We propose a minor change to standard Gaussian EDA which implicitly achieves this aim, and our experiments on a sequence of test functions confirm the good performance of our new approach.

show abstract

“…When it comes to multivariate models, since the dependencies of variables should be considered, a large population must be involved for the accuracy of the model estimation, which results in a high computational complexity. One can employ randomised dimensionality reduction for a cheap and effective way to decrease the dimension of the search space [7], [8]. Other approaches include the use of multiple populations to overcome the above mentioned problems [9].…”

Section: Introductionmentioning

confidence: 99%