Recent advances in metaheuristic algorithms: Does the Makara dragon exist?

Fong, Simon; Wang, Xi; Xu, Qiaoqing; Wong, Raymond K.; Fiaidhi, Jinan; Mohammed, Sabah

doi:10.1007/s11227-015-1592-8

Cited by 25 publications

(14 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Draa (2015) showed, based on a wide selection of problems, that the performance of the flower pollination algorithm, one of the recently introduced approaches, turns out to be relatively poor when compared with classical algorithms, in contradiction to what is claimed in the literature. Similarly, Fong et al (2016) found the lack of superiority of a novel bat-inspired algorithm over the classical PSO. Finally, Piotrowski (2015) and Piotrowski and Napiorkowski (2016) showed that very different algorithms rankings may be derived when one performs tests either by searching for the minimum versus the maximum of exactly the same functions.…”

Section: Background: Recent Criticisms Of Optimization Metaheuristicsmentioning

confidence: 99%

“…Simon et al (2011) discussed the large similarities between biogeography-based optimization and many older metaheuristics. A number of recently introduced physical processes-based approaches are also considered to include mainly replications of the former methods (Salcedo-Sanz 2016), and a few core components that are widely used in various metaheuristics are identified in Fong et al (2016) to expose the lack of novelty in some of the most recent methods. According to Sörensen (2015), introducing such redundant approaches can be partly explained by the common practice of hiding the lack of methodological novelty behind the new nomenclature that comes from the "inspiration"-resulting in algorithms based on "quantum frogs", "competitive imperialists" or "intelligent water drops".…”

Section: Background: Recent Criticisms Of Optimization Metaheuristicsmentioning

confidence: 99%

See 1 more Smart Citation

Searching for structural bias in particle swarm optimization and differential evolution algorithms

Piotrowski

Napiorkowski

2016

Swarm Intell

View full text Add to dashboard Cite

During the last two decades, a large number of metaheuristics have been proposed, leading to various studies that call for a deeper insight into the behaviour, efficiency and effectiveness of such methods. Among numerous concerns that are briefly reviewed in this paper, the presence of a structural bias (i.e. the tendency, not justified by the fitness landscape, to visit some regions of the search space more frequently than other regions) has recently been detected in simple versions of the genetic algorithm and particle swarm optimization. As of today, it remains unclear how frequently such a behaviour occurs in population-based swarm intelligence and evolutionary computation methods, and to what extent structural bias affects their performance. The present study focuses on the search for structural bias in various variants of particle swarm optimization and differential evolution algorithms, as well as in the traditional direct search methods proposed by Nelder-Mead and Rosenbrock half a century ago. We found that these historical direct search methods are structurally unbiased. However, most tested new metaheuristics are structurally biased, and at least some presence of structural bias can be observed in almost all their variants. The presence of structural bias seems to be stronger in particle swarm optimization algorithms than in differential evolution algorithms. The relationships between the strength of the structural bias and the dimensionality of the search space, the number of allowed function calls and the population size are complex and hard to generalize. For 14 algorithms tested on the CEC2011 realworld problems and the CEC2014 artificial benchmarks, no clear relationship between the strength of the structural bias and the performance of the algorithm was found. However, at least for artificial benchmarks, such old and structurally unbiased methods like Nelder-Mead algorithm performed relatively well. This is a warning that the presence of structural bias in novel metaheuristics may hamper their search abilities.

show abstract

Section: Background: Recent Criticisms Of Optimization Metaheuristicsmentioning

confidence: 99%

Section: Background: Recent Criticisms Of Optimization Metaheuristicsmentioning

confidence: 99%

Searching for structural bias in particle swarm optimization and differential evolution algorithms

Piotrowski

Napiorkowski

2016

Swarm Intell

View full text Add to dashboard Cite

show abstract

“…Even without taking No Free Lunch theorems (Wolpert and Macready, 1997) into account, it is implausible to believe that this is true for all of them. This is not to say that the results are incorrect, but it does reflect the difficulty of designing fair comparative studies (Črepinšek et al, 2016;Fong et al, 2016;García-Martínez et al, 2017;Piotrowski, 2015). We can speculate that all of these algorithms will sometimes perform better on some problems when compared against other algorithms, since problem landscapes are diverse, and small differences in the topography of a landscape can favour different approaches.…”

Section: Discussionmentioning

confidence: 99%

Mitigating Metaphors: A Comprehensible Guide to Recent Nature-Inspired Algorithms

Lones

2019

SN COMPUT. SCI.

View full text Add to dashboard Cite

In recent years, there has been an explosion of new metaheuristic algorithms that explore different sources of inspiration within the biological and natural worlds. A particular issue with this approach is the tendency for authors to use terminology that is derived from the domain of inspiration, rather than the broader domains of metaheuristics and optimisation. This, in turn, makes it difficult to both comprehend these algorithms and understand their relationships to other metaheuristics. This guide attempts to address this issue, at least to some extent, by providing descriptions of the 32 most cited nature-inspired algorithms published in the last 20 years using standard metaheuristic terms. It also discusses commonalities between these algorithms and more classical nature-inspired metaheuristics such as evolutionary algorithms and particle swarm optimisation.

show abstract

Section: Applying Meta-heuristic Algorithm On Neural Network Trainingmentioning

confidence: 99%

“…Many strategies such as evolutionary strategy, social behaviour and information exchange are implemented, thereby many versions were made possible. Readers who want to probe into details of the variety of metaheuristics are referred to a latest review [19].Artificial neural network (ANN) are traditionally constructed in layout of multi-layered feed-forward neural network; some used back-propagation (BP) as error feedback to modify the weights for training up the cognitive power of the network. The weight training strategy has been traditionally gradient descent (GD).…”

mentioning

confidence: 99%

How Meta-heuristic Algorithms Contribute to Deep Learning in the Hype of Big Data Analytics

Fong¹,

Deb²,

Yang

2017

Advances in Intelligent Systems and Computing

Self Cite

View full text Add to dashboard Cite

Final accepted version (with author's formatting)This version is available at: http://eprints.mdx.ac.uk/22276/ Copyright:Middlesex University Research Repository makes the University's research available electronically.Copyright and moral rights to this work are retained by the author and/or other copyright owners unless otherwise stated. The work is supplied on the understanding that any use for commercial gain is strictly forbidden. A copy may be downloaded for personal, non-commercial, research or study without prior permission and without charge.Works, including theses and research projects, may not be reproduced in any format or medium, or extensive quotations taken from them, or their content changed in any way, without first obtaining permission in writing from the copyright holder(s). They may not be sold or exploited commercially in any format or medium without the prior written permission of the copyright holder(s).Full bibliographic details must be given when referring to, or quoting from full items including the author's name, the title of the work, publication details where relevant (place, publisher, date), pagination, and for theses or dissertations the awarding institution, the degree type awarded, and the date of the award.If you believe that any material held in the repository infringes copyright law, please contact the Repository Team at Middlesex University via the following email address:eprints@mdx.ac.ukThe item will be removed from the repository while any claim is being investigated. ccfong@umac.mo, suashdeb@gmail.com, x.yang@mdx.ac.uk Abstract. Deep learning (DL) is one of the most emerging type of contemporary machine learning techniques that mimic the cognitive patterns of animal visual cortex to learn the new abstract features automatically by deep and hierarchical layers. DL is believed to be a suitable tool so far for extracting insights from very huge volume of so-called big data. Nevertheless, one of the three "V" or big data is velocity that implies the learning has to be incremental as data are accumulating up rapidly. DL must be fast and accurate. By the technical design of DL, it is extended from feed-forward artificial neural network with many multi-hidden layers of neurons called deep neural network (DNN). In the training process of DNN, it has certain inefficiency due to very long training time required. Obtaining the most accurate DNN within a reasonable run-time is a challenge, given there are potentially many parameters in the DNN model configuration, and high-dimensionality of the feature space in the training dataset. Meta-heuristic has a history of optimizing machine learning models successfully. How well meta-heuristic could be used to optimize DL in the context of big data analytics is a thematic topic which we pondered on in this paper. As a position paper, we review the recent advances of applying metaheuristics on DL, discuss about their pros and cons, and point out some feasible research directions for bridging the gaps between meta-heuristics and DL.Keyword...

show abstract

Recent advances in metaheuristic algorithms: Does the Makara dragon exist?

Cited by 25 publications

References 34 publications

Searching for structural bias in particle swarm optimization and differential evolution algorithms

Searching for structural bias in particle swarm optimization and differential evolution algorithms

Mitigating Metaphors: A Comprehensible Guide to Recent Nature-Inspired Algorithms

How Meta-heuristic Algorithms Contribute to Deep Learning in the Hype of Big Data Analytics

Contact Info

Product

Resources

About