Evolutionary neural AutoML for deep learning

Liang, Jason; Meyerson, Elliot; Hodjat, Babak; Fink, Dan; Mutch, Karl; Miikkulainen, Risto

doi:10.1145/3321707.3321721

Cited by 98 publications

(40 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The performance of BP algorithm and NEAT algorithm reduce sharply when iterations is the same as that of GA-CANS. In addition, one kind of AutoML [30], autosklearn, is used to compare with GA-CANS, which takes a long time to get the optimal model. When the training time of auto-sklearn is reduced to the same as that of GA-CANS, the appropriate network model cannot be obtained in a short time, so the accuracy reduces rapidly.…”

Section: ) Improvement Of Roulette Methods On Invalid Connection Deletionmentioning

confidence: 99%

Correlation Analysis-Based Neural Network Self-Organizing Genetic Evolutionary Algorithm

et al. 2019

View full text Add to dashboard Cite

Recent years, there has been an ever increasing interest and investment on Artificial Intelligence (AI), both academic and industrial. As the hotspots in AI, Artificial Neural Networks (ANNs) have already been applied to a lot of different applications. However, traditional ANNs have disadvantages, such as fixed and redundant structure, resulting in requirement of large amount of training data and training time. Biological researches have shown that the biological neural network behaves in a more flexible way, with synapses building or withering according to requirement. In this paper, we present a Correlation Analysis Based Neural Network Self-Organizing Genetic Evolutionary Algorithm. Based on correlation analysis of training process, self-organizing combined with genetic evolutionary algorithm is applied to improve the performance efficiency and structural efficiency of the built neural network. Results show that our algorithm could generate neural networks with more compact structure and reasonable classification accuracy.

show abstract

Section: ) Improvement Of Roulette Methods On Invalid Connection Deletionmentioning

confidence: 99%

Correlation Analysis-Based Neural Network Self-Organizing Genetic Evolutionary Algorithm

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Some studies explore a large number of datasets [4], [5]. Our comparison adopts 12 datasets, which is below the two mentioned works but is still higher than used in eleven other studies (e.g., [6], [7]). More importantly, we consider eight AutoML technologies, which is a number only outperformed by [8] (which tested only one dataset) and [9] (which did not use any datasets).…”

Section: Related Workmentioning

confidence: 99%

A Comparison of AutoML Tools for Machine Learning, Deep Learning and XGBoost

Ferreira

Pilastri²,

Martins³

et al. 2021

2021 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

This paper presents a benchmark of supervised Automated Machine Learning (AutoML) tools. Firstly, we analyze the characteristics of eight recent open-source AutoML tools (Auto-Keras, Auto-PyTorch, Auto-Sklearn, AutoGluon, H2O AutoML, rminer, TPOT and TransmogrifAI) and describe twelve popular OpenML datasets that were used in the benchmark (divided into regression, binary and multi-class classification tasks). Then, we perform a comparison study with hundreds of computational experiments based on three scenarios: General Machine Learning (GML), Deep Learning (DL) and XGBoost (XGB). To select the best tool, we used a lexicographic approach, considering first the average prediction score for each task and then the computational effort. The best predictive results were achieved for GML, which were further compared with the best OpenML public results. Overall, the best GML AutoML tools obtained competitive results, outperforming the best OpenML models in five datasets. These results confirm the potential of the general-purpose AutoML tools to fully automate the Machine Learning (ML) algorithm selection and tuning.

show abstract

“…Therefore, the networks are often trained for a fixed (low) number of epochs (e.g., 8-10 epochs). To overcome the burden of evolution we can use clusters of Graphic Processing Units (GPUs) (e.g., Amazon AWS, or Google Cloud) [22], evaluate the candidate solutions in a limited amount of data instances [7], or train for a fixed amount of epochs/time and let evolution resume the training in a subsequent generation by loading the previous weights [23]. In the current work we use a variant of Deep Evolutionary Network Structured Representation (DENSER) [20] to search for Convolutional Neural Networks (CNNs) to distinguish between gamma radiations and protons.…”

Section: Neuroevolutionmentioning

confidence: 99%

Automatic Design of Artificial Neural Networks for Gamma-Ray Detection

Assunção

Correia

Conceição³

et al. 2019

IEEE Access

View full text Add to dashboard Cite

The goal of this work is to investigate the possibility of improving current gamma/hadron discrimination based on their shower patterns recorded on the ground. To this end we propose the use of Convolutional Neural Networks (CNNs) for their ability to distinguish patterns based on automatically designed features. In order to promote the creation of CNNs that properly uncover the hidden patterns in the data, and at same time avoid the burden of hand-crafting the topology and learning hyper-parameters we resort to NeuroEvolution; in particular we use Fast-DENSER++, a variant of Deep Evolutionary Network Structured Representation. The results show that the best CNN generated by Fast-DENSER++ improves by a factor of 2 when compared with the results reported by classic statistical approaches. Additionally, we experiment ensembling the 10 best generated CNNs, one from each of the evolutionary runs; the ensemble leads to an improvement by a factor of 2.3. These results show that it is possible to improve the gamma/hadron discrimination based on CNNs that are automatically generated and are trained with instances of the ground impact patterns.

show abstract

Evolutionary neural AutoML for deep learning

Cited by 98 publications

References 33 publications

Correlation Analysis-Based Neural Network Self-Organizing Genetic Evolutionary Algorithm

Correlation Analysis-Based Neural Network Self-Organizing Genetic Evolutionary Algorithm

A Comparison of AutoML Tools for Machine Learning, Deep Learning and XGBoost

Automatic Design of Artificial Neural Networks for Gamma-Ray Detection

Contact Info

Product

Resources

About