Can AutoML outperform humans? An evaluation on popular OpenML datasets using AutoML Benchmark

Abstract

In the last few years, Automated Machine Learning (AutoML) has gained much attention. With that said, the question arises whether AutoML can outperform results achieved by human data scientists. This paper compares four AutoML frameworks on 12 different popular datasets from OpenML; six of them supervised classification tasks and the other six supervised regression ones. Additionally, we consider a real-life dataset from one of our recent projects. The results show that the automated frameworks perform better or equal than the machine learning community in 7 out of 12 OpenML tasks. 

Leveraging Automated Machine Learning for Text Classification: Evaluation of AutoML Tools and Comparison with Human Performance

“…The fact that all three Kaggle competitions are clearly won by humans is understandable since apparently, these are the tasks in which contestants put as much effort as possible. This circumstance was already observed in the work of (Hanussek et al, 2020) and it underlines the insight that, at this time, AutoML cannot beat humans in situation in which extraordinary results are required. This is again shown by our discovery, that concerning the cases in which AutoML outperforms humans, this outperformance is rather little and in most of the cases only one or two AutoML tools manage to do so (although the average is 2.5, which is attributable to the first task where all four AutoML tools beat human performance).…”

Deep learning super‐resolution electron microscopy based on deep residual attention network

“…In addition, the model is applicable to other imaging equipment, such as X‐ray and microwave imaging equipment, and has broad development prospects in the future. AutoML 38 has achieved better performance than artificially designed networks in latest research studies. Thus, we consider applying AutoML to our research, which will be the key to our upcoming research topic.…”

Ten quick tips for biomarker discovery and validation analyses using machine learning

“…However, low-dimensional intuitions of patterns in high-dimensional data can also be misleading, if the sample distances in the original feature space are not well preserved and partly reflect idiosyncrasies of the visualization method [ 72 ]. To facilitate model selection for the non-expert, automated machine learning (AutoML) approaches have been proposed, which use combinatorial search algorithms and heuristics to replace manual tasks in model selection [ 73 ]. But not all models are suitable for all types of data.…”