Analysis of the AutoML Challenge Series 2015–2018

Guyon, Isabelle; Sun-Hosoya, Lisheng; Boullé, Marc; Escalante, Hugo Jair; Escalera, Sérgio; Liu, Zhengying; Jajetic, Damir; Ray, Bisakha; Saeed, Mehreen; Sebag, Michèle; Statnikov, Alexander; Tu, Wei-Wei; Viegas, Evelyne

doi:10.1007/978-3-030-05318-5_10

Cited by 88 publications

(84 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Few participants used this model in combination with another one (either a linear model or a Gaussian process). This trend is in agreement with other challenges12 [7], and in general with supervised learning trends. It is also interesting that even when this was an AutoML challenge most participants did not perform an optimization of hyperparameters, this could be due to the limited resources and the size of the datasets.…”

Section: Overview Of Resultssupporting

confidence: 91%

AutoML @ NeurIPS 2018 Challenge: Design and Results

Tu¹,

Guyon

Silver

et al. 2019

The NeurIPS '18 Competition

Self Cite

View full text Add to dashboard Cite

We organized a competition on Autonomous Lifelong Machine Learning with Drift that was part of the competition program of NeurIPS 2018. This data driven competition asked participants to develop computer programs capable of solving supervised learning problems where the i.i.d. assumption did not hold. Large data sets were arranged in a lifelong learning and evaluation scenario and CodaLab was used as the challenge platform. The challenge attracted more than 300 participants in its two month duration. This chapter describes the design of the challenge and summarizes its main results.

show abstract

Section: Overview Of Resultssupporting

confidence: 91%

AutoML @ NeurIPS 2018 Challenge: Design and Results

Tu¹,

Guyon

Silver

et al. 2019

The NeurIPS '18 Competition

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this section, we evaluate the usefulness of AutoML for application in business and industry by empirically comparing the most successful automated machine learning algorithms with (a) an industrial prototype as well as (b) a straight-forward improvement inspired by Hyperband [13], [43] (c). This selection spans a wide range of different approaches for pipeline optimization (see Section III) to tackle the CASH problem: the industrial prototype DSM [44] uses random model and hyperparameter search and thus serves as a baseline; Auto-sklearn [17] has won the recent AutoML challenges [33]. Additionally, we report results with TPOT [45], which is developed based on genetic programming [40] instead of Auto-sklearn's Bayesian optimization.…”

Section: Experimental Evaluationmentioning

confidence: 99%

“…TPOT [45]: This algorithm uses tree-based classifiers which is similar to the second entry of the latest AutoML challenge [33]. TPOT differs from the other presented methods since it used Genetic programming for optimization.…”

Section: Experimental Evaluationmentioning

confidence: 99%

“…To compare the presented automated machine learning approaches, we select some datasets with various classification and regression tasks from previous AutoML challenges [33]. We chose all datasets that are supported by DSM, TPOT and Auto-sklearn.…”

Section: Experimental Evaluationmentioning

confidence: 99%

“…PERFORMANCE OF SELECTED AUTOMATED MACHINE LEARNING ALGORITHMS ON AUTOML CHALLENGE DATA SETS[33]. "TEST" REFERS TO OUR OWN TEST SPLIT; "TIME" IS IN MINUTES; "PORTFOLIO HYPERBAND" ONLY TESTED ON BINARY CLASSIFICATION.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Automated Machine Learning in Practice: State of the Art and Recent Results

Tuggener

Amirian²,

Rombach³

et al. 2019

2019 6th Swiss Conference on Data Science (SDS)

View full text Add to dashboard Cite

A main driver behind the digitization of industry and society is the belief that data-driven model building and decision making can contribute to higher degrees of automation and more informed decisions. Building such models from data often involves the application of some form of machine learning. Thus, there is an ever growing demand in work force with the necessary skill set to do so. This demand has given rise to a new research topic concerned with fitting machine learning models fully automatically-AutoML. This paper gives an overview of the state of the art in AutoML with a focus on practical applicability in a business context, and provides recent benchmark results on the most important AutoML algorithms.

show abstract

Algebraic surrogate‐based process optimization using Bayesian symbolic learning

2023

View full text Add to dashboard Cite

Here, we propose a strategy for the global optimization of process flowsheets, a fundamental problem in process systems engineering, based on algebraic surrogates that are built from rigorous simulations via Bayesian symbolic regression. The applied method provides a closed‐form expression that can be optimized to global optimality using state‐of‐the‐art solvers, where BARON or ANTIGONE were the solvers of choice. When predicting unseen test data, the algebraic models show a similar accuracy level compared to traditional surrogates based on Gaussian processes. However, they can be more easily optimized to global optimality due to their analytical closed‐form structure, which allows the user to apply well‐established global deterministic solvers. We show the capabilities of our approach in several case studies, ranging from process units to full flowsheets. The performance of our approach is assessed by comparing the CPU time for model building, the prediction accuracy of the identified model, and the CPU time for the subsequent optimization with a proven benchmark.

show abstract

Analysis of the AutoML Challenge Series 2015–2018

Cited by 88 publications

References 40 publications

AutoML @ NeurIPS 2018 Challenge: Design and Results

AutoML @ NeurIPS 2018 Challenge: Design and Results

Automated Machine Learning in Practice: State of the Art and Recent Results

Algebraic surrogate‐based process optimization using Bayesian symbolic learning

Contact Info

Product

Resources

About