AutoML to Date and Beyond: Challenges and Opportunities

Abstract

As big data becomes ubiquitous across domains, and more and more stakeholders aspire to make the most of their data, demand for machine learning tools has spurred researchers to explore the possibilities of automated machine learning (AutoML). AutoML tools aim to make machine learning accessible for non-machine learning experts (domain experts), to improve the efficiency of machine learning, and to accelerate machine learning research. But although automation and efficiency are among AutoML’s main selling points, the process still requires human involvement at a number of vital steps, including understanding the attributes of domain-specific data, defining prediction problems, creating a suitable training dataset, and selecting a promising machine learning technique. These steps often require a prolonged back-and-forth that makes this process inefficient for domain experts and data scientists alike and keeps so-called AutoML systems from being truly automatic. In this review article, we introduce a new classification system for AutoML systems, using a seven-tiered schematic to distinguish these systems based on their level of autonomy. We begin by describing what an end-to-end machine learning pipeline actually looks like, and which subtasks of the machine learning pipeline have been automated so far. We highlight those subtasks that are still done manually—generally by a data scientist—and explain how this limits domain experts’ access to machine learning. Next, we introduce our novel level-based taxonomy for AutoML systems and define each level according to the scope of automation support provided. Finally, we lay out a roadmap for the future, pinpointing the research required to further automate the end-to-end machine learning pipeline and discussing important challenges that stand in the way of this ambitious goal.

Addressing UX Practitioners’ Challenges in Designing ML Applications: an Interactive Machine Learning Approach

“…In traditional ML workflows, a ML practitioner first defines (or receives) model requirements and then completes data-oriented tasks (data collection, cleaning, and labeling), followed by model-oriented tasks (feature engineering, model training, evaluation, deployment, and monitoring), with some feedback loops in between [2]. Approaches to automating aspects of this workflow-such as selecting model architectures [38,66], tuning hyperparameters [26], and engineering features [41,60]-have been developed by the ML community under a paradigm known as automated machine learning (AutoML) [37]. Major cloud providers of AutoML [1, 18,29,32,36,44] believe that the paradigm can enable ML non-expert developers and semi-expert "citizen data scientists" to create fully-fledged, deployable models, often with little to no code [18,29,36,44].…”

There’s Plenty of Room Right Here: Biological Systems as Evolved, Overloaded, Multi-Scale Machines

“…Those are indeed dissimilar to living systems. There is now a widening array of computational substrates and robots that are often massively parallel (such as GPUs and computational metamaterials [ 8 ]), stochastic (hard to predict) [ 9 ], able to exploit non-obvious (and potentially not-yet-understood) properties of the exotic substrates they are built from [ 10 ], emergent, produced by evolutionary techniques [ 11 ], and built by other machines [ 12 ] or programmed by other algorithms [ 13 , 14 , 15 ]. The benefit of considering biological systems as members of this broader class is that it avails powerful conceptual frameworks from computer science to be deployed in biology in a deep way, and therefore to understand life far beyond its current limited use in computational biology.…”

Structure Learning and Hyperparameter Optimization Using an Automated Machine Learning (AutoML) Pipeline

“…In the framework of AutoML, the model's creation becomes easier, in the sense that it requires a minimal (or at the best scenario not at all) designer's manual correction. As a result, it can be viewed as an alternative to more traditional considerations [5,6], operating as a vehicle, to alleviate the high demands for experts in building ML applications [7]; this is a fact that appears to be very appealing in commercial software production [8]. In addition, under the umbrella of AutoML, the ML approaches become accessible to non-expert users [9], thus enabling the organizations to leverage the power of ML in effectively solving a variety of real-world problems [1].…”

“…The second stage feature engineering mechanisms to extract features from the raw data and to make convenient the design of ML algorithms in effectively describing the data [4]. Mainly projected on: (a) feature selection algorithms that act to reject features from the ori feature set, which appear to be redundant, or they deteriorate the model's perform [8], and (b) feature extraction methods that perform dimensionality reduction using cialized functional data transformations thus, altering the original data features [4,9] third stage concerns the model generation and is recognized as the very core of an toML approach. Model generation is performed in terms of two processes, namely, s ture identification and architecture optimization [4,9].…”