Evolution of Scikit-Learn Pipelines with Dynamic Structured Grammatical Evolution

Abstract

The deployment of Machine Learning (ML) models is a difficult and time-consuming job that comprises a series of sequential and correlated tasks that go from the data pre-processing, and the design and extraction of features, to the choice of the ML algorithm and its parameterisation. The task is even more challenging considering that the design of features is in many cases problem specific, and thus requires domain-expertise. To overcome these limitations Automated Machine Learning (AutoML) methods seek to automate, with few or no human-intervention, the design of pipelines, i.e., automate the selection of the sequence of methods that have to be applied to the raw data. These methods have the potential to enable non-expert users to use ML, and provide expert users with solutions that they would unlikely consider. In particular, this paper describes AutoML-DSGE -a novel grammar-based framework that adapts Dynamic Structured Grammatical Evolution (DSGE) to the evolution of Scikit-Learn classification pipelines. The experimental results include comparing AutoML-DSGE to another grammar-based AutoML framework, Resilient Classification Pipeline Evolution (RECIPE), and show that the average performance of the classification pipelines generated by AutoML-DSGE is always superior to the average performance of RECIPE; the differences are statistically significant in 3 out of the 10 used datasets.

Automatic Classification of Text Complexity

“…Finally, from a computational point-of-view, interesting future lines of research regard: the use of semantic features [75,76] for the task analyzed in this article, the automatic augmentation of a text dataset used, not for semantic, but syntactic and linguistic tasks, and the use of evolutionary search techniques (such as [77][78][79][80]) for automatically tuning the classification engine without any manual intervention.…”

Mild Cognitive Impairment Detection Using Machine Learning Models Trained on Data Collected from Serious Games

“…A pipeline in the context of ML can be described as a utility method that allows the design of a procedure from the data preprocessing to the training of the classifier offering some advantages over the manual execution of these steps. The purpose of the pipeline is to assemble the above methods that can be cross-validated together while setting different parameters in the context of using the Scikit-learn library [55]. The pipeline method eventually implements the solution for avoiding data leakage.…”

Grammar-based evolutionary approach for automated workflow composition with domain-specific operators and ensemble diversity

“…Moreover, some researchers have employed grammars to define the structure of a valid workflow, enabling them to avoid, for example, the application of a neural network to a dataset with categorical features. Both G3P [23,22] and grammatical evolution [43,44] have been applied to guide the optimisation process. However, their grammars still impose restrictions on the workflow structure.…”