ATSC-NEX: Automated Time Series Classification With Sequential Model-Based Optimization and Nested Cross-Validation

Abstract

New methods to perform time series classification arise frequently and multiple state-of-theart approaches achieve high performance on benchmark datasets with respect to accuracy and computation time. However, often the modeling procedures do not include proper validation but rather rely only on either external test dataset or one-level cross-validation. ATSC-NEX is an automated procedure that employs sequential model-based optimization together with nested cross-validation to build an accurate and properly validated time series classification model. It aims to find an optimal pipeline configuration that includes the selection of input type and settings, as well as model type and hyperparameters. The results of a case study in which a model for the identification of diesel engine type is developed, show that the algorithm can efficiently find a well-performing pipeline configuration. The comparison between ATSC-NEX and some state-of-the-art methods on several benchmark datasets shows that similar accuracy can be achieved.

An Optimized Machine Learning and Deep Learning Framework for Facial and Masked Facial Recognition

“…Nested cross-validation is a method for assessing how well machine learning models are working. It is a development of conventional cross-validation, where an inner loop is used to tune hyperparameters on a validation set and an outer loop is used to assess model performance on a test set [30]. Different combinations of hyperparameters are tested in the inner loop using a cross-validation method, and the hyperparameters that produce the best results on the validation set are chosen for the outer loop.…”

A Novel Machine Learning-Based Approach for Induction Machine Fault Classifier Development—A Broken Rotor Bar Case Study

“…The presented model development approach is based on an early version of the ATSC-NEX algorithm proposed in [15]. An overview of the approach is shown in Figure 1.…”

“…We use SMBO to optimize, e.g., the model hyperparameters, and nested CV to estimate the generalization performance of the model. The software implementation is based on an early version of the ATSC-NEX algorithm presented in [15]. In addition, we evaluate the applicability of the random convolutional kernel transformation (ROCKET) algorithm on FFT data and the resulting fault identification performance and compare the performance to a feature-based approach.…”