An Examination of the State-of-the-Art for Multivariate Time Series Classification

Dhariyal, Bhaskar; Nguyen, Thach Le; Gsponer, Severin; Ifrim, Georgiana

doi:10.1109/icdmw51313.2020.00042

Cited by 14 publications

(4 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The recent empirical surveys (Ruiz et al, 2020;Dhariyal et al, 2020) provide a detailed overview of progress in MTSC. This section describes a subset of the MTSC classifiers discussed in these surveys, that we used in this work to evaluate the impact of channel selection methods.…”

Section: Multivariate Time Series Classificationmentioning

confidence: 99%

“…To evaluate the impact of channel selection, we work with recent multivariate time series classifiers, ROCKET (Dempster et al, 2020), Weasel-Muse (Schäfer and Leser, 2018) and MrSEQL-SAX (Le Nguyen et al, 2019a). These approaches were shown to have state-of-the-art accuracy and can complete training and prediction on the full UEA MTSC benchmark in less than 7 days (Dhariyal et al, 2020;Ruiz et al, 2020). One important aspect of the classifiers mentioned above is that they use data from all the channels during training, and it is evident from the accuracy and runtime of these algorithms that this is not the best strategy.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Scalable Classifier-Agnostic Channel Selection for Multivariate Time Series Classification

Dhariyal¹,

Nguyen²,

Ifrim³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Accuracy is a key focus of current work in time series classification. However, in many applications speed and data reduction is equally important, especially when the data scale and storage requirements increase rapidly. Current multivariate time series classification algorithms need hundreds of compute hours to complete training and prediction. This is due to the nature of multivariate time series data which grows with the number of time series, their length and the number of channels. In many applications not all the channels are useful for the classification task, hence we require methods that can efficiently select useful channels and thus save computational resources. We propose and evaluate two methods for channel selection. Our techniques work by representing each class by a prototype time series and performing channel selection based on the prototype-distance between classes. The main hypothesis is that useful channels enable better separation between classes and hence channels with higher distance between class-prototypes are more useful. On the UEA Multivariate Time Series Classification (MTSC) benchmark we show that these techniques achieve significant data reduction and classifier speedup, for similar levels of classification accuracy. Channel selection is applied as a pre-processing step before training state-of-the-art MTSC algorithms and saves about 70% of computation time and data storage, with preserved accuracy. Furthermore, our methods enable even efficient classifiers, such as ROCKET, to achieve better accuracy as compared to using no channel selection or forward channel selection. To further study the impact of our techniques we present experiments on classifying synthetic multivariate time series datasets with more than 100 channels, as well as a real-world case study on a dataset with 50 channels. We find that our channel selection methods lead to significant data reduction with preserved or improved accuracy.

show abstract

Section: Multivariate Time Series Classificationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Scalable Classifier-Agnostic Channel Selection for Multivariate Time Series Classification

Dhariyal¹,

Nguyen²,

Ifrim³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Existing reviews for time-series classification focus on comparing algorithms that do not generate early predictions. Representative reviews for standard time-series classification methods, as well as their empirical comparison can be found in [1], [4], [8], [11], [34]. Furthermore, ETSC methods are mostly evaluated and compared against only a few alternative algorithms.…”

Section: Introductionmentioning

confidence: 99%

Early Time-Series Classification Algorithms: An Empirical Comparison

Akasiadis¹,

Evgenios²,

Michelioudakis³

et al. 2022

Preprint

View full text Add to dashboard Cite

Early Time-Series Classification (ETSC) is the task of predicting the class of incoming time-series by observing as few measurements as possible. Such methods can be employed to obtain classification forecasts in many time-critical applications. However, available techniques are not equally suitable for every problem, since differentiations in the data characteristics can impact algorithm performance in terms of earliness, accuracy, F1-score, and training time. We evaluate six existing ETSC algorithms on publicly available data, as well as on two newly introduced datasets originating from the life sciences and maritime domains. Our goal is to provide a framework for the evaluation and comparison of ETSC algorithms and to obtain intuition on how such approaches perform on real-life applications. The presented framework may also serve as a benchmark for new related techniques.

show abstract

“…We must mention that the superiority of ROCKET was also reported in independent works for other (not necessarily multivariate) time series classification tasks, including the studies delivered by Dempster et al [9]- [11], Salehinejad et al [33], Pantiskas et al [30] and more. Among papers emphasizing the superiority of ROCKET is the work of Dhariyal et al [8] who focused on the comparisons of ROCKET with neural approaches. The authors conclude their findings saying that "recent deep learning MTSC methods do not perform as well as expected.…”

mentioning

confidence: 99%

Variable-Length Multivariate Time Series Classification Using ROCKET: A Case Study of Incident Detection

Bier

Jastrzębska

Olszewski³

2022

IEEE Access

View full text Add to dashboard Cite

Multivariate time series classification is a machine learning problem that can be applied to automate a wide range of real-world data analysis tasks. ROCKET proved to be an outstanding algorithm capable to classify time series accurately and quickly. The textbook variant of the multivariate time series classification problem assumes that time series to be classified are all of the same length, while in realworld applications this assumption not necessarily holds. The literature of this domain does not pay enough attention to data processing pipelines for variable-length time series. Thus, in this paper, we present a thorough analysis of three preprocessing pipelines that handle variable-length time series that need to be classified with a method that requires the data to be of equal length. These three methods are truncation, padding, and forecasting of missing value. Experiments conducted on benchmark datasets, showed that the recommended procedure involves padding. Forecasting ensures similar classification accuracy, but comes at a much higher computational cost. Truncation is not a viable option. Furthermore, in the paper, we present a novel domain of application of multivariate time series classification algorithms, that is incident detection in cash transactions. This area poses substantive challenges for automated model training procedures since the data is not only variable-length, but also heavily imbalanced. In the study, we list various incident types and present trained classifiers capable to aid human auditors in their daily work. INDEX TERMS classification, incident detection, multivariate time series, ROCKET, varying-length time series I. INTRODUCTIONT IME series classification has become a vital domain of machine learning. The multitude of exciting real-life applications drives the development of the field and inspires fruitful research that aim at delivering new approaches, improving the existing ones, and adapting them to new types of data. This paper focuses on the task of classifying multivariate time series of unequal lengths. That is, each multivariate series can have a different length.The study presented in this paper is related to our project that deals with incident detection in cash transaction data. The business context of the project is loss prevention in quick-service restaurant operations. In our context, an incident is when a cashier (server, employee) is under-ringing items, voiding items, or performing other intentional operations that cause money losses for the restaurant.Thanks to many years of successful cooperation with our clients in the quick-service restaurant industry, we have accumulated an adequate database of transactions, and we rely on human experts to conduct incident audits. The humanauditor-based loss prevention scheme employed at the moment is the cornerstone of the daily operations of our company. However, there is a pressing need to automate audit tasks.We want to build an automated classifier that detects operations in which a cashier intentionally mishandles t...

show abstract

An Examination of the State-of-the-Art for Multivariate Time Series Classification

Cited by 14 publications

References 22 publications

Scalable Classifier-Agnostic Channel Selection for Multivariate Time Series Classification

Scalable Classifier-Agnostic Channel Selection for Multivariate Time Series Classification

Early Time-Series Classification Algorithms: An Empirical Comparison

Variable-Length Multivariate Time Series Classification Using ROCKET: A Case Study of Incident Detection

Contact Info

Product

Resources

About