Transformation Based Ensembles for Time Series Classification

Abstract

Until recently, the vast majority of data mining time series classification (TSC) research has focused on alternative distance measures for 1-Nearest Neighbour (1-NN) classifiers based on either the raw data, or on compressions or smoothing of the raw data. Despite the extensive evidence in favour of 1-NN classifiers with Euclidean or Dynamic Time Warping distance, there has also been a flurry of recent research publications proposing classification algorithms for TSC. Generally, these classifiers describe different ways of incorporating summary measures in the time domain into more complex classifiers. Our hypothesis is that the easiest way to gain improvement on TSC problems is simply to transform into an alternative data space where the discriminatory features are more easily detected. To test our hypothesis, we perform a range of benchmarking experiments in the time domain, before evaluating nearest neighbour classifiers on data transformed into the power spectrum, the autocorrelation function, and the principal component space. We demonstrate that on some problems there is dramatic improvement in the accuracy of classifiers built on the transformed data over classifiers built in the time domain, but that there is also a wide variance in accuracy for a particular classifier built on different data transforms. To overcome this variability, we propose a simple transformation based ensemble, then demonstrate that it improves performance and reduces the variability of classifiers built in the time domain only. Our advice to a practitioner with a real world TSC problem is to try transforms before developing a complex classifier; it is the easiest way to get a potentially large increase in accuracy, and may provide further insights into the underlying relationships that characterise the problem.

A shapelet transform for time series classification

“…Table 5 shows the accuracy of three classifiers trained with 9 of the 26 datasets. The classifiers are: an SVM built on selected shapelets (results comparable with Table 4), Dynamic Time Warping with 1-NN on the raw data and an ensemble of 1-NN classifiers built on transformations into the power spectrum, autocorrelation function and principle component space (described in [1]). These data sets were selected as they are common to both papers.…”

“…In [1] we show that transforming a TSC problem into an alternative data space prior to classification can provide a greater level of improvement than developing classifier refinements. Hence, we propose a shapelet transform that creates a new classification data set independently of the classifier.…”

“…Despite the evidence in favour of 1-NN classifiers with Euclidean or Dynamic Time Warping (DTW) distance, there has been a spate of recent research proposing alternative approaches. These include shapelets [13,20,19], weighted dynamic time warping [10], support vector machines built on variable intervals [16], tree based ensembles constructed on summary statistics [6], fusion of alternative distance measures [2], and transform-based ensembles [1]. We consider the shapelet approach one of the most promising of these new methods.…”

Extraction of Interpretable Multivariate Patterns for Early Diagnostics

“…A recent study notes "transforming the data is the simplest way of achieving improvement in problems where the discriminating features are based on similarity in change and similarity in shape" [5]. Following this principle, we extract all time series subsequences of different lengths from each dimension of the multivariate time series.…”

DTWDIR: An Enhanced DTW Algorithm for Autistic Child Behaviour Monitoring

“…Many researches depended on reducing the size of datasets [25], [26], [27], and [28] when it applied KNN, as it has to use all the training examples on each test which increases execution time with large datasets [29][30] [31]. Bagnal et al [32] determined that 1-NN with an elastic measure such as DTW is the best approach for smaller data sets, but that as the number of series increases "the accuracy of elastic measures converges with that of Euclidean distance". Euclidian distance calculation is the square of the difference between two points and that means the value's sign are lost therefore we developed DTWDir which considers the absolute distance value and also its sign.…”