Shapelets are discriminative subsequences for time series classification. Recently, learning time-series shapelets (LTS) was proposed to learn shapelets by gradient descent directly. Although learning-based shapelet methods achieve better results than previous methods, they still have two shortcomings. First, the learned shapelets are fixed after training and cannot adapt to time series with deformations at the testing phase. Second, the shapelets learned by back-propagation may not be similar to any real subsequences, which is contrary to the original intention of shapelets and reduces model interpretability. In this paper, we propose a novel shapelet learning model called Adversarial Dynamic Shapelet Networks (ADSNs). An adversarial training strategy is employed to prevent the generated shapelets from diverging from the actual subsequences of a time series. During inference, a shapelet generator produces sample-specific shapelets, and a dynamic shapelet transformation uses the generated shapelets to extract discriminative features. Thus, ADSN can dynamically generate shapelets that are similar to the real subsequences rather than having arbitrary shapes. The proposed model has high modeling flexibility while retaining the interpretability of shapelet-based methods. Experiments conducted on extensive time series data sets show that ADSN is state-of-the-art compared to existing shapelet-based methods. The visualization analysis also shows the effectiveness of dynamic shapelet generation and adversarial training.
Time series classification is an essential task in many real-world application domains. As a popular deep learning network, convolutional neural networks have achieved excellent performance in time series classification tasks. The filters of the convolutional neural networks are fixed length and shared by each sample. However, each time series usually has different time scale features. Therefore, convolutional neural networks are not capable of extracting multi-scale features for each sample flexibly. In this paper, we propose dynamic multi-scale convolutional neural network to extract multi-scale feature representations existing in each time series dynamically. Specifically, we design a variable-length filters generator to produce a set of variable-length filters conditioned on the input time series. To make model differentiable, we use the learnable soft masks to control the lengths of variable-length filters. Therefore, the feature representations of different time scales can be captured through the variable-length filters. Then, the max-overtime pooling is used to select the most discriminative local patterns. Finally, the fully connected layer with softmax output is employed to calculate the final probability distribution for each class. Experiments conducted on extensive time series datasets show that our approach can improve the performance of time series classification through the learning of variable-length filters. Furthermore, we demonstrate the effectiveness of dynamically learning variable-length filters for each sample through the visualization analysis. INDEX TERMS Convolutional neural networks, Multi-scale temporal features, Time series classification
a b s t r a c tEcho state networks (ESNs) are randomly connected recurrent neural networks (RNNs) that can be used as a temporal kernel for modeling time series data, and have been successfully applied on time series prediction tasks. Recently, ESNs have been applied to time series classification (TSC) tasks. However, previous ESN-based classifiers involve either training the model by predicting the next item of a sequence, or predicting the class label at each time step. The former is essentially a predictive model adapted from time series prediction work, rather than a model designed specifically for the classification task. The latter approach only considers local patterns at each time step and then averages over the classifications. Hence, rather than selecting the most discriminating sections of the time series, this approach will incorporate non-discriminative information into the classification, reducing accuracy. In this paper, we propose a novel end-to-end framework called the Echo Memory Network (EMN) in which the time series dynamics and multi-scale discriminative features are efficiently learned from an unrolled echo memory using multi-scale convolution and max-over-time pooling. First, the time series data are projected into the high dimensional nonlinear space of the reservoir and the echo states are collected into the echo memory matrix, followed by a single multi-scale convolutional layer to extract multi-scale features from the echo memory matrix. Max-over-time pooling is used to maintain temporal invariance and select the most important local patterns. Finally, a fully-connected hidden layer feeds into a softmax layer for classification. This architecture is applied to both time series classification and human action recognition datasets. For the human action recognition datasets, we divide the action data into five different components of the human body, and propose two spatial information fusion strategies to integrate the spatial information over them. With one training-free recurrent layer and only one layer of convolution, the EMN is a very efficient end-to-end model, and ranks first in overall classification ability on 55 TSC benchmark datasets and four 3D skeleton-based human action recognition tasks.
The Echo State Networks (ESNs) is an efficient recurrent neural network consisting of a randomly generated reservoir (a large number of neurons with sparse random recurrent connections) and a trainable linear layer. It has received widespread attention for its simplicity and effectiveness, especially for time series prediction tasks. However, there is no explicit mechanism in ESNs to capture the inherent multiscale characteristics of time series. To this end, we propose a model consisting of multi-reservoir structure named long-short term echo state networks (LS-ESNs) to capture the multi-scale temporal characteristics of time series. Specifically, LS-ESNs consists of three independent reservoirs, and each reservoir has recurrent connections of a specific timescale to model the temporal dependencies of time series. The multi-scale echo states are then collected from each reservoir and concatenated together. Finally, the concatenated echo states representations are fed to the linear regression layer to obtain the results. Experiments on two time series prediction benchmark data sets and a real-world power load data sets demonstrate the effectiveness of the proposed LS-ESNs. INDEX TERMS Time series prediction, echo state networks (ESNs), multi-scale temporal dependencies, long short term reservoir.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.