End-to-end deep representation learning for time series clustering: a comparative study

Abstract

Time series are ubiquitous in data mining applications. Similar to other types of data, annotations can be challenging to acquire, thus preventing from training Time Series Classification (TSC) models. In this context, clustering methods can be an appropriate alternative as they create homogeneous groups allowing a better analysis of the data structure. Time series clustering has been investigated for many years and multiple approaches have already been proposed. Following the advent of deep learning in computer vision, researchers recently started to study the use of deep clustering to cluster time series data. The existing approaches mostly rely on representation learning (imported from computer vision), which consists of learning a representation of the data and performing the clustering task using this new representation. The goal of this paper is to provide a careful study and an experimental comparison of the existing literature on time series representation learning for deep clustering. In this paper, we went beyond the sole comparison of existing approaches and proposed to decompose deep clustering methods into three main components: (1) network architecture, (2) pretext loss, and (3) clustering loss. We evaluated all combinations of these components (totaling 300 different models) with the objective to study their relative influence on the

Deep Representation Learning for Cluster-Level Time Series Forecasting

“…Like time series forecasting, time series clustering is also an intensively investigated temporal data mining technique [15,21,22]. In general, we can broadly categorize time series forecasting as either distance-or features-based.…”

“…However, the incorporation of DTW into the clustering process often increases the computational complexity of the clustering process [16]. With this understanding, in recent years, researchers have proposed performing clustering by utilizing latent space embedding of neural networks [19,21]. For instance, in [19], the authors proposed deep embedding clustering (DEC), which uses a denoising autoencoder to extract latent features from the input series.…”

A Review and Evaluation of Elastic Distance Functions for Time Series Clustering

“…This is an inherent problem with attempting to evaluate exploratory, unsupervised algorithms by comparing them with what we know to be true a priori: if a clustering simply finds what we already know, its utility is limited. Furthermore, as observed in (Lafabregue et al, 2022), some of the datasets have the same time series but with different labels. We aim to mitigate against these problems by using a large number of problems, but also explore the effect of removing problems where the algorithms perform little better than forming a single cluster.…”

“…(Li et al, 2021), deep learning based clustering algorithms e.g. (Lafabregue et al, 2022) and statistical model based approaches (Caiado et al, 2015) have been proposed for TSCL. These approaches are not the focus of this research.…”

A study of Knowledge Distillation in Fully Convolutional Network for Time Series Classification

“…Following the trend, the last few years have also seen an explosion in the amount of time series data of various modalities such as electrocardiogram [3], power consumption [4], human motion [5] and satellite images [6] among others. Due to its wide range of applications, time series analysis has attracted researchers who developed deep learning-based models for time series clustering [7], averaging [8], forecasting [9] and classification [10]. In this paper, we focus on the task of time series classification (TSC) for which a recent study has shown that DNNs based on 1D temporal convolutions are achieving great performances [10].…”