Semi-supervised Time Series Classification Model with Self-supervised Learning

Liang, Xi; Yun, Zichao; Liu, Han; Wang, Ruidong; Huang, Xunhua; Fan, Haoyi

doi:10.1016/j.engappai.2022.105331

Cited by 23 publications

(6 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To verify the efficacy of DiffShape in mitigating the issue of lacking labeled samples, we perform classification analyses on time series datasets with only a few labels per class without using unlabeled data. Specifically, we select Supervised, LTS (Grabocka et al 2014), ADSN (Ma et al 2020), SemiTime (Fan et al 2021), SSSTC (Xi et al 2022), MTFC (Wei et al 2023) and TS-TFC (Liu et al 2023b) as baselines. Similar to the previous section, we employ the 12 UCR time series datasets with only 2, 5, and 10 labeled samples per class for analyses.…”

Section: Results On a Few Labeled Time Seriesmentioning

confidence: 99%

“…Baselines. DiffShape is compared with 10 SSC methods, including Supervised, Pseudo-Label (Lee et al 2013), Temporal Ensembling (TE) (Laine and Aila 2016), LPDeepSSL (Iscen et al 2019), MTL (Jawed, Grabocka, and Schmidt-Thieme 2020), TS-TCC (Eldele et al 2021), SemiTime (Fan et al 2021), SSSTC (Xi et al 2022), MTFC (Wei et al 2023), TS-TFC (Liu et al 2023b). Supervised methods only use labeled data for classification training via cross-entropy.…”

Section: Methodsmentioning

confidence: 99%

“…Regarding temporal dependencies, MTL (Jawed, Grabocka, and Schmidt-Thieme 2020) utilizes the sampled subsequence to predict the value of the adjacent next subsequence. Differently, SemiTime (Fan et al 2021) and SSTSC (Xi et al 2022) introduce unsupervised temporal relation prediction losses. Regarding time-frequency information, MTFC (Wei et al 2023) and TS-TFC (Liu et al 2023b) incorporate timeand frequency-domain features of time series to enable the model to learn the class distribution more effectively.…”

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

Diffusion Language-Shapelets for Semi-supervised Time-Series Classification

Liu,

Pei,

Lan

et al. 2024

AAAI

View full text Add to dashboard Cite

Semi-supervised time-series classification could effectively alleviate the issue of lacking labeled data. However, existing approaches usually ignore model interpretability, making it difficult for humans to understand the principles behind the predictions of a model. Shapelets are a set of discriminative subsequences that show high interpretability in time series classification tasks. Shapelet learning-based methods have demonstrated promising classification performance. Unfortunately, without enough labeled data, the shapelets learned by existing methods are often poorly discriminative, and even dissimilar to any subsequence of the original time series. To address this issue, we propose the Diffusion Language-Shapelets model (DiffShape) for semi-supervised time series classification. In DiffShape, a self-supervised diffusion learning mechanism is designed, which uses real subsequences as a condition. This helps to increase the similarity between the learned shapelets and real subsequences by using a large amount of unlabeled data. Furthermore, we introduce a contrastive language-shapelets learning strategy that improves the discriminability of the learned shapelets by incorporating the natural language descriptions of the time series. Experiments have been conducted on the UCR time series archive, and the results reveal that the proposed DiffShape method achieves state-of-the-art performance and exhibits superior interpretability over baselines.

show abstract

Section: Results On a Few Labeled Time Seriesmentioning

confidence: 99%

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Diffusion Language-Shapelets for Semi-supervised Time-Series Classification

Liu,

Pei,

Lan

et al. 2024

AAAI

View full text Add to dashboard Cite

show abstract

“…Jawed et al provide a powerful alternative to supervised signals for feature learning by utilizing unlabeled training data through a prediction task, optimizing the multitasking learning approach and model prediction as a secondary task along with the primary task of classification, with a model that has better performance [14]. Xi et al proposed that the past-anchor-future strategy can extract higher-quality semantic context from unlabeled time series data, and that self-supervised temporal relation learning can effectively assist supervised models [38]. Rezaei et al pre-trained the model on a large unlabeled dataset by inputting the time series features of the sampled packets, and then the learned weights were transferred to a small labeled dataset, which has the same accuracy as a fully supervised method with a large labeled dataset [31].…”

Section: Related Workmentioning

confidence: 99%

Small Sample Time Series Classification Based on Data Augmentation and Semi-supervised Learning

Liu,

Yao,

Wang

et al. 2024

ITC

View full text Add to dashboard Cite

Realistic scenarios produce labeled data and unlabeled data, however, there are significant challenges in labeling time series data. It is imperative to effectively integrate the relationship between labeled and unlabeled data within semi-supervised classification model. This paper presents a novel semi-supervised classification method, namely Data Augmentation-Fast Shapelet Semi-Supervised Classification, which employs a data augmentation module to enhance the diversity of data and improve the generalization ability of the model, as well as a feature fusion module to enhance the semi-supervised network. A conditional generative adversarial network is used to synthesize excellent labeled time series samples to enhance the homogeneous data in the sample space, the fast shapelets method is used to quickly extract the important shape feature vectors in the time series, self-supervised and supervised learning are combined to fully learn the unlabeled and labeled data of the time series dataset. Thejoint loss function combines the loss functions of the two networks to optimize multiple objectives. Reinforcement learning is used to determine the weight coefficients of the joint loss function, at the same time, the reward function is modified to bias the supervisory loss, which improves the classification performance of the model under limited labeled data, and the model can also better achieve the semi-supervised classification task. The proposed method is validated on the UCR benchmark dataset, Electrocardiogram dataset, and Electroencephalogram dataset, the results show that the semi-supervised classification method can perform a more accurate semi-supervised classification of the time series, with an accuracy better than the comparison methods. Meanwhile, we use the plant electrical signal dataset obtained from actual measurements for testing, the visualizationanalysis can clearly show the model role in the semi-supervised classification task, and the experimental results fully demonstrate the effectiveness and applicability of the proposed method.

show abstract

“…Moreover, the previously described methods were used as a reference to develop novel approaches for time-series which were not specifically designed for medical application, but tested on medical datasets. For example, Gorade et al [59] proposed a BYOL-based non-contrastive large scale time-series representation learning approach via simultaneous bootstrapping of low and high frequency input features; Xi et al [60] introduced a self-supervised temporal relation mining module in their work for semi-supervised time series classification; Cheng et al [61] proposed a subjectaware contrastive learning method for biosignal whose core elements were a subject-specific contrastive loss and an adversarial training to promote subject-invariance during pretraining. Ultimately, Zhang et al [62] developed a contrastive pre-training for time series via Time-Frequency consistency.…”

Section: A Pretext Tasksmentioning

confidence: 99%

Applications of Self-Supervised Learning to Biomedical Signals: where are we now

Pup¹,

Atzori²

2023

Preprint

View full text Add to dashboard Cite

<p>Over the last decade, deep learning applications in biomedical research have exploded, demonstrating the ability to often outperform previous machine learning approaches in various tasks. However, training deep learning models requires large amounts of data annotated by experts, whose collection is often time- and cost- prohibitive in the biomedical domain. Self-Supervised Learning (SSL) has emerged as a prominent solution for these problems, as it allows to learn powerful data representations in an unsupervised manner. Despite most applications in biomedical research targeted images, the high amount of recent works targeting biosignals can make it difficult for researchers to have a complete picture of the current situation. The aim of this paper is to outline and clarify the state of the art in the domain. The article briefly summarizes the nature and acquisition modality of biomedical signals, introduces the SSL method, and provides a complete but synthetic overview of the main works applying SSL for the analysis of biomedical signals. The analysis of the scientific literature highlights the importance of SSL, confirming its potential to improve the integration of deep learning into clinical tasks. </p>

show abstract

Semi-supervised Time Series Classification Model with Self-supervised Learning

Cited by 23 publications

References 20 publications

Diffusion Language-Shapelets for Semi-supervised Time-Series Classification

Diffusion Language-Shapelets for Semi-supervised Time-Series Classification

Small Sample Time Series Classification Based on Data Augmentation and Semi-supervised Learning

Applications of Self-Supervised Learning to Biomedical Signals: where are we now

Contact Info

Product

Resources

About