Self-Supervised Learning with Attention-based Latent Signal Augmentation for Sleep Staging with Limited Labeled Data

Li, Ké; Chen, Lisi; Shang, Shuo; Wang, Haiyan; Liu, Yang; Kalnis, Panos; Yao, Bin

doi:10.24963/ijcai.2022/537

Cited by 7 publications

(11 citation statements)

References 2 publications

(3 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, TS-TCC proposed temporal and contextual contrasting approaches to learn instance-wise representations about the sleep EEG data [8]. In addition, SSLAPP developed a contrastive learning approach with attention-based augmentations in the embedding space to add more positive pairs [26]. Last, CoSleep [14] and SleepECL [27] are yet another two contrastive methods that exploit information, e.g., inter-epoch dependency and frequency domain views, from EEG data to obtain more positive pairs for contrastive learning.…”

Section: Self-supervised Learning For Sleep Stagingmentioning

confidence: 99%

“…We compare the performance of the adopted pretrained SSC models against state-of-the-art self-supervised methods proposed specifically for the sleep stage classification problem. the reported results of SleepDPC [25], CoSleep [14], and SS-LAPP [26] on Sleep-EDF dataset. We compare these methods against existing SSC models with the best-performing SSL method.…”

Section: Comparison With Baselinesmentioning

confidence: 99%

See 1 more Smart Citation

Self-supervised Learning for Label-Efficient Sleep Stage Classification: A Comprehensive Evaluation

Eldele¹,

Ragab²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

The past few years have witnessed a remarkable advance in deep learning for EEG-based sleep stage classification (SSC). However, the success of these models is attributed to possessing a massive amount of labeled data for training, limiting their applicability in real-world scenarios. In such scenarios, sleep labs can generate a massive amount of data, but labeling these data can be expensive and time-consuming. Recently, the selfsupervised learning (SSL) paradigm has shined as one of the most successful techniques to overcome the scarcity of labeled data. In this paper, we evaluate the efficacy of SSL to boost the performance of existing SSC models in the few-labels regime. We conduct a thorough study on three SSC datasets, and we find that fine-tuning the pretrained SSC models with only 5% of labeled data can achieve competitive performance to the supervised training with full labels. Moreover, self-supervised pretraining helps SSC models to be more robust to data imbalance and domain shift problems.

show abstract

Section: Self-supervised Learning For Sleep Stagingmentioning

confidence: 99%

Section: Comparison With Baselinesmentioning

confidence: 99%

Self-supervised Learning for Label-Efficient Sleep Stage Classification: A Comprehensive Evaluation

Eldele¹,

Ragab²,

Chen³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Also, TS-TCC proposed temporal and contextual contrasting approaches to learn instance-wise representations about the sleep EEG data [10]. In addition, SSLAPP developed a contrastive learning approach with attention-based augmentations in the embedding space to add more positive pairs [26]. Last, CoSleep [11] and SleepECL [27] are yet another two contrastive methods that exploit information, e.g., inter-epoch dependency and frequency domain views, from EEG data to obtain more positive pairs for contrastive learning.…”

Section: Self-supervised Learning For Sleep Stagingmentioning

confidence: 99%

“…We compare the performance of the adopted pretrained SSC models against state-of-the-art self-supervised methods proposed specifically for the sleep stage classification problem. reported results of SleepDPC [25] and SSLAPP [26] on Sleep-EDF dataset. To have a fair evaluation, we re-implemented these methods to be consistent with our experimental settings.…”

Section: Comparison With Baselinesmentioning

confidence: 99%

Self-Supervised Learning for Label- Efficient Sleep Stage Classification: A Comprehensive Evaluation

Eldele

Ragab

Chen

et al. 2023

IEEE Trans. Neural Syst. Rehabil. Eng.

View full text Add to dashboard Cite

The past few years have witnessed a remarkable advance in deep learning for EEG-based sleep stage classification (SSC). However, the success of these models is attributed to possessing a massive amount of labeled data for training, limiting their applicability in real-world scenarios. In such scenarios, sleep labs can generate a massive amount of data, but labeling can be expensive and time-consuming. Recently, the self-supervised learning (SSL) paradigm has emerged as one of the most successful techniques to overcome labels' scarcity. In this paper, we evaluate the efficacy of SSL to boost the performance of existing SSC models in the few-labels regime. We conduct a thorough study on three SSC datasets, and we find that finetuning the pretrained SSC models with only 5% of labeled data can achieve competitive performance to the supervised training with full labels. Moreover, self-supervised pretraining helps SSC models to be more robust to data imbalance and domain shift problems.

show abstract

“…In particular, DPC was firstly used to train from scratch two encoders, one for the time view and the other for the frequency view; then, contrastive multiview [142] was used to refine the weights of the two encoders. Finally, Lee et al [140] presented SSLAPP, a completely different approach based on the combination of adversarial representation learning and pairwise representation learning, achieving performances superior to CoSleep, Sleep-DPC, and other fully supervised strategies.…”

Section: B Self-supervised Learning On Eegmentioning

confidence: 99%

Applications of Self-Supervised Learning to Biomedical Signals: where are we now

Pup¹,

Atzori²

2023

Preprint

View full text Add to dashboard Cite

<p>Over the last decade, deep learning applications in biomedical research have exploded, demonstrating the ability to often outperform previous machine learning approaches in various tasks. However, training deep learning models requires large amounts of data annotated by experts, whose collection is often time- and cost- prohibitive in the biomedical domain. Self-Supervised Learning (SSL) has emerged as a prominent solution for these problems, as it allows to learn powerful data representations in an unsupervised manner. Despite most applications in biomedical research targeted images, the high amount of recent works targeting biosignals can make it difficult for researchers to have a complete picture of the current situation. The aim of this paper is to outline and clarify the state of the art in the domain. The article briefly summarizes the nature and acquisition modality of biomedical signals, introduces the SSL method, and provides a complete but synthetic overview of the main works applying SSL for the analysis of biomedical signals. The analysis of the scientific literature highlights the importance of SSL, confirming its potential to improve the integration of deep learning into clinical tasks. </p>

show abstract

Self-Supervised Learning with Attention-based Latent Signal Augmentation for Sleep Staging with Limited Labeled Data

Cited by 7 publications

References 2 publications

Self-supervised Learning for Label-Efficient Sleep Stage Classification: A Comprehensive Evaluation

Self-supervised Learning for Label-Efficient Sleep Stage Classification: A Comprehensive Evaluation

Self-Supervised Learning for Label- Efficient Sleep Stage Classification: A Comprehensive Evaluation

Applications of Self-Supervised Learning to Biomedical Signals: where are we now

Contact Info

Product

Resources

About