Analysis of Augmentations for Contrastive ECG Representation Learning

Abstract

This paper presents a systematic investigation into the effectiveness of Self-Supervised Learning (SSL) methods for Electrocardiogram (ECG) arrhythmia detection. We begin by conducting a novel distribution analysis on three popular ECGbased arrhythmia datasets: PTB-XL, Chapman, and Ribeiro. To the best of our knowledge, our study is the first to quantify these distributions in this area. We then perform a comprehensive set of experiments using different augmentations and parameters to evaluate the effectiveness of various SSL methods, namely SimCRL, BYOL, and SwAV, for ECG representation learning, where we observe the best performance achieved by SwAV. Furthermore, our analysis shows that SSL methods achieve highly competitive results to those achieved by supervised stateof-the-art methods. To further assess the performance of these methods on both In-Distribution (ID) and Out-of-Distribution (OOD) ECG data, we conduct cross-dataset training and testing experiments. Our comprehensive experiments show almost identical results when comparing ID and OOD schemes, indicating that SSL techniques can learn highly effective representations that generalize well across different OOD datasets. This finding can have major implications for ECG-based arrhythmia detection. Lastly, to further analyze our results, we perform detailed perdisease studies on the performance of the SSL methods on the three datasets.

ML-LMCL: Mutual Learning and Large-Margin Contrastive Learning for Improving ASR Robustness in Spoken Language Understanding

“…Unlike knowledge distillation (Hinton et al, 2015), mutual learning doesn't need a powerful teacher network which is not always available. Mutual learning is first proposed to leverage information from multiple models and allow effective dual knowledge transfer in image processing tasks (Zhang et al, 2018;Zhao et al, 2021) Contrastive learning Contrastive learning aims at learning example representations by minimizing the distance between the positive pairs in the vector space and maximizing the distance between the negative pairs (Saunshi et al, 2019;Liang et al, 2022;Liu et al, 2022a), which is first proposed in the field of computer vision (Chopra et al, 2005;Schroff et al, 2015;Sohn, 2016;Chen et al, 2020a;Wang and Liu, 2021). In the NLP area, contrastive learning is applied to learn sentence embeddings (Giorgi et al, 2021;Yan et al, 2021), translation (Pan et al, 2021;Ye et al, 2022) and summarization Cao and Wang, 2021).…”

Applications of Self-Supervised Learning to Biomedical Signals: where are we now

“…Mehari et al [95] compared standard contrastive learning approaches (e.g. SimCLR, BYOL, SwAV, CPC) to assess their ability to extract good representations from the ECG signal, while Soltanieh et al [97] focused on the efficacy of different data augmentation techniques. Lee et al [94] compared different contrastive Learning approaches as well, but at the same time proposed a variant of VICReg (VIbCReg).…”

CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization

“…Although using negative samples has been effective at text representation learning, e.g., word2vec (Mikolov et al, 2013) and BERT (Devlin et al, 2019), there exist two major challenges for it to succeed in concrete language tasks. First, a suitable training objective is critical to avoid performance degradation (Saunshi et al, 2019). Second, it is nontrivial to construct "natural" samples that mimic the diverse errors made by state-of-the-art systems that vary in words and syntax (Goyal and Durrett, 2021).…”